config

package module
v0.5.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 17, 2021 License: Apache-2.0 Imports: 14 Imported by: 2

README

logstash-config : parser and abstract syntax tree for Logstash config files

Test Status Go Report Card
GoDoc License

Overview

The Go package config provides a ready to use parser for Logstash (github) configuration files.

The basis of the grammar for the parsing of the Logstash configuration format is the original Logstash Treetop grammar which could be used with only minor changes.

logstash-config uses pigeon to generate the parser from the PEG (parser expression grammar). Special thanks to Martin Angers (mna).

This package is currently under development, no API guaranties.

Install

go get -t github.com/breml/logstash-config/...

Usage

mustache

mustache is a command line tool that allows to syntax check, lint and format Logstash configuration files. The name of the tool is inspired by the original Logstash Logo (wooden character with an eye-catching mustache).

The check command verifies the syntax of Logstash configuration files:

mustache check file.conf

The lint command checks for problems in Logstash configuration files.

The following checks are performed:

  • Valid Logstash configuration file syntax
  • No comments in exceptional places (these are comments, that are valid by the Logstash configuration file syntax, but but are located in exceptional or uncommon locations)
  • Precence of an id attribute for each plugin in the Logstash configuration

If the --auto-fix-id flag is passed, each plugin gets automatically an ID. Be aware, that this potentially reformats the Logstash configuration files.

mustache lint --auto-fix-id file.conf

With the format command, mustache returns the provided configuration files in a standardized format (indentation, location of comments). By default, the reformatted file is print to standard out. If the flag --write-to-source is provided, the Logstash config files are reformatted in place.

mustache format --write-to-source file.conf

Use the --help flag to get more information about the usage of the tool.

Rebuild parser

  1. Get and install pigeon.
  2. Run go generate in the root directory of this repository.

Author

Copyright 2017-2021 by Lucas Bremgartner (breml)

License

Apache 2.0

Documentation

Overview

Package config provides a ready to use parser for Logstash (https://github.com/elastic/logstash) configuration files.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetFarthestFailure 

func GetFarthestFailure() (string, bool)

GetFarthestFailure returns the farthest position where the parser had a parse error. The farthest position is normally close to the real source for the error.

func Parse 

func Parse(filename string, b []byte, opts ...Option) (interface{}, error)

Parse parses the data from b using filename as information in the error messages.

func ParseFile 

func ParseFile(filename string, opts ...Option) (i interface{}, err error)

ParseFile parses the file identified by filename.

func ParseReader 

func ParseReader(filename string, r io.Reader, opts ...Option) (interface{}, error)

ParseReader parses the data from r using filename as information in the error messages.

Example 
logstashConfig := `filter {
    mutate {
      add_tag => [
        "tag"
      ]
    }
  }`
got, err := ParseReader("example.conf", strings.NewReader(logstashConfig))
if err != nil {
	log.Fatalf("Parse error: %s\n", err)
}

Output:

filter {
  mutate {
    add_tag => [
      "tag"
    ]
  }
}

Types

type Cloner added in v0.4.1 

type Cloner interface {
	Clone() interface{}
}

Cloner is implemented by any value that has a Clone method, which returns a copy of the value. This is mainly used for types which are not passed by value (e.g map, slice, chan) or structs that contain such types.

This is used in conjunction with the global state feature to create proper copies of the state to allow the parser to properly restore the state in the case of backtracking.

type Option 

type Option func(*parser) Option

Option is a function that can set an option on the parser. It returns the previous setting as an Option.

func AllowInvalidUTF8 

func AllowInvalidUTF8(b bool) Option

AllowInvalidUTF8 creates an Option to allow invalid UTF-8 bytes. Every invalid UTF-8 byte is treated as a utf8.RuneError (U+FFFD) by character class matchers and is matched by the any matcher. The returned matched value, c.text and c.offset are NOT affected.

The default is false.

func Debug added in v0.4.1 

func Debug(b bool) Option

Debug creates an Option to set the debug flag to b. When set to true, debugging information is printed to stdout while parsing.

The default is false.

func Entrypoint 

func Entrypoint(ruleName string) Option

Entrypoint creates an Option to set the rule name to use as entrypoint. The rule name must have been specified in the -alternate-entrypoints if generating the parser with the -optimize-grammar flag, otherwise it may have been optimized out. Passing an empty string sets the entrypoint to the first rule in the grammar.

The default is to start parsing at the first rule in the grammar.

func ExceptionalCommentsWarning added in v0.4.1 

func ExceptionalCommentsWarning(enabled bool) Option

The ExceptionalCommentsWarning option controls if the parser does emit warnings for comments in exceptional locations (true). Otherwise these comments are silently ignored and are lost, during the parsing of the configuration (default).

func GlobalStore 

func GlobalStore(key string, value interface{}) Option

GlobalStore creates an Option to set a key to a certain value in the globalStore.

func IgnoreComments added in v0.4.1 

func IgnoreComments(enabled bool) Option

The IgnoreComments option controls if the parse ignores comments (true). Otherwise the comments are parsed and returned, if the configuration is returned (default).

func InitState added in v0.4.1 

func InitState(key string, value interface{}) Option

InitState creates an Option to set a key to a certain value in the global "state" store.

func MaxExpressions 

func MaxExpressions(maxExprCnt uint64) Option

MaxExpressions creates an Option to stop parsing after the provided number of expressions have been parsed, if the value is 0 then the parser will parse for as many steps as needed (possibly an infinite number).

The default for maxExprCnt is 0.

func Memoize added in v0.4.1 

func Memoize(b bool) Option

Memoize creates an Option to set the memoize flag to b. When set to true, the parser will cache all results so each expression is evaluated only once. This guarantees linear parsing time even for pathological cases, at the expense of more memory and slower times for typical cases.

The default is false.

func Recover 

func Recover(b bool) Option

Recover creates an Option to set the recover flag to b. When set to true, this causes the parser to recover from panics and convert it to an error. Setting it to false can be useful while debugging to access the full stack trace.

The default is true.

func Statistics added in v0.4.1 

func Statistics(stats *Stats, choiceNoMatch string) Option

Statistics adds a user provided Stats struct to the parser to allow the user to process the results after the parsing has finished. Also the key for the "no match" counter is set.

Example usage: 

input := "input"
stats := Stats{}
_, err := Parse("input-file", []byte(input), Statistics(&stats, "no match"))
if err != nil {
    log.Panicln(err)
}
b, err := json.MarshalIndent(stats.ChoiceAltCnt, "", "  ")
if err != nil {
    log.Panicln(err)
}
fmt.Println(string(b))

type Stats 

type Stats struct {
	// ExprCnt counts the number of expressions processed during parsing
	// This value is compared to the maximum number of expressions allowed
	// (set by the MaxExpressions option).
	ExprCnt uint64

	// ChoiceAltCnt is used to count for each ordered choice expression,
	// which alternative is used how may times.
	// These numbers allow to optimize the order of the ordered choice expression
	// to increase the performance of the parser
	//
	// The outer key of ChoiceAltCnt is composed of the name of the rule as well
	// as the line and the column of the ordered choice.
	// The inner key of ChoiceAltCnt is the number (one-based) of the matching alternative.
	// For each alternative the number of matches are counted. If an ordered choice does not
	// match, a special counter is incremented. The name of this counter is set with
	// the parser option Statistics.
	// For an alternative to be included in ChoiceAltCnt, it has to match at least once.
	ChoiceAltCnt map[string]map[string]int
}

Stats stores some statistics, gathered during parsing

Source Files

View all Source files

Directories

Path Synopsis
ast
Package ast declares the types used to represent syntax trees for Logstash configurations.
 Package ast declares the types used to represent syntax trees for Logstash configurations.
astutil
cmd
mustache
internal
app
app/check
app/format
app/lint
format

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.