generator

package module
v0.0.0-...-8959c62 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 8, 2021 License: MIT-0 Imports: 2 Imported by: 5

README

Go Config Generator For Fluentd And Fluentbit

A Go Library for programmatically generating Fluentd and Fluent Bit Configuration.

For example usage, see the unit test file.

The library exports the following interface:

type FluentConfig interface {
	// AddInput adds an input/source directive
	AddInput(name string, tag string, options map[string]string) FluentConfig

	// AddIncludeFilter adds a regex filter that only allows logs which match the regex
	AddIncludeFilter(regex string, key string, tag string) FluentConfig

	// AddExcludeFilter adds a regex filter that only allows logs which do not match the regex
	AddExcludeFilter(regex string, key string, tag string) FluentConfig

	// AddFieldToRecord adds a key value pair to log records
	AddFieldToRecord(key string, value string, tag string) FluentConfig

	// AddExternalConfig adds an @INCLUDE directive to reference an external config file
	AddExternalConfig(filePath string, position IncludePosition) FluentConfig

	// AddOutput adds an output/log destination
	AddOutput(name string, tag string, options map[string]string) FluentConfig

	// AddCustom adds custom configuration, needed for a custom template
	AddCustom(conf interface{}) FluentConfig

	// WithFluentdTemplate sets a custom Fluentd template
	WithFluentdTemplate(template string) FluentConfig

	// WithFluentBitTemplate sets a custom FluentBit template
	WithFluentBitTemplate(template string) FluentConfig

	// WriteFluentdConfig outputs the config in Fluentd syntax
	WriteFluentdConfig(wr io.Writer) error

	// WriteFluentBitConfig outputs the config in Fluent Bit syntax
	WriteFluentBitConfig(wr io.Writer) error
}

The library uses Fluent Bit terminology in the interface; AddInput adds a 'source' in Fluentd syntax, and the name argument is used as the plugin @type.

License Summary

This sample code is made available under the MIT-0 license. See the LICENSE file.

Security disclosures

If you think you’ve found a potential security issue, please do not post it in the Issues. Instead, please follow the instructions here or email AWS security directly at aws-security@amazon.com.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FluentConfig

type FluentConfig interface {
	AddInput(name string, tag string, options map[string]string) FluentConfig
	AddIncludeFilter(regex string, key string, tag string) FluentConfig
	AddExcludeFilter(regex string, key string, tag string) FluentConfig
	AddFieldToRecord(key string, value string, tag string) FluentConfig
	AddExternalConfig(filePath string, position IncludePosition) FluentConfig
	AddOutput(name string, tag string, options map[string]string) FluentConfig
	AddCustom(conf interface{}) FluentConfig
	WithFluentdTemplate(template string) FluentConfig
	WithFluentBitTemplate(template string) FluentConfig
	WriteFluentdConfig(wr io.Writer) error
	WriteFluentBitConfig(wr io.Writer) error
}

FluentConfig is the interface for generating fluentd and fluentbit config

func New

func New() FluentConfig

New Creates a new Fluent Config generater

type FluentConfigGenerator

type FluentConfigGenerator struct {
	Inputs                    []LogPipe
	ModifyRecords             map[string]RecordModifier
	IncludeFilters            []RegexFilter
	ExcludeFilters            []RegexFilter
	IncludeConfigHeadOfFile   []string
	IncludeConfigAfterInputs  []string
	IncludeConfigAfterFilters []string
	IncludeConfigEndOfFile    []string
	Outputs                   []LogPipe
	Custom                    interface{}
	// contains filtered or unexported fields
}

FluentConfigGenerator implements FluentConfig The templating fields must be public, so that the template library can use them when the templates are executed

func (*FluentConfigGenerator) AddCustom

func (config *FluentConfigGenerator) AddCustom(conf interface{}) FluentConfig

AddCustom adds custom configuration, needed for a custom template

func (*FluentConfigGenerator) AddExcludeFilter

func (config *FluentConfigGenerator) AddExcludeFilter(regex string, key string, tag string) FluentConfig

AddExcludeFilter adds a regex filter that only allows logs which do not match the regex

func (*FluentConfigGenerator) AddExternalConfig

func (config *FluentConfigGenerator) AddExternalConfig(filePath string, position IncludePosition) FluentConfig

AddExternalConfig adds an @INCLUDE directive to reference an external config file

func (*FluentConfigGenerator) AddFieldToRecord

func (config *FluentConfigGenerator) AddFieldToRecord(key string, value string, tag string) FluentConfig

AddFieldToRecord adds a key value pair to log records

func (*FluentConfigGenerator) AddIncludeFilter

func (config *FluentConfigGenerator) AddIncludeFilter(regex string, key string, tag string) FluentConfig

AddIncludeFilter adds a regex filter that only allows logs which match the regex

func (*FluentConfigGenerator) AddInput

func (config *FluentConfigGenerator) AddInput(name string, tag string, options map[string]string) FluentConfig

AddInput add an input/source directive

func (*FluentConfigGenerator) AddOutput

func (config *FluentConfigGenerator) AddOutput(name string, tag string, options map[string]string) FluentConfig

AddOutput adds an output/log destination

func (*FluentConfigGenerator) WithFluentBitTemplate

func (config *FluentConfigGenerator) WithFluentBitTemplate(template string) FluentConfig

WithFluentBitTemplate sets a custom FluentBit template

func (*FluentConfigGenerator) WithFluentdTemplate

func (config *FluentConfigGenerator) WithFluentdTemplate(template string) FluentConfig

WithFluentdTemplate sets a custom Fluentd template

func (*FluentConfigGenerator) WriteFluentBitConfig

func (config *FluentConfigGenerator) WriteFluentBitConfig(wr io.Writer) error

WriteFluentBitConfig outputs the config in Fluent Bit syntax

func (*FluentConfigGenerator) WriteFluentdConfig

func (config *FluentConfigGenerator) WriteFluentdConfig(wr io.Writer) error

WriteFluentdConfig outputs the config in Fluentd syntax

type IncludePosition

type IncludePosition int

IncludePosition is a type to represent the position of an @INCLUDE directive to include external config

const (
	// HeadOfFile: The external config is included as the first directive in the config file
	HeadOfFile IncludePosition = iota
	// AfterInputs: The external config is included after the input/source directives
	AfterInputs
	// AfterFilters: The external config is included after the filter directives
	AfterFilters
	// EndOfFile: The external config is included at the end of the file
	EndOfFile
)

This library assumes that the config is structured as follows: 1. Sources/inputs 2. Filters 3. Outputs

type LogPipe

type LogPipe struct {
	Name    string
	Tag     string
	Options map[string]string
}

LogPipe can represent an input or an output plugin

type RecordModifier

type RecordModifier struct {
	NewFields map[string]string
}

RecordModifier tracks new fields added to log records

type RegexFilter

type RegexFilter struct {
	Regex string
	Key   string
	Tag   string
}

RegexFilter represents a filter for logs

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL