scanner

package
v0.1.48 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 5, 2021 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config

type Config struct {
	IncludePaths           []string
	ExcludeMatchers        []string
	SyncWorkersIntervalSec int
	StateStoreIntervalSec  int
	RecordMaxSizeBytes     int
	EventMaxRecords        int
	StopWhenNoData         bool
	Schemas                []*SchemaConfig
}

func NewDefaultConfig

func NewDefaultConfig() *Config

func (*Config) Apply

func (c *Config) Apply(other *Config)

func (*Config) Check

func (c *Config) Check() error

func (*Config) String

func (c *Config) String() string

type Meta

type Meta struct {
	// Tags contains map of key-value pairs
	Tags map[string]string

	// Fields contains map of key-value pairs
	Fields map[string]string
}

Meta descirbes Tags and Fields assignment. Tags and Fields are maps, where key is the value of Tag or Field and the value is a template for the value. The template can contains variables placed in curly braces e.g. "var1={var1} or var2={var2}"...

type Scanner

type Scanner struct {
	// contains filtered or unexported fields
}

Scanner struct manages data collecting jobs for sending the data into Logrange server.

func NewScanner

func NewScanner(cfg *Config, storage storage.Storage) (*Scanner, error)

NewScanner creates a new file scanner according the config provided. It returns the pointer to Scanner object, or an error, if any.

func (*Scanner) Run

func (s *Scanner) Run(ctx context.Context, events chan<- *model.Event) error

Run starts internal go routines which will scan folders and send the obtained data asynchronously

func (*Scanner) WaitAllJobsDone added in v0.1.1

func (s *Scanner) WaitAllJobsDone() error

WaitAllJobsDone closes the Scanner object

type SchemaConfig

type SchemaConfig struct {
	// PathMatcher contains a regexp for matching file names. It can contain
	// named groups, which values can be assigned for tags and fields (see Meta field)
	PathMatcher string
	DataFormat  parser.DataFormat
	DateFormats []string

	Meta Meta
}

SchemaConfig describes the configuration for parsing of the data

func (*SchemaConfig) Check

func (sc *SchemaConfig) Check() error

func (*SchemaConfig) String

func (sc *SchemaConfig) String() string

type Stats

type Stats struct {
	Workers []*stats
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL