parser

package
v0.1.48 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 5, 2021 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var KnownDataFormats = map[DataFormat]parserConstructorFunc{
	FmtPure: func(cfg *Config) (Parser, error) {
		return NewPureParser(cfg.File, cfg.MaxRecSizeBytes)
	},
	FmtText: func(cfg *Config) (Parser, error) {
		return NewLineParser(cfg.File, date.NewDefaultParser(cfg.DateFmts...), cfg.MaxRecSizeBytes)
	},
	FmtK8Json: func(cfg *Config) (Parser, error) {
		return NewK8sJsonParser(cfg.File, cfg.MaxRecSizeBytes)
	},
	FmtLogfmt: func(cfg *Config) (Parser, error) {
		return NewLogfmtParser(cfg.File, cfg.MaxRecSizeBytes, cfg.FieldMap)
	},
}

Functions

func NewLineParser

func NewLineParser(fileName string, dateParser date.Parser, maxRecSize int) (*lineParser, error)

func NewLogfmtParser added in v0.1.44

func NewLogfmtParser(fileName string, maxRecSize int, fieldMap map[string]string) (*logfmtParser, error)

func NewPureParser added in v0.1.1

func NewPureParser(fileName string, maxRecSize int) (*pureParser, error)

NewPureParser creates new instance of pureParser

Types

type Config

type Config struct {
	File            string
	MaxRecSizeBytes int
	DataFmt         DataFormat
	DateFmts        []string
	FieldMap        map[string]string
}

func (*Config) Check

func (c *Config) Check() error

type DataFormat

type DataFormat string
const (
	FmtPure   DataFormat = "pure"
	FmtText   DataFormat = "text"
	FmtK8Json DataFormat = "k8json"
	FmtLogfmt DataFormat = "logfmt"
)

func ToDataFormat added in v0.1.43

func ToDataFormat(str string) (DataFormat, error)

type K8sJsonLogParser

type K8sJsonLogParser struct {
	// contains filtered or unexported fields
}

K8sJsonLogParser implements parser.Parser, for reading lines(strings with \n separation) from a text file, treating every line as a k8s json log message (fixed fields). The parser doesn't pay much attention to parsing dates, as they are presented as time.Time object in case of k8s log and in such a case JSON deserialization should take care of it. Additionally to message and date the parser also saves some additional metadata like 'pipe' (stdout/strerr), which is provided in k8s logs, the result of parsing is model.Record.

func NewK8sJsonParser

func NewK8sJsonParser(fileName string, maxRecSize int) (*K8sJsonLogParser, error)

func (*K8sJsonLogParser) Close

func (jp *K8sJsonLogParser) Close() error

func (*K8sJsonLogParser) GetStats

func (jp *K8sJsonLogParser) GetStats() *Stats

func (*K8sJsonLogParser) GetStreamPos

func (jp *K8sJsonLogParser) GetStreamPos() int64

func (*K8sJsonLogParser) NextRecord

func (jp *K8sJsonLogParser) NextRecord(ctx context.Context) (*model.Record, error)

func (*K8sJsonLogParser) SetStreamPos

func (jp *K8sJsonLogParser) SetStreamPos(pos int64) error

type K8sJsonLogRec

type K8sJsonLogRec struct {
	Log    string    `json:"log"`
	Stream string    `json:"stream"`
	Time   time.Time `json:"time"`
}

K8sJsonLogRec defines format of k8s json log line

type LogfmtJsonLogRec added in v0.1.44

type LogfmtJsonLogRec struct {
	Log    string    `json:"log"`
	Stream string    `json:"stream"`
	Time   time.Time `json:"time"`
}

K8sJsonLogRec defines format of k8s json log line

type Parser

type Parser interface {
	io.Closer

	// NextRecord parses next record. It returns error if it could not parse
	// a record from the pipe. io.EOF is returned if no new records found, but
	// end is reached.
	NextRecord(ctx context.Context) (*model.Record, error)

	// SetStreamPos specifies the pipe position for the next record read
	SetStreamPos(pos int64) error

	// GetStreamPos returns position of the last successfully (error was nil)
	// returned record by nextRecord(). If nextRecord() returned non-nil
	// error the getStreamPos() returned value is not relevant and should
	// not be used as a valid pipe position.
	GetStreamPos() int64

	// GetStat returns the parser statistic
	GetStats() *Stats
}

Parser provides an interface for retrieving records from a data-pipe. Implementations of the interface are supposed to be initialized by the pipe (io.Reader)

func NewParser

func NewParser(cfg *Config) (Parser, error)

NewParser creates new parser based on the type the parser specified in cfg.DataFmt.

type Stats

type Stats struct {
	DataFormat DataFormat
	FileStats  *fileStats
	FmtStats   *fmtStats // for text parsers only...
}

Stats struct contains information about the parser statistics

func (*Stats) Update

func (s *Stats) Update(dFmt *date.Format)

Update adds 1 hit to the date format statistic

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL