logparser

package module
v1.1.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 29, 2024 License: Apache-2.0 Imports: 11 Imported by: 6

README

Logparser

Logparser is a log parsing library used by node-agent and aws-agent to extract metrics from unstructured logs. Also in this repository, you can find a CLI tool that calculates a log summary from stdin.

Run

cat some.log | docker run -i --rm ghcr.io/coroot/logparser

Sample output

▇                      12 ( 0%) - ERROR [LearnerHandler-/10.10.34.11:52225:LearnerHandler@562] - Unexpected exception causing shutdown while sock still open
▇                       1 ( 0%) - ERROR [CommitProcessor:1:NIOServerCnxn@180] - Unexpected Exception:
▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 314 (23%) - WARN [SendWorker:188978561024:QuorumCnxManager$SendWorker@679] - Interrupted while waiting for message on queue
▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇   291 (21%) - WARN [RecvWorker:188978561024:QuorumCnxManager$RecvWorker@762] - Connection broken for id 188978561024, my id = 1, error =
▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇     266 (19%) - WARN [RecvWorker:188978561024:QuorumCnxManager$RecvWorker@765] - Interrupting SendWorker
▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇     262 (19%) - WARN [SendWorker:188978561024:QuorumCnxManager$SendWorker@688] - Send worker leaving thread
▇▇▇▇▇▇                 86 ( 6%) - WARN [WorkerSender[myid=1]:QuorumCnxManager@368] - Cannot open channel to 2 at election address /10.10.34.12:3888
▇▇▇                    39 ( 2%) - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@793] - Connection request from old client /10.10.34.19:33442; will be dropped if server is in r-o mode
▇▇▇                    37 ( 2%) - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@349] - caught end of stream exception
▇▇                     19 ( 1%) - WARN [LearnerHandler-/10.10.34.12:35276:LearnerHandler@575] - ******* GOODBYE /10.10.34.12:35276 ********
▇                       3 ( 0%) - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@354] - Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
▇                       1 ( 0%) - WARN [LearnerHandler-/10.10.34.13:42241:Leader@576] - First is 0x0

1998 messages processed in 0.137 seconds:
  error: 13
  warning: 1318
  info: 667

License

Logparser is licensed under the Apache License, Version 2.0.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type CriDecoder 

type CriDecoder struct{}

func (CriDecoder) Decode 

func (d CriDecoder) Decode(src string) (string, error)

type Decoder 

type Decoder interface {
	Decode(string) (string, error)
}

type DockerJsonDecoder 

type DockerJsonDecoder struct{}

func (DockerJsonDecoder) Decode 

func (d DockerJsonDecoder) Decode(src string) (string, error)

type DockerLogJson 

type DockerLogJson struct {
	Log string
}

type Level 

type Level int
const (
	LevelUnknown Level = iota
	LevelCritical
	LevelError
	LevelWarning
	LevelInfo
	LevelDebug
)

func GuessLevel 

func GuessLevel(line string) Level

func LevelByPriority 

func LevelByPriority(priority string) Level

func (Level) String 

func (l Level) String() string

type LogCounter 

type LogCounter struct {
	Level    Level
	Hash     string
	Sample   string
	Messages int
}

type LogEntry 

type LogEntry struct {
	Timestamp time.Time
	Content   string
	Level     Level
}

type Message 

type Message struct {
	Timestamp time.Time
	Content   string
	Level     Level
}

type MultilineCollector 

type MultilineCollector struct {
	Messages chan Message
	// contains filtered or unexported fields
}

func NewMultilineCollector 

func NewMultilineCollector(ctx context.Context, timeout time.Duration, limit int) *MultilineCollector

func (*MultilineCollector) Add 

func (m *MultilineCollector) Add(entry LogEntry)

type OnMsgCallbackF added in v1.0.7 

type OnMsgCallbackF func(ts time.Time, level Level, patternHash string, msg string)

type Parser 

type Parser struct {
	// contains filtered or unexported fields
}

func NewParser 

func NewParser(ch <-chan LogEntry, decoder Decoder, onMsgCallback OnMsgCallbackF) *Parser

func (*Parser) GetCounters 

func (p *Parser) GetCounters() []LogCounter

func (*Parser) Stop 

func (p *Parser) Stop()

type Pattern 

type Pattern struct {
	// contains filtered or unexported fields
}

func NewPattern 

func NewPattern(input string) *Pattern

func (*Pattern) Hash 

func (p *Pattern) Hash() string

func (*Pattern) String 

func (p *Pattern) String() string

func (*Pattern) WeakEqual 

func (p *Pattern) WeakEqual(other *Pattern) bool

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.