log

package
v0.0.0-...-fc9d343 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 3, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Log Chunk

A log chunk is a storage abstraction representing a sequential, continuous section of a single log. Logs can be coherently represented by a set of ordered chunks.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EmptyIterator

func EmptyIterator() *emptyIterator

EmptyIterator returns a convenience log iterator with no data.

func NewLogServiceV0

func NewLogServiceV0(bucket pail.Bucket) *logServiceV0

NewLogServiceV0 returns a new V0 Evergreen log service.

func NewSender

func NewSender(ctx context.Context, name string, svc LogService, opts SenderOptions) (send.Sender, error)

NewSender creates a new log sender backed by an Evergreen log service.

func StreamFromLogIterator

func StreamFromLogIterator(it LogIterator) chan LogLine

StreamFromLogIterator streams log lines from the given iterator to the returned channel. It is the responsibility of the caller to close the iterator.

Types

type GetOptions

type GetOptions struct {
	// LogNames are the names of the logs to fetch and merge, prefixes may
	// be specified. At least one name must be specified.
	//
	// Log lines from multiple logs are always merged in a deterministic
	// order by timestamp.
	LogNames []string
	// Start is the start time (inclusive) of the time range filter,
	// represented as a Unix timestamp in nanoseconds. Defaults to
	// unbounded unless DefaultTimeRangeOfFirstLog is set to true.
	Start *int64
	// End is the end time (inclusive) of the time range filter,
	// represented as a Unix timestamp in nanoseconds. Defaults to
	// unbounded unless DefaultTimeRangeOfFirstLog is set to true.
	End *int64
	// DefaultTimeRangeOfFirstLog defaults the start and end time of the
	// time range filter to the first and last timestamp, respectively, of
	// the first specified log in LogNames.
	DefaultTimeRangeOfFirstLog bool
	// LineLimit limits the number of lines read from the log. Ignored if
	// less than or equal to 0.
	LineLimit int
	// TailN is the number of lines to read from the tail of the log.
	// Ignored if less than or equal to 0.
	TailN int
}

GetOptions represents the arguments for fetching Evergreen logs.

type LineParser

type LineParser func(string) (LogLine, error)

LineParser functions parse a raw log line into the service representation of a log line for uniform ingestion of logs by the Evergreen log sender. Parsers need not set the log name or, in most cases, the priority.

type LogIterator

type LogIterator interface {
	// Next returns true if the iterator has not yet been exhausted or
	// closed, false otherwise.
	Next() bool
	// Item returns the current log line held by the iterator.
	Item() LogLine
	// Exhausted returns true if the iterator has not yet been exhausted,
	// regardless if it has been closed or not.
	Exhausted() bool
	// Err returns any errors that are captured by the iterator.
	Err() error
	// Close closes the iterator. This function should be called once the
	// iterator is no longer needed.
	Close() error
}

LogIterator is an interface that enables iterating over lines of Evergreen logs.

type LogIteratorReader

type LogIteratorReader struct {
	// contains filtered or unexported fields
}

LogIteratorReader implements the io.Reader interface for log lines with additional utility functionality.

func NewLogIteratorReader

func NewLogIteratorReader(it LogIterator, opts LogIteratorReaderOptions) *LogIteratorReader

NewLogIteratorReader returns a reader that reads the log lines from the iterator with the given options. The reader will attempt to close the iterator.

func (*LogIteratorReader) NextTimestamp

func (r *LogIteratorReader) NextTimestamp() *int64

NextTimestamp returns the Unix nanosecond timestamp of the next unread line, if applicable. This is mostly used for pagination as the next unread line's timestamp is the start key of the next page—call after the reader successfully returns an io.EOF error for this use case.

func (*LogIteratorReader) Read

func (r *LogIteratorReader) Read(p []byte) (int, error)

type LogIteratorReaderOptions

type LogIteratorReaderOptions struct {
	// PrintTime, when true, prints the timestamp of each log line along
	// with the line in the following format:
	//		[2006/01/02 15:04:05.000] This is a log line.
	PrintTime bool
	// TimeZone is the time zone to use when printing the timestamp of each
	// each log line. Optional. Defaults to UTC.
	TimeZone *time.Location
	// PrintPriority, when true, prints the priority of each log line along
	// with the line in the following format:
	//		[P: 30] This is a log line.
	// If PrintTime is also set to true, priority will be printed first:
	//		[P:100] [2006/01/02 15:04:05.000] This is a log line.
	PrintPriority bool
	// SoftSizeLimit assists with pagination of long logs. When set the
	// reader will attempt to read as close to the limit as possible while
	// also reading every line for each timestamp reached. Optional.
	SoftSizeLimit int
}

LogIteratorReaderOptions describes the options for creating a new LogIteratorReader.

type LogLine

type LogLine struct {
	LogName   string
	Priority  level.Priority
	Timestamp int64
	Data      string
}

LogLine represents a single line in an Evergreen log.

type LogService

type LogService interface {
	// Get returns a log iterator with the given options.
	Get(context.Context, GetOptions) (LogIterator, error)
	// Append appends given lines to the specified log.
	Append(context.Context, string, []LogLine) error
}

LogService is a simple abstraction bridging the logical representation of an Evergreen log with its physical storage. Namely, it supports writing and retrieving logs directly to and from an underlying storage service. Any more sophisticated business logic pertaining to log handling (e.g., collection, logical organization, retrieval patterns) should be implemented on top of this interface in separate layers of the application.

type SenderOptions

type SenderOptions struct {
	// LogName is the identifying name of the log to use when persisting
	// data.
	LogName string
	// Parse is the function for parsing raw log lines collected by the
	// sender.
	// The injectable line parser allows the sender to be agnostic to the
	// raw log line formats it ingests.
	// Defaults to a basic line parser that adds the raw string as the log
	// line data field.
	Parse LineParser
	// Local is the sender for "fallback" operations and to collect any
	// logger error output.
	Local send.Sender
	// MaxBufferSize is the maximum number of bytes to buffer before
	// persisting log data. Defaults to 10MB.
	MaxBufferSize int
	// FlushInterval is time interval at which to flush log lines,
	// regardless of whether the max buffer size has been reached. A flush
	// interval equal to 0 will disable timed flushes.
	FlushInterval time.Duration
}

SenderOptions support the use and creation of an Evergreen log sender.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL