collector

package
v0.0.0-...-51f9457 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 9, 2021 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package collector implements the LogDog Collector daemon's log parsing and registration logic.

The LogDog Collector is responsible for ingesting logs from the Butler sent via transport, stashing them in intermediate storage, and registering them with the LogDog Coordinator service.

Index

Constants

View Source
const (
	// DefaultMaxMessageWorkers is the default number of concurrent worker
	// goroutones to employ for a single message.
	DefaultMaxMessageWorkers = 4
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Collector

type Collector struct {
	// Coordinator is used to interface with the Coordinator client.
	//
	// On production systems, this should wrapped with a caching client (see
	// the stateCache sub-package) to avoid overwhelming the server.
	Coordinator coordinator.Coordinator

	// Storage is the intermediate storage instance to use.
	Storage storage.Storage

	// StreamStateCacheExpire is the maximum amount of time that a cached stream
	// state entry is valid. If zero, DefaultStreamStateCacheExpire will be used.
	StreamStateCacheExpire time.Duration

	// MaxMessageWorkers is the maximum number of concurrent workers to employ
	// for any given message. If <= 0, DefaultMaxMessageWorkers will be applied.
	MaxMessageWorkers int
}

Collector is a stateful object responsible for ingesting LogDog logs, registering them with a Coordinator, and stowing them in short-term storage for streaming and processing.

A Collector's Close should be called when finished to release any internal resources.

func (*Collector) Close

func (c *Collector) Close()

Close releases any internal resources and blocks pending the completion of any outstanding operations. After Close, no new Process calls may be made.

func (*Collector) Process

func (c *Collector) Process(ctx context.Context, msg []byte) error

Process ingests an encoded ButlerLogBundle message, registering it with the LogDog Coordinator and stowing it in a temporary Storage for streaming retrieval.

If a transient error occurs during ingest, Process will return an error. If no error occurred, or if there was an error with the input data, no error will be returned.

Directories

Path Synopsis
Package coordinator implements a minimal interface to the Coordinator service that is sufficient for Collector usage.
Package coordinator implements a minimal interface to the Coordinator service that is sufficient for Collector usage.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL