README
¶
logs-agent
logs-agent collects logs and submits them to datadog's infrastructure.
Structure
logs reads the config files, and instantiates what's needed.
Each log line comes from a source (e.g. file, network, docker), and then enters one of the available pipeline - tailer|listener|container -> decoder -> processor -> sender -> auditor
Tailer tails a file and submits data to the processors
Listener listens on local network (TCP, UDP, Unix) and submits data to the processors
Container scans docker logs from stdout/stderr and submits data to the processors
Decoder converts bytes arrays into messages
Processor updates the messages, filtering, redacting or adding metadata, and submits to the forwarder
Sender submits the messages to the intake, and notifies the auditor
Auditor notes that messages were properly submitted, stores offsets for agent restarts
Tests
# Run the unit tests
inv test --targets=./pkg/logs --timeout=10
Documentation
¶
Index ¶
- Constants
- func BlockUntilAutoConfigRanOnce(getAC func() *autodiscovery.AutoConfig, timeout time.Duration)
- func Flush(ctx context.Context)
- func GetMessageReceiver() *diagnostic.BufferedMessageReceiver
- func GetStatus() status.Status
- func IsAgentRunning() bool
- func Start(getAC func() *autodiscovery.AutoConfig) error
- func StartServerless(getAC func() *autodiscovery.AutoConfig, logsChan chan *config.ChannelMessage, ...) error
- func Stop()
- type Agent
Constants ¶
const (
// AgentJSONIntakeProtocol agent json protocol
AgentJSONIntakeProtocol = "agent-json"
)
Variables ¶
This section is empty.
Functions ¶
func BlockUntilAutoConfigRanOnce ¶
func BlockUntilAutoConfigRanOnce(getAC func() *autodiscovery.AutoConfig, timeout time.Duration)
BlockUntilAutoConfigRanOnce blocks until the AutoConfig has been run once. It also returns after the given timeout.
func Flush ¶
Flush flushes synchronously the running instance of the Logs Agent. Use a WithTimeout context in order to have a flush that can be cancelled.
func GetMessageReceiver ¶
func GetMessageReceiver() *diagnostic.BufferedMessageReceiver
GetMessageReceiver returns the diagnostic message receiver
func IsAgentRunning ¶
func IsAgentRunning() bool
IsAgentRunning returns true if the logs-agent is running.
func Start ¶
func Start(getAC func() *autodiscovery.AutoConfig) error
Start starts logs-agent getAC is a func returning the prepared AutoConfig. It is nil until the AutoConfig is ready, please consider using BlockUntilAutoConfigRanOnce instead of directly using it. The parameter serverless indicates whether or not this Logs Agent is running in a serverless environment.
func StartServerless ¶
func StartServerless(getAC func() *autodiscovery.AutoConfig, logsChan chan *config.ChannelMessage, extraTags []string) error
StartServerless starts a Serverless instance of the Logs Agent.
Types ¶
type Agent ¶
type Agent struct {
// contains filtered or unexported fields
}
Agent represents the data pipeline that collects, decodes, processes and sends logs to the backend + ------------------------------------------------------ + | | | Collector -> Decoder -> Processor -> Sender -> Auditor | | | + ------------------------------------------------------ +
func NewAgent ¶
func NewAgent(sources *config.LogSources, services *service.Services, processingRules []*config.ProcessingRule, endpoints *config.Endpoints) *Agent
NewAgent returns a new Logs Agent
func NewServerless ¶
func NewServerless(sources *config.LogSources, services *service.Services, processingRules []*config.ProcessingRule, endpoints *config.Endpoints) *Agent
NewServerless returns a Logs Agent instance to run in a serverless environment. The Serverless Logs Agent has only one input being the channel to receive the logs to process. It is using a NullAuditor because we've nothing to do after having sent the logs to the intake.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package parser supports transforming raw log "lines" into messages with some associated metadata (timestamp, severity, etc.).
|
Package parser supports transforming raw log "lines" into messages with some associated metadata (timestamp, severity, etc.). |
|
Package pb is a generated protocol buffer package.
|
Package pb is a generated protocol buffer package. |