Documentation
¶
Overview ¶
Package hec provides a client for Splunk's HTTP Event Collector.
The Client supports opening multiple EventWriters which can be used as a simple io.Writer for sending complete events to Splunk. This makes it easy to drop in as a write target for most log packages.
For example, to route Go's standard logger to a Splunk instance, configure a new token on the Splunk instance and configure a client:
server := "https://splunk.example.com/services/collector"
token := "1111-2222-3333-4444"
client := hec.New(server, token)
ew := client.NewEventWriter("", "myprogram", "golog", "hostname", "main")
log.SetOutput(w)
log.Println("This log example will be sent to Splunk")
// must call Close before exiting the program to ensure buffered
// data is flushed through to Splunk.
client.Close(time.Minute)
Individual events can also be transmitted directly using the client.WriteEvent method.
Index ¶
- Variables
- type Client
- type Config
- func WithBufferSize(n int) Config
- func WithDropOnFull() Config
- func WithErrLog(log *log.Logger) Config
- func WithFlushInterval(d time.Duration) Config
- func WithHTTPClient(client *http.Client) Config
- func WithInsecureTransport() Config
- func WithNoCompression() Config
- func WithNoRetry() Config
- func WithQueueDepth(n int) Config
- func WithRequestTimeout(timeout time.Duration) Config
- func WithRetryBackOff(b backoff.BackOff) Config
- type Event
- type EventWriter
Constants ¶
This section is empty.
Variables ¶
var ( // DefaultFlushInterval specifies how often the buffer will be flushed // to the Splunk server if DefaultBufferSize bytes of data have not // been accumulated first. DefaultFlushInterval = 5 * time.Second // DefaultBufferSize specifies the maximum amount of raw (uncompressed) // data the client will store before triggering a flush to the Splunk // server ahead of DefaultFlushInterval being reached. DefaultBufferSize = 128 * 1024 // DefaultQueueDepth specifies the number of events that the writer will // buffer before either blocking, or dropping them depending on configuration. DefaultQueueDepth = 10000 // DefaultRequestTimeout sets the maximum amount of time an individual // data transmission request to Splunk should take before its canceled // and retried. DefaultRequestTimeout = 30 * time.Second )
var ( ErrQueueFull = errors.New("log queue is full") ErrTimeout = errors.New("timeout waiting for flush") )
Errors returned by the client.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client implements an HTTP Event Collector client.
func New ¶
New creates a new HTTP Event Collector Client.
url must specify the complete url of the collector endpoint to send to eg. https://splunk.example.com:9081/services/collector
token must contain the HEC token configured on the server.
A background goroutine is started by this function. Call Close or Abort to shut it down cleanly.
Call NewEventWriter to receive an io.Writer compatible target to send data to.
func (*Client) Abort ¶
Abort closes the connection to the server as rapidly as possible. It will attempt to flush any outstanding data, but will not make any further retries if the first attempt fails.
Abort will return nil if all pending data was flushed successfully, else it will return an error.
func (*Client) Close ¶
Close flushes any remaining data to the server and waits until the specified timeout for the flush to complete, else returns ErrTimeout.
Close may safely be called multiple times.
No further writes are permitted once Close has been called.
func (*Client) NewEventWriter ¶
func (c *Client) NewEventWriter(timeFormat, source, sourcetype, host, index string) *EventWriter
NewEventWriter returns an EventWriter instance which implements an io.Writer interface.
timeFormat specifies an optional time format prefix string to be added to each log entry received. This can be any string that time.Format accepts.
source, sourceytype, host and index are passed through to the Splunk server for every submitted event.
Multiple EventWriters can be opened and used concurrently; they are safe to use from concurrent goroutines.
func (*Client) WriteEvent ¶
WriteEvent queues a single event for transmission. This method is goroutine-safe.
type Config ¶
type Config func(c *Client)
Config is the argument type expected by New for optional configuration parameters.
func WithDropOnFull ¶
func WithDropOnFull() Config
WithDropOnFull configures the client to drop incoming log messages should the inbound queue fill up (eg. due to the remote server being offline, or the message rate being too high). The default behaviour will cause writes to block if the queue is full.
WithQueueDepth can be used to change the amount of data the client will queue.
func WithErrLog ¶
WithErrLog defines a logger that should receive messages about problems this client has sending data to Splunk. Defaults to stderr. Set to nil to suppress all notices.
func WithFlushInterval ¶
WithFlushInterval overrides DefaultFlushInterval.
func WithHTTPClient ¶
WithHTTPClient specifies an HTTP client for the HEC client to use when contacting the Splunk server. This client should be configured with any required root certificates necessary to securely authenticate the remote server.
func WithInsecureTransport ¶
func WithInsecureTransport() Config
WithInsecureTransport provides a shortcut to force the client to ignore invalid TLS certificates provided by the server. This can be useful for local testing, but should not be used in production.
func WithNoCompression ¶
func WithNoCompression() Config
WithNoCompression disables the use of gzip compression when sending data to the server.
func WithNoRetry ¶
func WithNoRetry() Config
WithNoRetry will cause the client to only make a single attempt to flush each buffer to the server. A failure will cause future writes to fail.
By default an exponential retry policy is used instead - See WithRetryBackoff.
func WithRequestTimeout ¶
WithRequestTimeout sets a limit for the maximum amount of time a request can take to flush data to Splunk. If the timeout is reached, the request will be retried. This should not be too low, so that only genuinely stuck requests are retried.
func WithRetryBackOff ¶
WithRetryBackOff allows a custom retry policy to be specified. By default, an exponential backoff algorithm is used to handle transient network or server failures.
type Event ¶
type Event struct {
// Data holds the event payload to be indexed. This can be a text
// string (possibly including newlines), or a JSON marshalable structure.
Data interface{} `json:"event"`
// Timestamp for the event; defaults to the current time. Supports
// millisecond accuracy.
Time time.Time `json:"-"`
// Host metadata field; overrides the value set on the Splunk input.
Host string `json:"host,omitempty"`
// Source metadata field; overrides the value set on the Splunk input.
Source string `json:"source,omitempty"`
// Sourcetype metadata field; overrides the default value set on the Splunk input.
SourceType string `json:"sourcetype,omitempty"`
// Sourcetype metadata field; overrides the default value set on the Splunk input.
Index string `json:"index,omitempty"`
}
Event defines a single event to send to Splunk, along with optional metadata.
type EventWriter ¶
type EventWriter struct {
// contains filtered or unexported fields
}
EventWriter implements an io.Writer interface for sending events to the collector.
Each call to Write will emit a single (possibly multi-line) event.
Source Files