hekalocal

package module
v0.0.0-...-89f488b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 3, 2016 License: MIT Imports: 12 Imported by: 0

README

To work on these plugins:

  1. Install goat (just download the latest release and put the appropriate executable somewhere on your path as goat).
  2. Install cmake, probably via brew install cmake.
  3. Run ./devsetup.sh to get dependencies and do build steps required by Heka.

Now you should be able to run goat test and have all the tests run.

Running the tests

While developing, you should be running tests constantly.

  1. Install a notifier plugin via brew install terminal-notifier or sudo apt-get install notify-send
  2. Run ./runtests.sh. Let it keep running while you edit the code, it will run the tests whenever you make changes.

Making a Heka build that includes these plugins

  1. Follow steps 1-2 from "To work on these plugins" above.
  2. Run ./build_heka.sh.

Now you should have a heka-xxxxxxx.tar.gz at the top level of the repo.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func NewTimestampUUID

func NewTimestampUUID(timestamp int64, hash []byte) uuid.UUID

NewTimestampUUID returns a modified UUID based on Version 1 UUID with the specified timestamp as the timestamp portion and the provided bytes (assumed to be a hashed version of the represented data) copied into the rest of the UUID bytes.

Types

type HashUUIDDecoder

type HashUUIDDecoder struct {
	Timestamp time.Time // Hard-code the timestamp for testing
}

HashUUIDDecoder sets the UUID to a hashed combination of the timestamp and payload.

func (*HashUUIDDecoder) Decode

func (d *HashUUIDDecoder) Decode(pack *pipeline.PipelinePack) (packs []*pipeline.PipelinePack, err error)

Decode is provided to make HashUUIDDecoder implement the Heka pipeline.Decoder interface.

func (*HashUUIDDecoder) Init

func (d *HashUUIDDecoder) Init(config interface{}) (err error)

Init is provided to make HashUUIDDecoder implement the Heka pipeline.Plugin interface.

type JSONDecoder

type JSONDecoder struct {
	// contains filtered or unexported fields
}

JSONDecoder parses JSON message payloads and fills their contents into the message fields. It also optionally fills in the Timestamp, UUID, and Type message fields.

func (*JSONDecoder) ConfigStruct

func (jd *JSONDecoder) ConfigStruct() interface{}

ConfigStruct is provided to make JSONDecoder implement the Heka pipeline.HasConfigStruct interface.

func (*JSONDecoder) Decode

func (jd *JSONDecoder) Decode(pack *pipeline.PipelinePack) (packs []*pipeline.PipelinePack, err error)

Decode is provided to make JSONDecoder implement the Heka pipeline.Decoder interface.

func (*JSONDecoder) Init

func (jd *JSONDecoder) Init(config interface{}) (err error)

Init is provided to make JSONDecoder implement the Heka pipeline.Plugin interface.

type JSONDecoderConfig

type JSONDecoderConfig struct {
	TimestampField   string            `toml:"timestamp_field"`
	UUIDField        string            `toml:"uuid_field"`
	TypeField        string            `toml:"type_field"`
	LoggerField      string            `toml:"logger_field"`
	EnvVersionField  string            `toml:"env_version_field"`
	HostnameField    string            `toml:"hostname_field"`
	SeverityField    string            `toml:"severity_field"`
	PIDField         string            `toml:"pid_field"`
	Flatten          bool              `toml:"flatten"`
	FlattenPrefix    string            `toml:"flatten_prefix"`
	FlattenToStrings bool              `toml:"flatten_to_strings"`
	MoveFields       map[string]string `toml:"move_fields"`
	KeepFields       []string          `toml:"keep_fields"`
	RemoveFields     []string          `toml:"remove_fields"`

	// The message payload will be hashed and made into a UUID along with the timestamp.
	HashUUID bool `toml:"hash_uuid"`
	// contains filtered or unexported fields
}

JSONDecoderConfig contains the optional field names from which to extract message fields.

type JSONEncoder

type JSONEncoder struct {
	// contains filtered or unexported fields
}

JSONEncoder serializes messages to JSON.

func (*JSONEncoder) ConfigStruct

func (enc *JSONEncoder) ConfigStruct() interface{}

ConfigStruct is provided to make JSONEncoder implement the Heka pipeline.HasConfigStruct interface.

func (*JSONEncoder) Encode

func (enc *JSONEncoder) Encode(pack *pipeline.PipelinePack) (output []byte, err error)

Encode is implemented to make JSONEncoder implement the pipeline.Encoder interface.

func (*JSONEncoder) Init

func (enc *JSONEncoder) Init(config interface{}) (err error)

Init is provided to make JSONEncoder implement the Heka pipeline.Plugin interface.

type JSONEncoderConfig

type JSONEncoderConfig struct {
	TimestampField  string `toml:"timestamp_field"`
	UUIDField       string `toml:"uuid_field"`
	SeverityField   string `toml:"severity_field"`
	TypeField       string `toml:"type_field"`
	LoggerField     string `toml:"logger_field"`
	EnvVersionField string `toml:"env_version_field"`
	HostnameField   string `toml:"hostname_field"`
	PIDField        string `toml:"pid_field"`

	ElasticsearchBulk  bool   `toml:"elasticsearch_bulk"`
	ElasticsearchIndex string `toml:"elasticsearch_index"`
	ElasticsearchType  string `toml:"elasticsearch_type"`
	ElasticsearchID    string `toml:"elasticsearch_id"`
	// contains filtered or unexported fields
}

JSONEncoderConfig contains the optional field names to put Message fields into.

type UnflattenDecoder

type UnflattenDecoder struct{}

UnflattenDecoder converts from fields with dotted names as keys to nested JSON-encoded objects. Currently only supports keys with single dots.

func (*UnflattenDecoder) Decode

Decode is provided to make UnflattenDecoder implement the Heka pipeline.Decoder interface.

func (*UnflattenDecoder) Init

func (d *UnflattenDecoder) Init(config interface{}) (err error)

Init is provided to make UnflattenDecoder implement the Heka pipeline.Plugin interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL