cypress

package module
v0.0.0-...-c2832fb Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 29, 2016 License: BSD-3-Clause Imports: 33 Imported by: 5

README

cypress

Toolkit

Cypress is a logging toolkit for creating streams of messages and transporting them.

It's native streams are compressed protocol buffer: reducing bandwidth and retaining/improving the fidelity of log data.

The toolkit comes with an array of handlers, capable sending streams across networks, saving them to disk, filtering them, and translating them into external formats.

All translations are bidirectional. So if you save a stream into s3, cypress can read that stream back natively. This extends to it's integration with SaaS logging products as well.

The aim is a flexible tool that gives users the highlevel building blocks to create a logging solution that works for them.

More than logs

Cypress handles metrics as well as logs. In fact, there are 5 messages types:

  • Log - A log message, representing arbitrary information
  • Metric - A metric as you'd find in statsd, etc
  • Trace - An application trace use to correlate activity
  • Audit - A high value log message
  • Heartbeat - A simple indicator of aliveness

The types allow the different handlers to create messages specially. For instance, the builtin metrics handler only considers messages of type Metric, as you'd expect.

Golang

Cypress is a logging framework in golang.

The core is most handlers are written in golang, allowing for extremely easily deployment.

Config

Cypress uses configuration files to control some of the various aspects of it's operation. The format is TOML and here is an example:

[s3]

allow_unsigned = true
sign_key = "evan@vektra.com"

Documentation

Overview

Package cypress is a generated protocol buffer package.

It is generated from these files: 

log.proto

It has these top-level messages: 

Interval
Attribute
Tag
Message
StreamHeader

Index

Constants

View Source 
const AUDIT = 3

Type code for an audit (ie, high value log) message

View Source 
const CypressPathEnv = "CYPRESS_PATH"
View Source 
const DEFAULT_VERSION = 1

The default version of messages generated

View Source 
const DefaultSocketPath = "/var/lib/cypress.sock"

The path on this system that the agent listens

View Source 
const HEARTBEAT = 4

Type code for a heartbeat, representing liveness.

View Source 
const LOG = 0

Type code representing a generic log message

View Source 
const METRIC = 1

Type code for a metric

View Source 
const TRACE = 2

Type code for an application trace

Variables

View Source 
var (
	// The given message was invalid
	ErrInvalidMessage = errors.New("invalid message")

	// The system could not deduce the encoding of a stream
	ErrUnknownStreamType = errors.New("unknown stream type")
)
View Source 
var (
	// Indicates that a Plugin does not have a Receiver
	ErrNoReceiver = errors.New("no receiver available")

	// Indicates that a Plugin does not have a Generator
	ErrNoGenerator = errors.New("no generator available")
)
View Source 
var (
	// Disable windowing, acknowledge each message immediately
	NoWindow int = -1

	// An average messages/sec rate to calculate against
	DefaultMPSRate int = 1000

	// A decent minimum window that assures some improved throughput
	MinimumWindow = MinimumSendWindow(1*time.Millisecond, DefaultMPSRate)

	// A window for use on a fast lan where transmission delay is very small
	FastLanWindow = MinimumWindow

	// A window for use on a slower lan (cloud infrastructer, across AZ)
	SlowLanWindow = MinimumSendWindow(3*time.Millisecond, DefaultMPSRate)

	// A window for use over faster internet paths
	FastInternetWindow = MinimumSendWindow(10*time.Millisecond, DefaultMPSRate)

	// A window for use over slowe internet paths
	SlowInternetWindow = MinimumSendWindow(50*time.Millisecond, DefaultMPSRate)
)
View Source 
var EmptyGlobalConfig bool

Whether or not to load the global Config from paths

View Source 
var ErrBadValue = errors.New("Invalid type for attribute value")

Error indicating that the value type and the attribute type mismatch

View Source 
var ErrClosed = errors.New("send closed")

Indicate that this Send is closed and can not be used

View Source 
var ErrOverflow = errors.New("overflow parsing varint")

Indicate that the varint is invalid

View Source 
var ErrStreamUnsynced = errors.New("stream unsynced")

Indicates that both sides of the stream have gotten confused and are no longer is sync.

View Source 
var GlobalConfigPaths []string

Paths that can hold the global config

View Source 
var GlobalPaths []string

Paths that can contain any global cypress data

View Source 
var HomeDir string
View Source 
var NONE = StreamHeader_NONE

No compression is applied

View Source 
var PotentialGlobalPaths = []string{
	"/etc/cypress",
	"/var/lib/cypress",
}

Paths that, if they exist, are added to GlobalConfigPaths

View Source 
var SNAPPY = StreamHeader_SNAPPY

Snappy compression is used

View Source 
var StreamHeader_Compression_name = map[int32]string{
	0: "NONE",
	1: "SNAPPY",
	2: "ZLIB",
}
View Source 
var StreamHeader_Compression_value = map[string]int32{
	"NONE":   0,
	"SNAPPY": 1,
	"ZLIB":   2,
}
View Source 
var StreamHeader_Mode_name = map[int32]string{
	0: "RAW",
	1: "RELIABLE",
}
View Source 
var StreamHeader_Mode_value = map[string]int32{
	"RAW":      0,
	"RELIABLE": 1,
}
View Source 
var StreamNotifyByte = []byte{'-'}
View Source 
var UserConfigPath = UserPath + "/config"

The path under a users home for the user config

View Source 
var UserPath = ".cypress"
View Source 
var ZLIB = StreamHeader_ZLIB

ZLib compression is used

Functions

func AddPlugin 

func AddPlugin(name string, creator func() Plugin)

Add a new plugin by name with a function to create a new instance of this plugin.

func AllPlugins 

func AllPlugins() []string

func Available 

func Available() bool

Indicate if the cypress agent is available

func Close 

func Close() error

Close the system logger

func ExpandPath 

func ExpandPath(path string) string

func GlobalFile 

func GlobalFile(path string) (string, bool)

Retrieve the path for under global cypress directories

func Glue 

func Glue(gen Generator, recv Receiver) error

Read messages from gen and send them to recv

func GlueFiltered 

func GlueFiltered(gen Generator, filt Filterer, recv Receiver) error

Read messages from gen and send them to recv

func LoadMergedConfig 

func LoadMergedConfig(path string, cfg *Config) error

func LogPath 

func LogPath() string

Return the default log path. The environment variable LOG_PATH is used if set, otherwise DefaultSocketPath.

func MinimumSendWindow 

func MinimumSendWindow(d time.Duration, mps int) int

Given the transmission delay of the network (t) and the expected messages per second (mps), calculate the minimum window to use to maximize throughput.

func Open 

func Open()

Open a connection to the system logger

func ParseKVStream 

func ParseKVStream(in io.Reader, r Receiver) error

Given a reader, parse it as KV lines and send the message to r

func ReadCompressed 

func ReadCompressed(r io.Reader, comp StreamHeader_Compression) io.Reader

Given a compression level, return a wrapped Reader

func ReadUvarint 

func ReadUvarint(r io.Reader, buf []byte) (uint64, error)

Read a unsigned varint from the reader using buf as scratch. This reads data from r one byte at a time, making it a little slower if r is a net.Conn, but it keeps r positioned correctly as opposed to using a buffered reader.

func StandardStreamFilter 

func StandardStreamFilter(f Filterer) error

func UserFile 

func UserFile(path string) (string, bool)

Retrieve the path for under the users .cypress directory

func Write 

func Write(m *Message) error

Write a Message to the system logger

func WriteCompressed 

func WriteCompressed(w io.WriteCloser, comp StreamHeader_Compression) io.WriteCloser

Given a compression level, return a wrapped Writer

func WriteUvarint 

func WriteUvarint(w io.Writer, x uint64) (int, error)

Write a uint64 value to w in unsigned varint format.

Types

type Attribute 

type Attribute struct {
	Key              uint32    `protobuf:"varint,1,req,name=key" json:"key" codec:"key,omitempty"`
	Skey             *string   `protobuf:"bytes,2,opt,name=skey" json:"skey,omitempty" codec:"skey,omitempty"`
	Ival             *int64    `protobuf:"zigzag64,3,opt,name=ival" json:"ival,omitempty" codec:"ival,omitempty"`
	Sval             *string   `protobuf:"bytes,4,opt,name=sval" json:"sval,omitempty" codec:"sval,omitempty"`
	Bval             []byte    `protobuf:"bytes,5,opt,name=bval" json:"bval,omitempty" codec:"bval,omitempty"`
	Tval             *Interval `protobuf:"bytes,6,opt,name=tval" json:"tval,omitempty" codec:"tval,omitempty"`
	Boolval          *bool     `protobuf:"varint,7,opt,name=boolval" json:"boolval,omitempty" codec:"boolval,omitempty"`
	Fval             *float64  `protobuf:"fixed64,8,opt,name=fval" json:"fval,omitempty" codec:"fval,omitempty"`
	XXX_unrecognized []byte    `json:"-" codec:"-"`
}

func (*Attribute) Equal 

func (this *Attribute) Equal(that interface{}) bool

func (*Attribute) GetBoolval 

func (m *Attribute) GetBoolval() bool

func (*Attribute) GetBval 

func (m *Attribute) GetBval() []byte

func (*Attribute) GetFval 

func (m *Attribute) GetFval() float64

func (*Attribute) GetIval 

func (m *Attribute) GetIval() int64

func (*Attribute) GetKey 

func (m *Attribute) GetKey() uint32

func (*Attribute) GetSkey 

func (m *Attribute) GetSkey() string

func (*Attribute) GetSval 

func (m *Attribute) GetSval() string

func (*Attribute) GetTval 

func (m *Attribute) GetTval() *Interval

func (*Attribute) KVString 

func (attr *Attribute) KVString(m *Message) string

func (*Attribute) KVStringInto 

func (attr *Attribute) KVStringInto(m *Message, buf *bytes.Buffer)

Write a Message's attributes in KV format to a buffer

func (*Attribute) Marshal 

func (m *Attribute) Marshal() (data []byte, err error)

func (*Attribute) MarshalTo 

func (m *Attribute) MarshalTo(data []byte) (n int, err error)

func (*Attribute) ProtoMessage 

func (*Attribute) ProtoMessage()

func (*Attribute) Reset 

func (m *Attribute) Reset()

func (*Attribute) SetKey 

func (attr *Attribute) SetKey(m *Message, key string)

Set the key (ie, the name) of an Attribute.

func (*Attribute) Size 

func (m *Attribute) Size() (n int)

func (*Attribute) String 

func (m *Attribute) String() string

func (*Attribute) StringKey 

func (a *Attribute) StringKey(m *Message) string

Return the key as a string of this Attribute within Message m

func (*Attribute) Unmarshal 

func (m *Attribute) Unmarshal(data []byte) error

func (*Attribute) Value 

func (attr *Attribute) Value() interface{}

func (*Attribute) VerboseEqual 

func (this *Attribute) VerboseEqual(that interface{}) error

type BufferReceiver 

type BufferReceiver struct {
	Messages []*Message
	// contains filtered or unexported fields
}

A simple type that stores any Received message into a buffer. Mostly for testing.

func (*BufferReceiver) Close 

func (b *BufferReceiver) Close() error

func (*BufferReceiver) Receive 

func (b *BufferReceiver) Receive(m *Message) error

Store the message into the internal buffer

func (*BufferReceiver) SyncTo 

func (b *BufferReceiver) SyncTo()

Used for testing to syncronize goroutines using the value

type ByteBuffer 

type ByteBuffer struct {
	bytes.Buffer
}

func (*ByteBuffer) Close 

func (bb *ByteBuffer) Close() error

type Channel 

type Channel chan *Message

A Go channel that fits the Receiver and Generator interfaces.

func (Channel) Close 

func (c Channel) Close() error

Close the channel down

func (Channel) Generate 

func (c Channel) Generate() (*Message, error)

Return a message by reading from the channel.

func (Channel) Receive 

func (c Channel) Receive(m *Message) error

Write a message to the channel

type Config 

type Config struct {
	// contains filtered or unexported fields
}

func GlobalConfig 

func GlobalConfig() *Config

Load and return the global Config

func ParseConfig 

func ParseConfig(r io.Reader) (*Config, error)

func (*Config) Add 

func (cfg *Config) Add(r io.Reader) error

func (*Config) AddString 

func (cfg *Config) AddString(s string) error

func (*Config) Load 

func (cfg *Config) Load(name string, v interface{}) error

type Connector 

type Connector interface {
	Connect() (*Send, error)
}

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

A type that, given a stream of many shapes, decodes it and generates Messages.

func NewDecoder 

func NewDecoder(r io.Reader) *Decoder

Create a new Decoder reading data from r

func (*Decoder) Decode 

func (d *Decoder) Decode() (*Message, error)

Decode the next Message in the stream

type Duration 

type Duration struct {
	time.Duration
}

func (*Duration) UnmarshalTOML 

func (d *Duration) UnmarshalTOML(data []byte) error

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

A type which encodes messages given to it in native protobuf format and writes them out.

func NewEncoder 

func NewEncoder(w io.Writer) *Encoder

Create an Encoder that will write it's output to w

func (*Encoder) Encode 

func (e *Encoder) Encode(m *Message) (uint64, error)

Encode and write a Message

type Filterer 

type Filterer interface {
	Filter(m *Message) (*Message, error)
}

A core interface, representing a type that takes a message and returns a new message. The returned message can be nil.

type FiltererPlugin 

type FiltererPlugin interface {
	Filterer() (Filterer, error)
}

type Flusher 

type Flusher interface {
	Flush() error
}

type Generator 

type Generator interface {
	Generate() (*Message, error)
	Close() error
}

A core interface, representing a type that can create a Message

type GeneratorHandler 

type GeneratorHandler interface {
	HandleGenerator(g Generator)
}

Use to allow types to handle new Generators as they're created

type GeneratorHandlerFunc 

type GeneratorHandlerFunc func(g Generator)

A GeneratorHandler that just calls itself as a function

func (GeneratorHandlerFunc) HandleGenerator 

func (f GeneratorHandlerFunc) HandleGenerator(g Generator)

type GeneratorPlugin 

type GeneratorPlugin interface {
	Generator() (Generator, error)
}

type Inter 

type Inter interface {
	Int() int64
}

Used to convert a value into an int64

type Interval 

type Interval struct {
	Seconds          uint64 `protobuf:"varint,1,req,name=seconds" json:"seconds" codec:"seconds"`
	Nanoseconds      uint32 `protobuf:"varint,2,req,name=nanoseconds" json:"nanoseconds" codec:"nanoseconds"`
	XXX_unrecognized []byte `json:"-" codec:"-"`
}

func (*Interval) Duration 

func (i *Interval) Duration() time.Duration

func (*Interval) Equal 

func (this *Interval) Equal(that interface{}) bool

func (*Interval) GetNanoseconds 

func (m *Interval) GetNanoseconds() uint32

func (*Interval) GetSeconds 

func (m *Interval) GetSeconds() uint64

func (*Interval) Marshal 

func (m *Interval) Marshal() (data []byte, err error)

func (*Interval) MarshalTo 

func (m *Interval) MarshalTo(data []byte) (n int, err error)

func (*Interval) ProtoMessage 

func (*Interval) ProtoMessage()

func (*Interval) Reset 

func (m *Interval) Reset()

func (*Interval) Size 

func (m *Interval) Size() (n int)

func (*Interval) String 

func (m *Interval) String() string

func (*Interval) Unmarshal 

func (m *Interval) Unmarshal(data []byte) error

func (*Interval) VerboseEqual 

func (this *Interval) VerboseEqual(that interface{}) error

type KVEncoder 

type KVEncoder struct {
	// contains filtered or unexported fields
}

An encoder that writes messages in Key/Value format

func NewKVEncoder 

func NewKVEncoder(w io.Writer) *KVEncoder

Create a KVEncoder that writes it's output to w

func (*KVEncoder) Encode 

func (kv *KVEncoder) Encode(m *Message) (uint64, error)

Encode and write a message

type KVParser 

type KVParser struct {
	Bare bool
	// contains filtered or unexported fields
}

A type which can transform a io.Reader into a set of Messages

func NewKVParser 

func NewKVParser(r io.Reader) *KVParser

Create a new KVParser from the data in r

func (*KVParser) Close 

func (p *KVParser) Close() error

To fit the Generator interface

func (*KVParser) Generate 

func (s *KVParser) Generate() (*Message, error)

Read and decode and return another Message

type Logger 

type Logger interface {
	Write(m *Message) error
	Close() error
}

A simple interface used to represent a system logger

func Connect 

func Connect() Logger

Connect to the default system logger

func ConnectTo 

func ConnectTo(path string) Logger

Connect to an agent on a path and return a Logger

type Message 

type Message struct {
	Version          int32          `protobuf:"varint,6,opt,name=version" json:"version" codec:"version"`
	Timestamp        *tai64n.TAI64N `protobuf:"bytes,1,req,name=timestamp" json:"timestamp,omitempty" codec:"timestamp"`
	Type             *uint32        `protobuf:"varint,2,req,name=type" json:"type,omitempty" codec:"type"`
	Attributes       []*Attribute   `protobuf:"bytes,3,rep,name=attributes" json:"attributes,omitempty" codec:"attributes"`
	SessionId        *string        `protobuf:"bytes,4,opt,name=session_id" json:"session_id,omitempty" codec:"session_id"`
	Tags             []*Tag         `protobuf:"bytes,5,rep" json:"Tags,omitempty" codec:"tags"`
	XXX_unrecognized []byte         `json:"-" codec:"-"`
}

func Audit 

func Audit() *Message

Create a new Message of type AUDIT

func Heartbeat 

func Heartbeat() *Message

Create a new Message of type HEARTBEAT

func Log 

func Log() *Message

Create a new Message of type LOG

func Metric 

func Metric() *Message

Create a new Message of type METRIC

func NewMessage 

func NewMessage() *Message

Create a new Message

func ParseKV 

func ParseKV(line string) (*Message, error)

Parse the given line in KV format into a Message

func ParseSimpleJSON 

func ParseSimpleJSON(data []byte) (*Message, error)

Generate a Message from json in data

func Trace 

func Trace() *Message

Create a new Message of type TRACE

func (*Message) Add 

func (m *Message) Add(key string, val interface{}) error

Add a new attribute to the Message. The type of the attribute is infered from the type of val. This understands: 

int, int32, uint32, int64, uint64, Inter
float32, float64,
string, Stringer
time.Duration
[]byte
error
A slice, array, map, or struct containing any understood type

func (*Message) AddBytes 

func (m *Message) AddBytes(key string, val []byte) error

Add an Attribute of type bytes

func (*Message) AddDuration 

func (m *Message) AddDuration(key string, dur time.Duration) error

Add an Attribute of type Internal from a time.Duration

func (*Message) AddFloat 

func (m *Message) AddFloat(key string, val float64) error

Add an Attribute of type float64

func (*Message) AddInt 

func (m *Message) AddInt(key string, val int64) error

Add an Attribute of type int64

func (*Message) AddInterval 

func (m *Message) AddInterval(key string, sec uint64, nsec uint32) error

Add an Attribute of type Internal

func (*Message) AddMany 

func (m *Message) AddMany(vals ...interface{}) error

Add many attributes to the Message. vals is pairs of (key, value) For example: 

m.AddMany("name", "evan", "age", 35)

This creates an attributed with a key of "name" and a value of "evan" and "age" with a value of 35.

func (*Message) AddString 

func (m *Message) AddString(key string, val string) error

Add an Attribute of type string

func (*Message) AddTag 

func (m *Message) AddTag(key string, val string)

Add a tag to the Message

func (*Message) Equal 

func (this *Message) Equal(that interface{}) bool

func (*Message) For 

func (m *Message) For(id string)

Set the SessionID of a Message

func (*Message) Get 

func (m *Message) Get(key string) (interface{}, bool)

Find an Attribute by name and return it's value.

func (*Message) GetAttributes 

func (m *Message) GetAttributes() []*Attribute

func (*Message) GetBool 

func (m *Message) GetBool(key string) (bool, bool)

Find an Attibute by name that is a boolean and return it

func (*Message) GetBytes 

func (m *Message) GetBytes(key string) ([]byte, bool)

Find an Attibute by name that is a byte slice and return it

func (*Message) GetFloat 

func (m *Message) GetFloat(key string) (float64, bool)

Find an Attibute by name that is a float and return it

func (*Message) GetInt 

func (m *Message) GetInt(key string) (int64, bool)

Find an Attibute by name that is an int and return it

func (*Message) GetInterval 

func (m *Message) GetInterval(key string) (*Interval, bool)

Find an Attibute by name that is an Interval and return it

func (*Message) GetSessionId 

func (m *Message) GetSessionId() string

func (*Message) GetString 

func (m *Message) GetString(key string) (string, bool)

Find an Attibute by name that is a string and return it

func (*Message) GetTag 

func (m *Message) GetTag(key string) (string, bool)

Return the value of a tag

func (*Message) GetTags 

func (m *Message) GetTags() []*Tag

func (*Message) GetTimestamp 

func (m *Message) GetTimestamp() *tai64n.TAI64N

func (*Message) GetType 

func (m *Message) GetType() uint32

func (*Message) GetVersion 

func (m *Message) GetVersion() int32

func (*Message) HstoreAttributes 

func (m *Message) HstoreAttributes() string

Return the Message's attributes as a string formatted for use in Postgresql HSTORE

func (*Message) HstoreAttributesInto 

func (m *Message) HstoreAttributesInto(buf *bytes.Buffer)

Write a Message's attributes formatted for Postgresql HSTORE into a buffer

func (*Message) HstoreTags 

func (m *Message) HstoreTags() string

Return the Message's tags as a string formatted for use in Postgresql HSTORE

func (*Message) HstoreTagsInto 

func (m *Message) HstoreTagsInto(buf *bytes.Buffer)

Write a Message's tags formatted for Postgresql HSTORE into a buffer

func (*Message) HumanString 

func (m *Message) HumanString() string

Return a Message as a string formatted for easy human reading

func (*Message) KVPairs 

func (m *Message) KVPairs() string

Return the Message's attributes as a KV formatted string

func (*Message) KVPairsInto 

func (m *Message) KVPairsInto(buf *bytes.Buffer)

Write a Message's attributes in KV format to a buffer

func (*Message) KVString 

func (m *Message) KVString() string

Return the Message as a KV formatted string

func (*Message) KVStringInto 

func (m *Message) KVStringInto(buf *bytes.Buffer)

Write a Message in KV format to a buffer

func (*Message) KVTagsInto 

func (m *Message) KVTagsInto(buf *bytes.Buffer)

Write a Message's tags in KV format to a buffer

func (*Message) Marshal 

func (m *Message) Marshal() (data []byte, err error)

func (*Message) MarshalJSON 

func (m *Message) MarshalJSON() ([]byte, error)

Generate JSON for a Message

func (*Message) MarshalTo 

func (m *Message) MarshalTo(data []byte) (n int, err error)

func (*Message) ProtoMessage 

func (*Message) ProtoMessage()

func (*Message) Remove 

func (m *Message) Remove(key string) error

Remove an attribute by name from the message

func (*Message) RemoveTag 

func (m *Message) RemoveTag(key string) error

Remove a tag by name from the message

func (*Message) Reset 

func (m *Message) Reset()

func (*Message) SimpleJSONMap 

func (m *Message) SimpleJSONMap() map[string]interface{}

Return a simple map representing the Message used to generate JSON

func (*Message) Size 

func (m *Message) Size() (n int)

func (*Message) String 

func (m *Message) String() string

func (*Message) StringType 

func (m *Message) StringType() string

Return the Message type as a string

func (*Message) SyslogString 

func (m *Message) SyslogString(colorize bool, align bool) string

Return a Message formatted as a syslog string. colorize indicates if ANSI color codes should be used to highlight portions. align controls if time field is aligned to 35 bytes (useful for when a set of messages are displayed on lines next to eachother).

func (*Message) Unmarshal 

func (m *Message) Unmarshal(data []byte) error

func (*Message) UnmarshalJSON 

func (m *Message) UnmarshalJSON(data []byte) error

Populate a Message from JSON in data

func (*Message) VerboseEqual 

func (this *Message) VerboseEqual(that interface{}) error

type MessageEncoder 

type MessageEncoder interface {
	Encode(m *Message) (uint64, error)
}

type Messages 

type Messages []*Message

A slices of Messages that can be used to order messages by Timestamp

func (Messages) Len 

func (m Messages) Len() int

func (Messages) Less 

func (m Messages) Less(i, j int) bool

func (Messages) Swap 

func (m Messages) Swap(i, j int)

type Plugin 

type Plugin interface{}

An interface implemented by plugins used by the router

func FindPlugin 

func FindPlugin(name string) (Plugin, bool)

Find a plugin by name and invoke it's creator function to create a new Plugin instance

type Probe 

type Probe struct {
	Header *StreamHeader
	Stream io.Reader
	// contains filtered or unexported fields
}

A type which can look at a reader and detect the format

func NewProbe 

func NewProbe(r io.Reader) *Probe

Create a new Probe from the data in r

func (*Probe) Compression 

func (p *Probe) Compression() StreamHeader_Compression

Indicate the compression in use

func (*Probe) Probe 

func (p *Probe) Probe() error

Inspect the stream and populate Header with the data

func (*Probe) Reader 

func (p *Probe) Reader() io.Reader

Create an io.Reader for the remainder of the stream

func (*Probe) Writer 

func (p *Probe) Writer(w io.WriteCloser) io.WriteCloser

Create an io.Writer that will match the parameters of the probed stream.

type Receiver 

type Receiver interface {
	Receive(msg *Message) error
	Close() error
}

A core interface, represending a type that can take a Message

type ReceiverPlugin 

type ReceiverPlugin interface {
	Receiver() (Receiver, error)
}

type Recv 

type Recv struct {
	// contains filtered or unexported fields
}

A type which can recieve a stream of Messages reliabliy. Recv works in coordination with Send to reliablity send Messages using ack'ing.

func NewRecv 

func NewRecv(rw io.ReadWriter) (*Recv, error)

Create a new Recv, reading and writing from rw.

func (*Recv) Close 

func (r *Recv) Close() error

To satisify the Generator interface

func (*Recv) Generate 

func (r *Recv) Generate() (*Message, error)

Generate a new Message reading from the stream. If the stream is in reliable mode (the default) then an ack is sent back.

type ReliableSend 

type ReliableSend struct {
	// contains filtered or unexported fields
}

func NewReliableSend 

func NewReliableSend(c Connector, buffer int) *ReliableSend

func (*ReliableSend) Ack 

func (r *ReliableSend) Ack(m *Message)

func (*ReliableSend) Close 

func (r *ReliableSend) Close() error

func (*ReliableSend) Flush 

func (r *ReliableSend) Flush() error

func (*ReliableSend) Nack 

func (r *ReliableSend) Nack(m *Message)

func (*ReliableSend) Outstanding 

func (r *ReliableSend) Outstanding() int

func (*ReliableSend) Receive 

func (r *ReliableSend) Receive(m *Message) error

func (*ReliableSend) Start 

func (r *ReliableSend) Start() error

type Run 

type Run struct {
	// contains filtered or unexported fields
}

A type which runs a command and generates messages from the commands standard output.

func NewRun 

func NewRun(prog string, args ...string) (*Run, error)

Create a new Run for the given program with arguments.

func (*Run) Close 

func (r *Run) Close() error

Close the output from the program and wait for it to finish.

func (*Run) Generate 

func (r *Run) Generate() (*Message, error)

Generate a Message from the programs next output line.

type Runner 

type Runner interface {
	Run(r Receiver) error
}

A core interface, represents a type that runs and sends messages to a downstream receiver

type Send 

type Send struct {
	OnClosed func()
	// contains filtered or unexported fields
}

A type use to send a stream of Messages reliably. This type works in coordination with Recv to make transport the stream reliably by buffering and acking messages.

func NewSend 

func NewSend(rw io.ReadWriteCloser, window int) *Send

Create a new Send, reading and writing from rw. Window controls the size of the ack window to use. See MinimumSendWindow and the Window variables for information window sizes. If the window is set to 0, the default window size is used. NOTE: The window size has a big effect on the throughput of Send, so be sure to consider it's value. The larger the window, the higher the memory usage and throughput. Fast lans only require a small window because there is a very small transmission delay.

func (*Send) Close 

func (s *Send) Close() error

func (*Send) Flush 

func (s *Send) Flush() error

func (*Send) Receive 

func (s *Send) Receive(m *Message) error

Send a Message to the remote side

func (*Send) Send 

func (s *Send) Send(m *Message, req SendRequest) error

Send a Message to the remote side. if req is not nil, then it will be updated as to the status of m, calling either Ack or Nack depending on if things go ok or not.

func (*Send) SendHandshake 

func (s *Send) SendHandshake() error

Send the start of a stream to the remote side. This will initialize the stream to use Snappy for compression and reliable transmission.

type SendRequest 

type SendRequest interface {
	Ack(*Message)
	Nack(*Message)
}

Used by Send to allow a sender to interact with the Message transmit and ack lifecycle

type SerialReceiver 

type SerialReceiver struct {
	Receiver Receiver
	// contains filtered or unexported fields
}

A type that wraps a Receiver in a mutex

func NewSerialReceiver 

func NewSerialReceiver(r Receiver) *SerialReceiver

func (*SerialReceiver) Receive 

func (s *SerialReceiver) Receive(m *Message) error

type StaticGeneratorMessages 

type StaticGeneratorMessages struct {
	// contains filtered or unexported fields
}

A type which holds a set of Messages and returns them when requested Mostly used for testing.

func StaticGenerator 

func StaticGenerator(m ...*Message) *StaticGeneratorMessages

Create a StaticGeneratorMessages from the given Message set

func (*StaticGeneratorMessages) Close 

func (s *StaticGeneratorMessages) Close() error

To satisfy the Generator interface

func (*StaticGeneratorMessages) Generate 

func (s *StaticGeneratorMessages) Generate() (*Message, error)

Return the next Message or nil if empty

type StreamDecoder 

type StreamDecoder struct {
	Header *StreamHeader
	// contains filtered or unexported fields
}

A type which uses Probe and Decoder generate Messages

func NewStreamDecoder 

func NewStreamDecoder(r io.Reader) (*StreamDecoder, error)

Create a new StreamDecoder from the data in r

func (*StreamDecoder) Close 

func (s *StreamDecoder) Close() error

To satisify the Generator interface

func (*StreamDecoder) Generate 

func (s *StreamDecoder) Generate() (*Message, error)

Read the next Message in the stream. If the stream has not been initialized, Probe() is called first.

func (*StreamDecoder) Probe 

func (s *StreamDecoder) Probe() error

Probe the stream and setup the decoder to read Messages

type StreamEncoder 

type StreamEncoder struct {
	// contains filtered or unexported fields
}

A type that encodes Messages to a stream with optional compression

func NewStreamEncoder 

func NewStreamEncoder(w io.WriteCloser) *StreamEncoder

Create a new StreamEncoder sending data to w. The format is either the native format or kv format if w is stdout and a tty.

func (*StreamEncoder) Close 

func (s *StreamEncoder) Close() error

func (*StreamEncoder) EncodedBytes 

func (s *StreamEncoder) EncodedBytes() uint64

Indicate how many bytes have been sent

func (*StreamEncoder) Flush 

func (s *StreamEncoder) Flush() error

func (*StreamEncoder) Init 

func (s *StreamEncoder) Init(comp StreamHeader_Compression) error

Initialize the StreamEncoder to a particular compression level and write the header

func (*StreamEncoder) OpenFile 

func (s *StreamEncoder) OpenFile(f *os.File) error

Probe the file and setup the encoder to match the probe's settings.

func (*StreamEncoder) Receive 

func (s *StreamEncoder) Receive(m *Message) error

Take a Message and encode it

func (*StreamEncoder) WriteCustomHeader 

func (s *StreamEncoder) WriteCustomHeader(hdr *StreamHeader) error

Write a StreamHeader

type StreamHeader 

type StreamHeader struct {
	Compression      *StreamHeader_Compression `protobuf:"varint,1,opt,name=compression,enum=cypress.StreamHeader_Compression" json:"compression,omitempty"`
	Mode             *StreamHeader_Mode        `protobuf:"varint,2,opt,name=mode,enum=cypress.StreamHeader_Mode" json:"mode,omitempty"`
	XXX_unrecognized []byte                    `json:"-" codec:"-"`
}

func (*StreamHeader) Equal 

func (this *StreamHeader) Equal(that interface{}) bool

func (*StreamHeader) GetCompression 

func (m *StreamHeader) GetCompression() StreamHeader_Compression

func (*StreamHeader) GetMode 

func (m *StreamHeader) GetMode() StreamHeader_Mode

func (*StreamHeader) Marshal 

func (m *StreamHeader) Marshal() (data []byte, err error)

func (*StreamHeader) MarshalTo 

func (m *StreamHeader) MarshalTo(data []byte) (n int, err error)

func (*StreamHeader) ProtoMessage 

func (*StreamHeader) ProtoMessage()

func (*StreamHeader) Reset 

func (m *StreamHeader) Reset()

func (*StreamHeader) Size 

func (m *StreamHeader) Size() (n int)

func (*StreamHeader) String 

func (m *StreamHeader) String() string

func (*StreamHeader) Unmarshal 

func (m *StreamHeader) Unmarshal(data []byte) error

func (*StreamHeader) UnmarshalFrom 

func (h *StreamHeader) UnmarshalFrom(r io.Reader) error

func (*StreamHeader) VerboseEqual 

func (this *StreamHeader) VerboseEqual(that interface{}) error

type StreamHeader_Compression 

type StreamHeader_Compression int32
const (
	StreamHeader_NONE   StreamHeader_Compression = 0
	StreamHeader_SNAPPY StreamHeader_Compression = 1
	StreamHeader_ZLIB   StreamHeader_Compression = 2
)

func (StreamHeader_Compression) Enum 

func (x StreamHeader_Compression) Enum() *StreamHeader_Compression

func (StreamHeader_Compression) String 

func (x StreamHeader_Compression) String() string

func (*StreamHeader_Compression) UnmarshalJSON 

func (x *StreamHeader_Compression) UnmarshalJSON(data []byte) error

type StreamHeader_Mode 

type StreamHeader_Mode int32
const (
	StreamHeader_RAW      StreamHeader_Mode = 0
	StreamHeader_RELIABLE StreamHeader_Mode = 1
)

func (StreamHeader_Mode) Enum 

func (x StreamHeader_Mode) Enum() *StreamHeader_Mode

func (StreamHeader_Mode) String 

func (x StreamHeader_Mode) String() string

func (*StreamHeader_Mode) UnmarshalJSON 

func (x *StreamHeader_Mode) UnmarshalJSON(data []byte) error

type Stringer 

type Stringer interface {
	String() string
}

Used to convert a valeu into a string

type Tag 

type Tag struct {
	Name             string  `protobuf:"bytes,1,req,name=name" json:"name" codec:"name"`
	Value            *string `protobuf:"bytes,2,opt,name=value" json:"value,omitempty" codec:"value,omitempty"`
	XXX_unrecognized []byte  `json:"-" codec:"-"`
}

func (*Tag) Equal 

func (this *Tag) Equal(that interface{}) bool

func (*Tag) GetName 

func (m *Tag) GetName() string

func (*Tag) GetValue 

func (m *Tag) GetValue() string

func (*Tag) Marshal 

func (m *Tag) Marshal() (data []byte, err error)

func (*Tag) MarshalTo 

func (m *Tag) MarshalTo(data []byte) (n int, err error)

func (*Tag) ProtoMessage 

func (*Tag) ProtoMessage()

func (*Tag) Reset 

func (m *Tag) Reset()

func (*Tag) Size 

func (m *Tag) Size() (n int)

func (*Tag) String 

func (m *Tag) String() string

func (*Tag) Unmarshal 

func (m *Tag) Unmarshal(data []byte) error

func (*Tag) VerboseEqual 

func (this *Tag) VerboseEqual(that interface{}) error

type TestPlugin 

type TestPlugin struct {
	Messages     chan *Message
	FilterFields map[string]interface{}
}

Used for testing only

func (*TestPlugin) Close 

func (t *TestPlugin) Close() error

func (*TestPlugin) Description 

func (t *TestPlugin) Description() string

func (*TestPlugin) Filter 

func (t *TestPlugin) Filter(m *Message) (*Message, error)

func (*TestPlugin) Filterer 

func (t *TestPlugin) Filterer() (Filterer, error)

func (*TestPlugin) Generate 

func (t *TestPlugin) Generate() (*Message, error)

func (*TestPlugin) Generator 

func (t *TestPlugin) Generator() (Generator, error)

func (*TestPlugin) Init 

func (t *TestPlugin) Init()

func (*TestPlugin) Receive 

func (t *TestPlugin) Receive(m *Message) error

func (*TestPlugin) Receiver 

func (t *TestPlugin) Receiver() (Receiver, error)

Source Files

View all Source files

Directories

Path Synopsis
cli
commands
cmd
cypress
plugins
all
elasticsearch
file
file/samefile
geoip
grep
json
lib/tcplog
logentries
loggly
logstash
metrics
papertrail
postgres
s3
spool
statsd
syslog
tcp
Package scanner provides a scanner and tokenizer for UTF-8-encoded text.
 Package scanner provides a scanner and tokenizer for UTF-8-encoded text.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.