veneur: github.com/stripe/veneur/ssf Index | Examples | Files

package ssf

import "github.com/stripe/veneur/ssf"

Package ssf provides an implementation of the Sensor Sensibility Format. It consists of two parts: One is the protobuf implementations of the SSF data structures SSFSpan and SSFSample, and the other is a set of helper routines for generating SSFSamples that can be reported on an SSFSpan.

The types in this package are meant to be used together with the neighboring packages trace and trace/metrics.

Index

Examples

Package Files

doc.go sample.pb.go samples.go

Variables

var (
    ErrInvalidLengthSample = fmt.Errorf("proto: negative length found during unmarshaling")
    ErrIntOverflowSample   = fmt.Errorf("proto: integer overflow")
)
var NamePrefix string

NamePrefix is a string prepended to every SSFSample name generated by the constructors in this package. As no separator is added between this prefix and the metric name, users must take care to attach any separators to the prefix themselves.

var SSFSample_Metric_name = map[int32]string{
    0:  "COUNTER",
    1:  "GAUGE",
    2:  "HISTOGRAM",
    3:  "SET",
    4:  "STATUS",
}
var SSFSample_Metric_value = map[string]int32{
    "COUNTER":   0,
    "GAUGE":     1,
    "HISTOGRAM": 2,
    "SET":       3,
    "STATUS":    4,
}
var SSFSample_Scope_name = map[int32]string{
    0:  "DEFAULT",
    1:  "LOCAL",
    2:  "GLOBAL",
}
var SSFSample_Scope_value = map[string]int32{
    "DEFAULT": 0,
    "LOCAL":   1,
    "GLOBAL":  2,
}
var SSFSample_Status_name = map[int32]string{
    0:  "OK",
    1:  "WARNING",
    2:  "CRITICAL",
    3:  "UNKNOWN",
}
var SSFSample_Status_value = map[string]int32{
    "OK":       0,
    "WARNING":  1,
    "CRITICAL": 2,
    "UNKNOWN":  3,
}

type SSFSample Uses

type SSFSample struct {
    // The underlying type of the metric
    Metric SSFSample_Metric `protobuf:"varint,1,opt,name=metric,proto3,enum=ssf.SSFSample_Metric" json:"metric,omitempty"`
    // no spaces, but . is allowed
    // e.g.: veneur.bar.baz
    Name       string            `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
    Value      float32           `protobuf:"fixed32,3,opt,name=value,proto3" json:"value,omitempty"`
    Timestamp  int64             `protobuf:"varint,4,opt,name=timestamp,proto3" json:"timestamp,omitempty"`
    Message    string            `protobuf:"bytes,5,opt,name=message,proto3" json:"message,omitempty"`
    Status     SSFSample_Status  `protobuf:"varint,6,opt,name=status,proto3,enum=ssf.SSFSample_Status" json:"status,omitempty"`
    SampleRate float32           `protobuf:"fixed32,7,opt,name=sample_rate,json=sampleRate,proto3" json:"sample_rate,omitempty"`
    Tags       map[string]string `protobuf:"bytes,8,rep,name=tags,proto3" json:"tags,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
    Unit       string            `protobuf:"bytes,9,opt,name=unit,proto3" json:"unit,omitempty"`
    // scope indicates to an SSF endpoint what it should do with a metric:
    //
    //     - DEFAULT (or absent) - aggregate counters and gauges locally,
    //       handle histograms and sets globally.
    //     - LOCAL - aggregate all metrics locally.
    //     - GLOBAL - aggregate all metrics globally.
    //
    Scope SSFSample_Scope `protobuf:"varint,10,opt,name=scope,proto3,enum=ssf.SSFSample_Scope" json:"scope,omitempty"`
}

SSFSample is similar of a StatsD-style, point in time metric. It has a Metric type, a name, a value and a timestamp. Additionally it can contain a message, a status, a sample rate, a map of tags as string keys and values and a unit type. Note that SSF doesn't understand units, they are just strings!

func Count Uses

func Count(name string, value float32, tags map[string]string, opts ...SampleOption) *SSFSample

Count returns an SSFSample representing an increment / decrement of a counter. It's a convenience wrapper around constructing SSFSample objects.

func Gauge Uses

func Gauge(name string, value float32, tags map[string]string, opts ...SampleOption) *SSFSample

Gauge returns an SSFSample representing a gauge at a certain value. It's a convenience wrapper around constructing SSFSample objects.

func Histogram Uses

func Histogram(name string, value float32, tags map[string]string, opts ...SampleOption) *SSFSample

Histogram returns an SSFSample representing a value on a histogram, like a timer or other range. It's a convenience wrapper around constructing SSFSample objects.

func RandomlySample Uses

func RandomlySample(rate float32, samples ...*SSFSample) []*SSFSample

RandomlySample takes a rate and a set of measurements, and returns a new set of measurements as if sampling had been performed: Each original measurement gets rejected/included in the result based on a random roll of the RNG according to the rate, and each included measurement has its SampleRate field adjusted to be its original SampleRate * rate.

Code:

samples := &ssf.Samples{}
// Sample some metrics at 50% - each of these metrics, if it
// gets picked, will report with a SampleRate of 0.5:
samples.Add(ssf.RandomlySample(0.5,
    ssf.Count("cheap.counter", 1, nil),
    ssf.Timing("cheap.timer", 1*time.Second, time.Nanosecond, nil),
)...)

// Sample another metric at 1% - if included, the metric will
// have a SampleRate of 0.01:
samples.Add(ssf.RandomlySample(0.01,
    ssf.Count("expensive.counter", 20, nil))...)

// Report these metrics:
metrics.Report(trace.DefaultClient, samples)

func Set Uses

func Set(name string, value string, tags map[string]string, opts ...SampleOption) *SSFSample

Set returns an SSFSample representing a value on a set, useful for counting the unique values that occur in a certain time bound.

func Status Uses

func Status(name string, state SSFSample_Status, tags map[string]string, opts ...SampleOption) *SSFSample

Status returns an SSFSample capturing the reported state of a service

func Timing Uses

func Timing(name string, value time.Duration, resolution time.Duration, tags map[string]string, opts ...SampleOption) *SSFSample

Timing returns an SSFSample (really a histogram) representing the timing in the given resolution.

func (*SSFSample) Descriptor Uses

func (*SSFSample) Descriptor() ([]byte, []int)

func (*SSFSample) GetMessage Uses

func (m *SSFSample) GetMessage() string

func (*SSFSample) GetMetric Uses

func (m *SSFSample) GetMetric() SSFSample_Metric

func (*SSFSample) GetName Uses

func (m *SSFSample) GetName() string

func (*SSFSample) GetSampleRate Uses

func (m *SSFSample) GetSampleRate() float32

func (*SSFSample) GetScope Uses

func (m *SSFSample) GetScope() SSFSample_Scope

func (*SSFSample) GetStatus Uses

func (m *SSFSample) GetStatus() SSFSample_Status

func (*SSFSample) GetTags Uses

func (m *SSFSample) GetTags() map[string]string

func (*SSFSample) GetTimestamp Uses

func (m *SSFSample) GetTimestamp() int64

func (*SSFSample) GetUnit Uses

func (m *SSFSample) GetUnit() string

func (*SSFSample) GetValue Uses

func (m *SSFSample) GetValue() float32

func (*SSFSample) Marshal Uses

func (m *SSFSample) Marshal() (dAtA []byte, err error)

func (*SSFSample) MarshalTo Uses

func (m *SSFSample) MarshalTo(dAtA []byte) (int, error)

func (*SSFSample) ProtoMessage Uses

func (*SSFSample) ProtoMessage()

func (*SSFSample) Reset Uses

func (m *SSFSample) Reset()

func (*SSFSample) Size Uses

func (m *SSFSample) Size() (n int)

func (*SSFSample) String Uses

func (m *SSFSample) String() string

func (*SSFSample) Unmarshal Uses

func (m *SSFSample) Unmarshal(dAtA []byte) error

func (*SSFSample) XXX_DiscardUnknown Uses

func (m *SSFSample) XXX_DiscardUnknown()

func (*SSFSample) XXX_Marshal Uses

func (m *SSFSample) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*SSFSample) XXX_Merge Uses

func (m *SSFSample) XXX_Merge(src proto.Message)

func (*SSFSample) XXX_Size Uses

func (m *SSFSample) XXX_Size() int

func (*SSFSample) XXX_Unmarshal Uses

func (m *SSFSample) XXX_Unmarshal(b []byte) error

type SSFSample_Metric Uses

type SSFSample_Metric int32
const (
    SSFSample_COUNTER   SSFSample_Metric = 0
    SSFSample_GAUGE     SSFSample_Metric = 1
    SSFSample_HISTOGRAM SSFSample_Metric = 2
    SSFSample_SET       SSFSample_Metric = 3
    SSFSample_STATUS    SSFSample_Metric = 4
)

func (SSFSample_Metric) EnumDescriptor Uses

func (SSFSample_Metric) EnumDescriptor() ([]byte, []int)

func (SSFSample_Metric) String Uses

func (x SSFSample_Metric) String() string

type SSFSample_Scope Uses

type SSFSample_Scope int32
const (
    SSFSample_DEFAULT SSFSample_Scope = 0
    SSFSample_LOCAL   SSFSample_Scope = 1
    SSFSample_GLOBAL  SSFSample_Scope = 2
)

func (SSFSample_Scope) EnumDescriptor Uses

func (SSFSample_Scope) EnumDescriptor() ([]byte, []int)

func (SSFSample_Scope) String Uses

func (x SSFSample_Scope) String() string

type SSFSample_Status Uses

type SSFSample_Status int32
const (
    SSFSample_OK       SSFSample_Status = 0
    SSFSample_WARNING  SSFSample_Status = 1
    SSFSample_CRITICAL SSFSample_Status = 2
    SSFSample_UNKNOWN  SSFSample_Status = 3
)

func (SSFSample_Status) EnumDescriptor Uses

func (SSFSample_Status) EnumDescriptor() ([]byte, []int)

func (SSFSample_Status) String Uses

func (x SSFSample_Status) String() string

type SSFSpan Uses

type SSFSpan struct {
    Version int32 `protobuf:"varint,1,opt,name=version,proto3" json:"version,omitempty"`
    // the trace_id is the (span) id of the root span
    TraceId int64 `protobuf:"varint,2,opt,name=trace_id,json=traceId,proto3" json:"trace_id,omitempty"`
    // the id for this span
    Id  int64 `protobuf:"varint,3,opt,name=id,proto3" json:"id,omitempty"`
    // the (span) id of the direct parent, if this span is not a root
    // span
    ParentId       int64 `protobuf:"varint,4,opt,name=parent_id,json=parentId,proto3" json:"parent_id,omitempty"`
    StartTimestamp int64 `protobuf:"varint,5,opt,name=start_timestamp,json=startTimestamp,proto3" json:"start_timestamp,omitempty"`
    EndTimestamp   int64 `protobuf:"varint,6,opt,name=end_timestamp,json=endTimestamp,proto3" json:"end_timestamp,omitempty"`
    // This flag being true signals that this span was an error. That definition
    // of error is not implicitly fatal, as a span may error but be fixed by
    // a subsequent retry, etc.
    Error bool `protobuf:"varint,7,opt,name=error,proto3" json:"error,omitempty"`
    // The name of the service
    // e.g. "veneur"
    Service string       `protobuf:"bytes,8,opt,name=service,proto3" json:"service,omitempty"`
    Metrics []*SSFSample `protobuf:"bytes,10,rep,name=metrics,proto3" json:"metrics,omitempty"`
    // Tags are name value pairs that describe a facet of the span. They apply to
    // the *entire* span as opposed to logs which apply to a specific time in
    // the span.
    Tags map[string]string `protobuf:"bytes,11,rep,name=tags,proto3" json:"tags,omitempty" protobuf_key:"bytes,1,opt,name=key,proto3" protobuf_val:"bytes,2,opt,name=value,proto3"`
    // An indicator span is one that represents an action that is included in a
    // service's Service Level Indicators (https://en.wikipedia.org/wiki/Service_level_indicator)
    // This is a signal to receivers that this span may be used to compute SLIs.
    // In practice a service's core feature — the thing you would "bill" for, such
    // as an API call or read/write operation — would be flagged as an indicator
    // span, and its child spans would further describe its duration.
    // It's also worth nothing that an indicator need not be the "root" or first
    // span in a trace. You might have various forms of middleware that happen
    // first or you might have multiple services participating in the same trace.
    Indicator bool `protobuf:"varint,12,opt,name=indicator,proto3" json:"indicator,omitempty"`
    // What to call this span. This could take the form of the endpoint
    // (/customer/:id), the function (class::name.method), a friendly name
    // (foo middleware) or whatever makes sense in your context.
    Name string `protobuf:"bytes,13,opt,name=name,proto3" json:"name,omitempty"`
}

SSFSpan is the primary unit of reporting in SSF. It embeds a set of SSFSamples, as well as start/stop time stamps and a parent ID (which allows assembling a span lineage for distributed tracing purposes).

Note that since this is protobuf, an SSFSpan does not *have* to include metrics, just as it does not *have* to include information necessary to reconstruct a trace.

Compatibility

On ingestion, an SSFSpan with an empty string for a name field but a tag "name" will have that name field replaced with the name tag, and the tag is removed.

Metric SSFSamples with a zero sample_rate (indicating it was left out) have the sample_rate field set to 1 on ingestion.

Validity Criteria

Programs consuming SSFSpans should take care to only process spans and metrics that fulfill the following criteria:

Metrics are considered valid if they have a name and a value.

SSFSpans are considered valid trace spans if they have non-zero id, trace_id, start_timestamp and end_timestamp fields.

func (*SSFSpan) Descriptor Uses

func (*SSFSpan) Descriptor() ([]byte, []int)

func (*SSFSpan) GetEndTimestamp Uses

func (m *SSFSpan) GetEndTimestamp() int64

func (*SSFSpan) GetError Uses

func (m *SSFSpan) GetError() bool

func (*SSFSpan) GetId Uses

func (m *SSFSpan) GetId() int64

func (*SSFSpan) GetIndicator Uses

func (m *SSFSpan) GetIndicator() bool

func (*SSFSpan) GetMetrics Uses

func (m *SSFSpan) GetMetrics() []*SSFSample

func (*SSFSpan) GetName Uses

func (m *SSFSpan) GetName() string

func (*SSFSpan) GetParentId Uses

func (m *SSFSpan) GetParentId() int64

func (*SSFSpan) GetService Uses

func (m *SSFSpan) GetService() string

func (*SSFSpan) GetStartTimestamp Uses

func (m *SSFSpan) GetStartTimestamp() int64

func (*SSFSpan) GetTags Uses

func (m *SSFSpan) GetTags() map[string]string

func (*SSFSpan) GetTraceId Uses

func (m *SSFSpan) GetTraceId() int64

func (*SSFSpan) GetVersion Uses

func (m *SSFSpan) GetVersion() int32

func (*SSFSpan) Marshal Uses

func (m *SSFSpan) Marshal() (dAtA []byte, err error)

func (*SSFSpan) MarshalTo Uses

func (m *SSFSpan) MarshalTo(dAtA []byte) (int, error)

func (*SSFSpan) ProtoMessage Uses

func (*SSFSpan) ProtoMessage()

func (*SSFSpan) Reset Uses

func (m *SSFSpan) Reset()

func (*SSFSpan) Size Uses

func (m *SSFSpan) Size() (n int)

func (*SSFSpan) String Uses

func (m *SSFSpan) String() string

func (*SSFSpan) Unmarshal Uses

func (m *SSFSpan) Unmarshal(dAtA []byte) error

func (*SSFSpan) XXX_DiscardUnknown Uses

func (m *SSFSpan) XXX_DiscardUnknown()

func (*SSFSpan) XXX_Marshal Uses

func (m *SSFSpan) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*SSFSpan) XXX_Merge Uses

func (m *SSFSpan) XXX_Merge(src proto.Message)

func (*SSFSpan) XXX_Size Uses

func (m *SSFSpan) XXX_Size() int

func (*SSFSpan) XXX_Unmarshal Uses

func (m *SSFSpan) XXX_Unmarshal(b []byte) error

type SampleOption Uses

type SampleOption func(*SSFSample)

SampleOption is a functional option that can be used for less commonly needed fields in sample creation helper functions. The options are applied by order of arguments (left to right), so when setting multiple of the same option, the rightmost wins.

func SampleRate Uses

func SampleRate(rate float32) SampleOption

SampleRate sets the rate at which a measurement is sampled. The rate is a number on the interval (0..1] (1 means that the value is not sampled). Any numbers outside this interval result in no change to the sample rate (by default, all SSFSamples created with the helpers in this package have a SampleRate=1).

func Scope Uses

func Scope(scope SampleScope) SampleOption

Scope is a SampleOption that sets the scope of a metric to be either "global" (i.e., aggregated on a central node), or "local" (i.e., aggregated exclusively on the central node).

func TimeUnit Uses

func TimeUnit(resolution time.Duration) SampleOption

TimeUnit sets the unit on a sample to the given resolution's SI unit symbol. Valid resolutions are the time duration constants from Nanosecond through Hour. The non-SI units "minute" and "hour" are represented by "min" and "h" respectively.

If a resolution is passed that does not correspond exactly to the duration constants in package time, this option does not affect the sample at all.

func Timestamp Uses

func Timestamp(ts time.Time) SampleOption

Timestamp is a functional option for creating an SSFSample. It sets the timestamp field on the sample to the timestamp passed.

func Unit Uses

func Unit(name string) SampleOption

Unit is a functional option for creating an SSFSample. It sets the sample's unit name to the name passed.

type SampleScope Uses

type SampleScope SSFSample_Scope

SampleScope is a slightly more ergonomic representation of the internal type SSFSample_Scope.

const (
    DefaultScope SampleScope = SampleScope(SSFSample_DEFAULT)
    Local        SampleScope = SampleScope(SSFSample_LOCAL)
    Global       SampleScope = SampleScope(SSFSample_GLOBAL)
)

type Samples Uses

type Samples struct {
    Batch []*SSFSample
}

Samples is a batch of SSFSamples, not attached to an SSF span, that can be submitted with package metrics's Report function.

func (*Samples) Add Uses

func (s *Samples) Add(sample ...*SSFSample)

Add appends a sample to the batch of samples.

Package ssf imports 8 packages (graph) and is imported by 41 packages. Updated 2019-08-11. Refresh now. Tools for package owners.