Documentation
¶
Overview ¶
Package metricsbp provides metrics related features for baseplate.go, based on go-kit metrics package.
There are three main parts of this package:
1. Wrappers of go-kit metrics to provide easy to use create on-the-fly metrics, similar to what we have in Baseplate.py.
2. Helper function for use cases of pre-create the metrics before using them.
3. Sampled counter/histogram implementations.
This package comes with benchmark test to show the performance difference between pre-created metrics, on-the-fly metrics, and on-the-fly with additional tags metrics:
$ go test -bench . -benchmem goos: darwin goarch: amd64 pkg: github.com/reddit/baseplate.go/metricsbp BenchmarkStatsd/pre-create/histogram-8 8583646 124 ns/op 44 B/op 0 allocs/op BenchmarkStatsd/pre-create/timing-8 10221859 120 ns/op 47 B/op 0 allocs/op BenchmarkStatsd/pre-create/counter-8 10205341 120 ns/op 47 B/op 0 allocs/op BenchmarkStatsd/pre-create/gauge-8 96462238 12.4 ns/op 0 B/op 0 allocs/op BenchmarkStatsd/on-the-fly/histogram-8 4665778 256 ns/op 99 B/op 2 allocs/op BenchmarkStatsd/on-the-fly/timing-8 4784816 273 ns/op 126 B/op 2 allocs/op BenchmarkStatsd/on-the-fly/counter-8 4818908 259 ns/op 125 B/op 2 allocs/op BenchmarkStatsd/on-the-fly/gauge-8 28754060 40.6 ns/op 0 B/op 0 allocs/op BenchmarkStatsd/on-the-fly-with-tags/histogram-8 2624264 453 ns/op 192 B/op 4 allocs/op BenchmarkStatsd/on-the-fly-with-tags/timing-8 2639377 449 ns/op 192 B/op 4 allocs/op BenchmarkStatsd/on-the-fly-with-tags/counter-8 2600418 457 ns/op 193 B/op 4 allocs/op BenchmarkStatsd/on-the-fly-with-tags/gauge-8 3429901 339 ns/op 112 B/op 3 allocs/op PASS ok github.com/reddit/baseplate.go/metricsbp 18.675s
Index ¶
- Constants
- Variables
- func BoolString(b bool) string
- func CheckNilFields(root interface{}) []string
- func Float64Ptr(v float64) *float64
- func InitFromConfig(ctx context.Context, cfg Config) io.Closer
- func LogWrapper(args LogWrapperArgs) log.Wrapper
- type Config
- type CreateServerSpanHook
- type LogWrapperArgs
- type RateArgs
- type SampledCounter
- type SampledHistogram
- type Statsd
- func (st *Statsd) Close() error
- func (st *Statsd) Counter(name string) metrics.Counter
- func (st *Statsd) CounterWithRate(args RateArgs) metrics.Counter
- func (st *Statsd) Ctx() context.Context
- func (st *Statsd) Gauge(name string) metrics.Gauge
- func (st *Statsd) Histogram(name string) metrics.Histogram
- func (st *Statsd) HistogramWithRate(args RateArgs) metrics.Histogram
- func (st *Statsd) RunSysStats()
- func (st *Statsd) RuntimeGauge(name string) metrics.Gauge
- func (st *Statsd) Timing(name string) metrics.Histogram
- func (st *Statsd) TimingWithRate(args RateArgs) metrics.Histogram
- func (st *Statsd) WriteTo(w io.Writer) (n int64, err error)
- type Tags
- type Timer
Examples ¶
Constants ¶
const ( // DefaultSampleRate is the default value to be used when *SampleRate in // Config is nil (zero value). DefaultSampleRate = 1 // DefaultBufferSize is the default value to be used when BufferSize in // Config is 0. DefaultBufferSize = 4096 )
Default values to be used in the config.
Variables ¶
var M = NewStatsd(context.Background(), Config{})
M is short for "Metrics".
This is the global Statsd to use. It's pre-initialized with one that sends all metrics to a blackhole.
var ReporterTickerInterval = time.Minute
ReporterTickerInterval is the interval the reporter sends data to statsd server. Default is one minute.
var SysStatsTickerInterval = time.Second * 10
SysStatsTickerInterval is the interval we pull and report sys stats. Default is 10 seconds.
Functions ¶
func BoolString ¶ added in v0.9.5
BoolString returns the string version of a boolean value that should be used in a statsd metric tag.
func CheckNilFields ¶
func CheckNilFields(root interface{}) []string
CheckNilFields returns all the nil value fields inside root.
root should be a value to a struct, or a pointer to a struct, otherwise this function will panic. The return value would be the field names of all the uninitialized fields recursively.
For example, for the following code:
type Bar struct {
A io.Reader
B io.Reader
c io.Reader
D struct{
A io.Reader
B io.Reader
}
}
func main() {
fields := CheckNilInterfaceFields(
&Bar{
A: strings.NewReader("foo"),
D: {
B: bytes.NewReader([]bytes("bar")),
},
},
)
}
fields should contain 3 strings: "Bar.B", "Bar.c", and "Bar.D.A".
Special case: When root itself is nil, or pointer to nil pointer, a single, empty string ("") will be returned.
The main use case of this function is for pre-created metrics. A common pattern is to define a struct contains all the metrics, initialize them in main function, then pass the struct down to the handler functions to use. It has better performance over creating metrics on the fly when using, but comes with a down side that if you added a new metric to the struct but forgot to initialize it in the main function, the code would panic when it's first used.
Use this function to check the metrics struct after initialization could help you panic earlier and in a more predictable way.
Example ¶
This example demonstrate how to use CheckNilFields in your microservice code to pre-create frequently used metrics.
package main
import (
"context"
"fmt"
"github.com/reddit/baseplate.go/metricsbp"
"github.com/go-kit/kit/metrics"
)
type SubMetrics struct {
MyHistogram metrics.Histogram
MyGauge metrics.Gauge
}
type PreCreatedMetrics struct {
MyCounter metrics.Counter
MySubMetrics SubMetrics
}
// This example demonstrate how to use CheckNilFields in your microservice code
// to pre-create frequently used metrics.
func main() {
// In reality these should come from flag or other configurations.
const (
prefix = "myservice"
statsdAddr = "localhost:1234"
sampleRate = 1
)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
metricsbp.M = metricsbp.NewStatsd(
ctx,
metricsbp.Config{
Namespace: prefix,
Endpoint: statsdAddr,
HistogramSampleRate: metricsbp.Float64Ptr(sampleRate),
},
)
// Initialize metrics
m := PreCreatedMetrics{
MyCounter: metricsbp.M.Counter("my.counter"),
MySubMetrics: SubMetrics{
MyHistogram: metricsbp.M.Histogram("my.histogram"),
// Forgot to initialize MyGauge here
},
}
missingFields := metricsbp.CheckNilFields(m)
if len(missingFields) > 0 {
panic(fmt.Sprintf("Uninitialized metrics: %v", missingFields))
}
// Other initializations.
// Replace with your actual service starter
startService := func(m PreCreatedMetrics /* and other args */) {}
startService(
m,
// other args
)
}
Output:
func Float64Ptr ¶
Float64Ptr converts float64 value into pointer.
func InitFromConfig ¶
InitFromConfig initializes the global metricsbp.M with the given context and Config and returns an io.Closer to use to close out the metrics client when your server exits.
It also registers CreateServerSpanHook and ConcurrencyCreateServerSpanHook with the global tracing hook registry.
func LogWrapper ¶ added in v0.6.0
func LogWrapper(args LogWrapperArgs) log.Wrapper
LogWrapper creates a log.Wrapper implementation with metrics emitting.
Types ¶
type Config ¶
type Config struct {
// Namespace is the standard prefix applied to all of your metrics, it should
// include the name of your service.
Namespace string `yaml:"namespace"`
// Endpoint is the endpoint for your metrics backend.
Endpoint string `yaml:"endpoint"`
// Tags are the base tags that will be applied to all metrics.
Tags Tags `yaml:"tags"`
// HistogramSampleRate is the fraction of histograms (including timings) that
// you want to send to your metrics backend.
//
// Optional, defaults to 1 (100%).
HistogramSampleRate *float64 `yaml:"histogramSampleRate"`
// When Endpoint is configured,
// BufferSize can be used to buffer writes to statsd collector together.
//
// Set it to an appropriate number will reduce the number of UDP messages sent
// to the statsd collector. (Recommendation: 4KB)
//
// It's guaranteed that every single UDP message will not exceed BufferSize,
// unless a single metric line exceeds it
// (usually around 100 bytes depending on the length of the metric path).
//
// When it's 0 (default), DefaultBufferSize will be used.
//
// To disable buffering, set it to negative value (e.g. -1) or a value smaller
// than a single statsd metric line (usually around 100, so 1 works),
// on ReporterTickerInterval we will write one UDP message per metric to the
// statsd collector.
BufferSize int `yaml:"bufferSize"`
// The log level used by the reporting goroutine.
LogLevel log.Level `yaml:"logLevel"`
// RunSysStats indicates that you want to publish system stats.
//
// Optional, default to false.
RunSysStats bool `yaml:"runSysStats"`
// By default, when Endpoint is empty,
// the Statsd uses a blackhole sink to send statsd metrics to.
// Set this to true to buffer all statsd metrics in memory until they are read
// explicitly (via Statsd.WriteTo).
//
// This is provided only for tests to verify the statsd metrics.
// Using it in production code with empty Endpoint will cause the memory used
// by Statsd to grow unbounded over the time.
// It also has no effects when Endpoint is non-empty.
//
// Optional, default to false.
BufferInMemoryForTesting bool `yaml:"-"`
}
Config is the confuration struct for the metricsbp package.
Can be deserialized from YAML.
type CreateServerSpanHook ¶
type CreateServerSpanHook struct {
// Optional, will fallback to M when it's nil.
Metrics *Statsd
}
CreateServerSpanHook registers each Server Span with a MetricsSpanHook.
Example ¶
This example demonstrates how to use CreateServerSpanHook.
package main
import (
"github.com/reddit/baseplate.go/metricsbp"
"github.com/reddit/baseplate.go/tracing"
)
func main() {
// initialize the CreateServerSpanHook
hook := metricsbp.CreateServerSpanHook{}
// register the hook with Baseplate
tracing.RegisterCreateServerSpanHooks(hook)
}
Output:
func (CreateServerSpanHook) OnCreateServerSpan ¶
func (h CreateServerSpanHook) OnCreateServerSpan(span *tracing.Span) error
OnCreateServerSpan registers MetricSpanHooks on a server Span.
type LogWrapperArgs ¶ added in v0.6.0
type LogWrapperArgs struct {
// The metrics path of the counter.
//
// Optional. If it's non-empty,
// every time the generaged log.Wrapper is called, counter will be added by 1.
Counter string
// Additional tags to be applied to the metrics, optional.
Tags Tags
// Statsd to use.
//
// Optional. If this is nil, metricsbp.M will be used instead.
Statsd *Statsd
// The base log.Wrapper implementation.
//
// Optional. If this is nil,
// then LogWrapper implementation will only emit metrics without logging.
Wrapper log.Wrapper
}
LogWrapperArgs defines the args used by LogWrapper.
type RateArgs ¶ added in v0.8.0
type RateArgs struct {
// Name of the metric, required.
Name string
// Sampling rate, required.
//
// If AlreadySampledAt is nil,
// this controls both the sample rate reported to statsd and the rate we use
// for reporting the metrics.
//
// If AlreadySampledAt is non-nil,
// The sample rate reported to statsd will be Rate*AlreadySampledAt,
// and Rate controls how we randomly report this metric.
// It's useful when you are reporting a metric from an already sampled code
// block, for example:
//
// const rate = 0.01
// if randbp.ShouldSampleWithRate(rate) {
// if err := myFancyWork(); err != nil {
// metricsbp.M.CounterWithRate(metricsbp.RateArgs{
// Name: "my.fancy.work.errors",
// // 100% report it because we are already sampling it.
// Rate: 1,
// // but adjust the reporting rate to the actual sample rate.
// AlreadySampledAt: metricsbp.Float64Ptr(rate),
// }).Add(1)
// }
// }
Rate float64
// Optional. Default to 1 (100%) if it's nil.
// See the comment on Rate for more details.
//
// It will be treated as 1 if >=1, and be trated as 0 if <=0.
AlreadySampledAt *float64
}
RateArgs defines the args used by -WithRate functions.
func (RateArgs) ReportingRate ¶ added in v0.8.0
ReportingRate returns the reporting rate according to the args.
type SampledCounter ¶
SampledCounter is a metrics.Counter implementation that actually sample the Add calls.
func (SampledCounter) Add ¶
func (c SampledCounter) Add(delta float64)
Add implements metrics.Counter.
type SampledHistogram ¶
SampledHistogram is a metrics.Histogram implementation that actually sample the Observe calls.
func (SampledHistogram) Observe ¶
func (h SampledHistogram) Observe(value float64)
Observe implements metrics.Histogram.
type Statsd ¶
type Statsd struct {
// contains filtered or unexported fields
}
Statsd defines a statsd reporter (with influx extension) and the root of the metrics.
It can be used to create metrics, and also maintains the background reporting goroutine,
It supports metrics tags in Influxstatsd format.
Please use NewStatsd to initialize it.
When a *Statsd is nil, any function calls to it will fallback to use M instead, so they are gonna be safe to use (unless M was explicitly overridden as nil). For example:
st := (*metricsbp.Statsd)(nil)
st.Counter("my-counter").Add(1) // does not panic unless metricsbp.M is nil
func NewStatsd ¶
NewStatsd creates a Statsd object.
It also starts a background reporting goroutine when Endpoint is not empty. The goroutine will be stopped when the passed in context is canceled.
NewStatsd never returns nil.
func (*Statsd) Close ¶
Close flushes all metrics not written to collector (if Endpoint was set), and cancel the context, thus stop all background goroutines started by this Statsd.
It blocks until the flushing is completed.
After Close() is called, no more metrics will be send to the remote collector, similar to the situation that this Statsd was initialized without Endpoint set.
After Close() is called, Ctx() will always return an already canceled context.
This function is useful for jobs that exit, to make sure that all metrics are flushed before exiting. For server code, there's usually no need to call Close(), just cancel the context object passed in is sufficient. But server code can also choose to pass in a background context, and use Close() call to do the cleanup instead of canceling the context.
func (*Statsd) Counter ¶
Counter returns a counter metrics to the name, with sample rate inherited from Config.
func (*Statsd) CounterWithRate ¶ added in v0.2.0
CounterWithRate returns a counter metrics to the name, with sample rate passed in instead of inherited from Config.
func (*Statsd) Ctx ¶
Ctx provides a read-only access to the context object this Statsd holds.
It's useful when you need to implement your own goroutine to report some metrics (usually gauges) periodically, and be able to stop that goroutine gracefully. For example:
func reportGauges() {
gauge := metricsbp.M.Gauge("my-gauge")
go func() {
ticker := time.NewTicker(time.Minute)
defer ticker.Stop()
for {
select {
case <- metricsbp.M.Ctx().Done():
return
case <- ticker.C:
gauge.Set(getValue())
}
}
}
}
func (*Statsd) Gauge ¶
Gauge returns a gauge metrics to the name.
Please note that gauges are considered "low level". In most cases when you use a Gauge, you want to use RuntimeGauge instead.
func (*Statsd) Histogram ¶
Histogram returns a histogram metrics to the name with no specific unit, with sample rate inherited from Config.
func (*Statsd) HistogramWithRate ¶ added in v0.2.0
HistogramWithRate returns a histogram metrics to the name with no specific unit, with sample rate passed in instead of inherited from Config.
func (*Statsd) RunSysStats ¶
func (st *Statsd) RunSysStats()
RunSysStats starts a goroutine to periodically pull and report sys stats.
All the sys stats will be reported as RuntimeGauges.
Canceling the context passed into NewStatsd will stop this goroutine.
func (*Statsd) RuntimeGauge ¶ added in v0.4.1
RuntimeGauge returns a Gauge that's suitable to report runtime data.
It will be applied with "runtime." prefix and instance+pid tags automatically.
All gauges reported from RunSysStats are runtime gauges.
func (*Statsd) Timing ¶
Timing returns a histogram metrics to the name with milliseconds as the unit, with sample rate inherited from Config.
func (*Statsd) TimingWithRate ¶ added in v0.2.0
TimingWithRate returns a histogram metrics to the name with milliseconds as the unit, with sample rate passed in instead of inherited from Config.
func (*Statsd) WriteTo ¶ added in v0.2.1
WriteTo calls the underlying statsd implementation's WriteTo function.
Doing this will flush all the buffered metrics to the writer, so in most cases you shouldn't be using it in production code. But it's useful in unit tests to verify that you have the correct metrics you want to report.
When you use this in test code you also want to set BufferInMemoryForTesting to true in the statsd Config, otherwise your test could become flaky.
type Tags ¶ added in v0.4.0
Tags allows you to specify tags as a convenient map and provides helpers to convert them into other formats.
func (Tags) AsStatsdTags ¶ added in v0.4.0
AsStatsdTags returns the tags in the format expected by the statsd metrics client, that is a slice of strings.
This method is nil-safe and will just return nil if the receiver is nil.
type Timer ¶
Timer is a timer wraps a histogram.
It's very similar to go-kit's Timer, with a few differences:
1. The reporting unit is millisecond and non-changeable.
2. It's nil-safe (zero values of *Timer or Timer will be safe to call, but they are no-ops)
Example ¶
This example demonstrates how to use Timer.
package main
import (
"context"
"github.com/reddit/baseplate.go/metricsbp"
)
// This example demonstrates how to use Timer.
func main() {
type timerContextKeyType struct{}
// variables should be properly initialized in production code
var (
ctx context.Context
timerContextKey timerContextKeyType
)
const metricsPath = "dummy.call.timer"
// initialize and inject a timer into context
ctx = context.WithValue(
ctx,
timerContextKey,
metricsbp.NewTimer(metricsbp.M.Timing(metricsPath)),
)
// do the work
dummyCall(ctx)
// get the timer out of context and report
if t, ok := ctx.Value(timerContextKey).(*metricsbp.Timer); ok {
t.ObserveDuration()
}
}
func dummyCall(_ context.Context) {}
Output:
func (*Timer) ObserveDuration ¶
ObserveDuration reports the time elapsed via the wrapped histogram.
This is a shortcut for:
t.ObserveWithEndTime(time.Now())
If either t or *t is zero value, it will be no-op.
The reporting unit is millisecond.
It returns self for chaining.
func (*Timer) ObserveWithEndTime ¶ added in v0.7.0
ObserveWithEndTime reports the time elapsed via the wrapped histogram.
If either t or *t is zero value, it will be no-op.
The reporting unit is millisecond.
It returns self for chaining.
func (*Timer) OverrideStartTime ¶ added in v0.7.0
OverrideStartTime overrides the start time for the Timer.
If t is nil, it will be no-op.
It returns self for chaining.
Source Files