README
¶
Lightstep implementation of the OpenTelemetry-Go Metrics SDK
This implementation began as a prototype implementation of the OpenTelemetry SDK specification and has been used as a reference point for the OpenTelemetry-Go community metrics SDK. Lightstep's internal code base is largely written in Go and has a number of specific requirements which made a direct upgrade to the OpenTelemetry-Go community Metrics SDK difficult.
Instead of waiting for the OpenTelemetry-Go community SDK to surpass the current OpenTelemetry specification, to support our requirements, we decided to enable the experimental features in an SDK we control.
This implementation of the OpenTelemetry SDK has been made public, however it is not covered by any stability guarantee.
Differences from the OpenTelemetry metrics SDK specification:
- ExponentialHistogram is the default aggregation for Histogram instruments. The explicit-boundary histogram aggregation is not supported.
- MinMaxSumCount is an
optional aggregation for Histogram instruments that encodes a
zero-bucket explicit-boundary histogram data
point.
Note that this aggregation only encodes the
.Minand.Maxfields when configured with delta temporality. Consider using the "lowmemory" temporality preference in the launcher.. - Synchronous Gauge instrument behavior is supported using an API hint.
- The OTLP exporter is the only provided exporter. The OTLP exporter
is based on the OTel-Core
batchprocessorand the OTel-Arrowotlpexportercollector components.
These differences aside, this SDK features a complete implementation of the OpenTelemetry SDK specification with support for multiple readers. It is possible, for example, to configure multiple OTLP exporters with different views and destinations.
Lightstep expects to continue maintaining this implementation until the community SDK supports configuring the behaviors listed above. Moreover, Lightstep expects to make several optimizations in this SDK to further optimize the synchronous instrument fast path and continue improving memory performance.
Using the Lightstep Metrics SDK directly
See the example in examples/example_test.go for an example using this SDK with minimum configuration.
Metric instrument "Hints" API
There is a standing feature request in OpenTelemetry for a "Hints" API to inform the SDK of recommended aggregations in the source, when registering instruments. This SDK implements an experimental form of Hints API, described as follows.
The Views implementation attempts to parse the Description of each
metric instrument as the JSON-encoded form of a
(github.com/lightstep/otel-launcher-go/lightstep/sdk/metric/view).Hint
structure. If successfully parsed, the embedded aggrgation kind and
configuration will be used, and the embedded Description field
replaces the original hint.
For example, to set the number of exponential histogram buckets, use a description like this:
{
"description": "measurement of ...",
"config": {
"histogram": {
"max_size": 320
}
}
}
To set the MinMaxSumCount aggregation for a specific histogram instrument:
{
"description": "measurement of ...",
"aggregation": "minmaxsumcount"
}
To override the Temporality selected for a specific instrument:
{
"description": "measurement of ...",
"temporality": "delta"
}
Synchronous Gauge instrument
OpenTelemetry metrics API does not support a synchronous Gauge instrument, however the desired semantics are fairly clear. This SDK supports the intended behavior of a synchronous Gauge instrument by distinguishing two possible behaviors as a function of the configured temporality.
Although the Gauge data point does not have the concept of temporality itself, the decision to report or not report a Gauge data point has traditionally been made in one of two ways:
- When the output system is generally expecting cumulative Counters (as in Prometheus), it is traditional to report the latest Gauge value indefinitely, even when the instrument and attribute set are not used again.
- When the output system is generally expecting delta Counters (as in Statsd), it is traditional to report Gauge values at most once.
Therefore, when the Temporality selector for the instrument returns Delta and the aggregation is a Gauge (which is only possible with a Hint, at this time), the resulting instrument will be a synchronous Gauge instrument.
For example, to configure a synchronous Gauge:
gauge, _ := meter.SyncUpDownCounter(
"some_gauge",
instrument.WithDescription(`{"aggregation": "gauge"}`),
)
Note that the API hint for Gauge aggregation can be combined with the API hint for temporality, allowing control over Gauge behavior independent of the default Temporality choice for UpDownCounter instruments.
Performance settings
The WithPerformance() option supports control over performance
settings.
IgnoreCollisions
With IgnoreCollisions set to true, the SDK will ignore fingerprint
collisions and bypass a safety mechanism that ensures correctness in
spite of fingerprint collisions in the fast path for synchronous
instruments.
InactiveCollectionPeriods
The InactiveCollectionPeriods setting controls how many Collect()
cycles with no updates are allowed before a record is removed from
intermediate state. Larger values indicate more memory will be used
and that callers are less likely to block while creating new
aggregators.
Setting this field to 1 means records will be removed from memory after one inactive collection cycle.
Setting this field to 0 causes the default value 10 to be used.
InstrumentCardinalityLimit
Synchronous instruments are implemented using a map of intermediate
state. When this map grows to InstrumentCardinalityLimit, new
attribute sets will be replaced by the overflow attribute set, which
is { otel.metric.overflow=true }. This limit is applied to all
instruments regardless of view configuration before attribute filters
are applied.
For instruments configured with Delta temporality, where it is
possible for the map to shrink, note that the size of this map
includes records maintained due to InactiveCollectionPeriods. The
inactivity period should be taken into account when setting
InstrumentCardinalityLimit to avoid overflow.
AggregatorCardinalityLimit
All views maintain a configurable cardinality limit, calculated after attribute filters are applied.
When the aggregator's output grows to AggregatorCardinalityLimit,
new attribute sets will be replaced by the overflow attribute set,
which is { otel.metric.overflow=true }.
MeasurementProcessor
The MeasurementProcessor interface that makes it possible to extend
the set of attributes from synchronous instrument events, which allows
metric attributes to be generated from the OpenTelemetry request
context and/or W3C Tracecontext baggage.
This hook also supports removing attributes from metric events based on attribute value before they are aggregated, for example to dynamically configure allowed cardinality values.
AttributeSizeLimit
This limit is used to truncate attribute key and string values to a reasonable size. The default limit is 8kB. Zero is not a valid limit.
Documentation
¶
Index ¶
Constants ¶
const DefaultInterval = 30 * time.Second
const DefaultTimeout = DefaultInterval
Variables ¶
var ErrAlreadyShutdown = fmt.Errorf("provider was already shut down")
var ErrAlreadyUnregistered = fmt.Errorf("callback was already unregistered")
var ErrMultipleReaderRegistration = fmt.Errorf("reader has multiple registrations")
Functions ¶
This section is empty.
Types ¶
type ManualReader ¶
ManualReader is a a simple Reader that allows an application to read metrics on demand. It simply stores the Producer interface provided through registration. Flush and Shutdown are no-ops.
func NewManualReader ¶
func NewManualReader(name string) *ManualReader
NewManualReader returns an Reader that stores the Producer for manual use and returns a configurable `name` as its String(),
func (*ManualReader) ForceFlush ¶
func (mr *ManualReader) ForceFlush(context.Context) error
ForceFlush is a no-op, always returns nil.
func (*ManualReader) Register ¶
func (mr *ManualReader) Register(p Producer)
Register stores the Producer which enables the caller to read metrics on demand.
func (*ManualReader) Shutdown ¶
func (mr *ManualReader) Shutdown(context.Context) error
Shutdown is a no-op, always returns nil.
func (*ManualReader) String ¶
func (mr *ManualReader) String() string
String returns the name of this ManualReader.
type MeterProvider ¶
type MeterProvider struct {
embedded.MeterProvider
// contains filtered or unexported fields
}
MeterProvider handles the creation and coordination of Meters. All Meters created by a MeterProvider will be associated with the same Resource, have the same Views applied to them, and have their produced metric telemetry passed to the configured Readers.
func NewMeterProvider ¶
func NewMeterProvider(options ...Option) *MeterProvider
NewMeterProvider returns a new and configured MeterProvider.
By default, the returned MeterProvider is configured with the default Resource and no Readers. Readers cannot be added after a MeterProvider is created. This means the returned MeterProvider, one created with no Readers, will be perform no operations.
func (*MeterProvider) ForceFlush ¶
func (mp *MeterProvider) ForceFlush(ctx context.Context) error
ForceFlush flushes all pending telemetry.
This method honors the deadline or cancellation of ctx. An appropriate error will be returned in these situations. There is no guaranteed that all telemetry be flushed or all resources have been released in these situations.
This method is safe to call concurrently.
func (*MeterProvider) Meter ¶
func (mp *MeterProvider) Meter(name string, options ...metric.MeterOption) metric.Meter
Meter returns a Meter with the given name and configured with options.
The name should be the name of the instrumentation scope creating telemetry. This name may be the same as the instrumented code only if that code provides built-in instrumentation.
If name is empty, the default (go.opentelemetry.io/otel/sdk/meter) will be used.
Calls to the Meter method after Shutdown has been called will return Meters that perform no operations.
This method is safe to call concurrently.
func (*MeterProvider) Shutdown ¶
func (mp *MeterProvider) Shutdown(ctx context.Context) error
Shutdown shuts down the MeterProvider flushing all pending telemetry and releasing any held computational resources.
This call is idempotent. The first call will perform all flush and releasing operations. Subsequent calls will perform no action.
Measurements made by instruments from meters this MeterProvider created will not be exported after Shutdown is called.
This method honors the deadline or cancellation of ctx. An appropriate error will be returned in these situations. There is no guaranteed that all telemetry be flushed or all resources have been released in these situations.
This method is safe to call concurrently.
type Option ¶
type Option interface {
// contains filtered or unexported methods
}
Option applies a configuration option value to a MeterProvider.
func WithPerformance ¶ added in v1.13.1
func WithPerformance(perf sdkinstrument.Performance) Option
WithPerformance supports modifying performance settings.
func WithReader ¶
WithReader associates a new Reader and associated View options with a new MeterProvider
func WithResource ¶
WithResource associates a Resource with a new MeterProvider.
type PeriodicReader ¶
type PeriodicReader struct {
// contains filtered or unexported fields
}
PeriodicReader is an implementation of Reader that manages periodic exporter, flush, and shutdown. This implementation re-uses data from one collection to the next, to lower memory costs.
func NewPeriodicReader ¶
func NewPeriodicReader(exporter PushExporter, interval time.Duration, opts ...PeriodicReaderOption) *PeriodicReader
NewPeriodicReader constructs a PeriodicReader from a push-based exporter given an interval.
func (*PeriodicReader) ForceFlush ¶
func (pr *PeriodicReader) ForceFlush(ctx context.Context) error
ForceFlush immediately waits for an existing collection, otherwise immediately begins collection without regards to timing and calls ForceFlush with current data. There is no automatic timeout; to apply one, use context.WithTimeout.
func (*PeriodicReader) Register ¶
func (pr *PeriodicReader) Register(producer Producer)
Register starts the periodic export loop.
func (*PeriodicReader) Shutdown ¶
func (pr *PeriodicReader) Shutdown(ctx context.Context) error
Shutdown stops the export loop, canceling its Context, and waits for it to return. Then it issues a ShutdownMetrics with final data. There is no automatic timeout; to apply one, use context.WithTimeout.
func (*PeriodicReader) String ¶
func (pr *PeriodicReader) String() string
String returns the exporter name and the configured interval.
type PeriodicReaderOption ¶
type PeriodicReaderOption func(*PeriodicReader)
func WithTimeout ¶
func WithTimeout(d time.Duration) PeriodicReaderOption
type Producer ¶
type Producer interface {
// Produce returns metrics from a single collection.
//
// Produce may be called concurrently.
//
// The `in` parameter supports re-use of memory from
// one collection to the next. Callers that pass `in`
// will write metrics into the same slices and structs.
//
// When `in` is nil, a new Metrics object is returned.
Produce(in *data.Metrics) data.Metrics
}
Producer is the interface used to perform collection by the reader.
type PushExporter ¶
type PushExporter interface {
// String is the name used in errors related to this exporter/reader.
String() string
// ExportMetrics is called periodically with data collected
// from the Producer.
ExportMetrics(context.Context, data.Metrics) error
// ShutdownMetrics is called when the user calls Shutdown with
// final data collected from the Producer.
ShutdownMetrics(context.Context, data.Metrics) error
// ForceFlushMetrics is called when the user calls ForceFlush
// with data collected from the Producer.
ForceFlushMetrics(context.Context, data.Metrics) error
}
PushExporter is an interface for push-based exporters.
type Reader ¶
type Reader interface {
// String describes this reader.
String() string
// Register is called when the SDK is fully
// configured. The Producer passed allows the
// Reader to begin collecting metrics using its
// Produce() method.
Register(Producer)
// ForceFlush is called when MeterProvider.ForceFlush() is called.
ForceFlush(context.Context) error
// Shutdown is called when MeterProvider.Shutdown() is called.
Shutdown(context.Context) error
}
Reader is the interface used between the SDK and an exporter. Control flow is bi-directional through the Reader, since the SDK initiates ForceFlush and Shutdown while the Reader initiates collection. The Register() method here informs the Reader that it can begin reading, signaling the start of bi-directional control flow.
Typically, push-based exporters that are periodic will implement PeroidicExporter themselves and construct a PeriodicReader to satisfy this interface.
Pull-based exporters will typically implement Register themselves, since they read on demand.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
package aggregator defines the interface used between the SDK and various kinds of aggregation.
|
package aggregator defines the interface used between the SDK and various kinds of aggregation. |
|
example
module
|
|
|
exporters
|
|
|
internal
|
|
|
doevery
package doevery provides primitives for per-call-site rate-limiting.
|
package doevery provides primitives for per-call-site rate-limiting. |
|
viewstate
package viewstate implements a View compiler, which combines information about the instrument kind (especially synchronous vs.
|
package viewstate implements a View compiler, which combines information about the instrument kind (especially synchronous vs. |