README
¶
StreamingFast Metrics Library
For now, this library contains simple wrapping structure around Prometheus metrics. This improves usage and developer experience of defining metrics for StreamingFast services. It is part of StreamingFast.
This library should be kept as SMALL as possible, as it is a dependency we want to sprinkle around widely.
Usage
All metrics must be created from a metrics Set. The idea is that each library and micro-services defines a single set of metrics.
All metrics to be published by Prometheus must be registered at some point.
Example metrics.go in an example bstream project:
// Exported, so it can be registered by a `metrics.go` in main packages.
var MetricsSet = dmetrics.NewSet()
// HitCount is exported if used by other packages elsewhere, use like `bstream.HitCount.Inc()`
var HitCount = MetricsSet.NewGauge("bstream_hit_count", "hello %s")
// myCount is not exported, because used only in here. Use as: `myCount.Inc()`
var myCount = MetricsSet.NewGauge("bstream_my_count", "hello %s")
In a main package, in a metrics.go (similar to logging.go):
func init() {
dmetrics.Register(
bstream.MetricsSet,
dauth.MetricsSet,
blockmeta.MetricsSet,
)
}
Background
Initially, we had defined our metrics directly as the Prometheus type giving a definition of metrics in the form:
var mapSize = newGauge(
"map_size",
"size of live blocks map",
)
func IncMapSize() {
mapSize.Inc()
}
func DecMapSize() {
mapSize.Dec()
}
The usage of this was then like this:
metrics.IncMapSize()
...
metrics.DecMapSize()
This was repeated for all metrics then defined. This is problematic as when there is multiple metrics, the source file for definitions becomes bloated with lots of repeated stuff and duplicated stuff.
To overcome this, this library wraps different Prometheus metrics to clean down the definitions file. and offer a nicer API around them , and also usage. The previous example can now be turned into:
var MapSize = dmetrics.NewGauge("map_size", "size of live blocks map")
And the usage is now like:
metrics.MapSize.Inc()
...
metrics.MapSize.Dec()
An incredible improvement in the definitions of the metrics themselves.
Contributing
Issues and PR in this repo related strictly to the dmetrics library.
Report any protocol-specific issues in their respective repositories
Please first refer to the general StreamingFast contribution guide, if you wish to contribute to this code base.
License
Documentation
¶
Index ¶
- Variables
- func Register(sets ...*Set)
- func Serve(addr string)
- type AppReadiness
- type AvgCounter
- type AvgDurationCounter
- func (c *AvgDurationCounter) AddDuration(dur time.Duration)
- func (c *AvgDurationCounter) AddElapsedTime(start time.Time)
- func (c *AvgDurationCounter) Average() time.Duration
- func (c *AvgDurationCounter) AverageString() string
- func (c *AvgDurationCounter) String() string
- func (c *AvgDurationCounter) Total() time.Duration
- type AvgRateCounter
- type AvgRatePromCounter
- type AvgRatePromGauge
- type CountableFunc
- type Counter
- func (c *Counter) AddFloat64(value float64)
- func (c *Counter) AddInt(value int)
- func (c *Counter) AddInt64(value int64)
- func (c *Counter) AddUint64(value uint64)
- func (g *Counter) Collect(in chan<- prometheus.Metric)
- func (g *Counter) Describe(in chan<- *prometheus.Desc)
- func (c *Counter) Inc()
- func (c *Counter) Native() prometheus.Counter
- type CounterVec
- func (c *CounterVec) AddFloat64(value float64, labels ...string)
- func (c *CounterVec) AddInt(value int, labels ...string)
- func (c *CounterVec) AddInt64(value int64, labels ...string)
- func (c *CounterVec) AddUint64(value uint64, labels ...string)
- func (g *CounterVec) Collect(in chan<- prometheus.Metric)
- func (c *CounterVec) DeleteLabelValues(labels ...string)
- func (g *CounterVec) Describe(in chan<- *prometheus.Desc)
- func (c *CounterVec) Inc(labels ...string)
- func (g *CounterVec) Native() *prometheus.CounterVec
- type Gauge
- type GaugeVec
- func (g *GaugeVec) Collect(in chan<- prometheus.Metric)
- func (g *GaugeVec) Dec(labels ...string)
- func (g *GaugeVec) DeleteLabelValues(labels ...string)
- func (g *GaugeVec) Describe(in chan<- *prometheus.Desc)
- func (g *GaugeVec) Inc(labels ...string)
- func (g *GaugeVec) Native() *prometheus.GaugeVec
- func (g *GaugeVec) SetFloat64(value float64, labels ...string)
- func (g *GaugeVec) SetInt(value int, labels ...string)
- func (g *GaugeVec) SetInt64(value int64, labels ...string)
- func (g *GaugeVec) SetUint64(value uint64, labels ...string)
- type HeadBlockNum
- type HeadTimeDrift
- type Histogram
- func (h *Histogram) Collect(in chan<- prometheus.Metric)
- func (h *Histogram) Describe(in chan<- *prometheus.Desc)
- func (h *Histogram) Native() prometheus.Histogram
- func (h *Histogram) ObserveDuration(value time.Duration)
- func (h *Histogram) ObserveFloat64(value float64)
- func (h *Histogram) ObserveInt(value int64)
- func (h *Histogram) ObserveInt64(value int64)
- func (h *Histogram) ObserveSince(value time.Time)
- func (h *Histogram) ObserveUint64(value int64)
- type HistogramVec
- func (h *HistogramVec) Collect(in chan<- prometheus.Metric)
- func (h *HistogramVec) DeleteLabelValues(labels ...string)
- func (h *HistogramVec) Describe(in chan<- *prometheus.Desc)
- func (h *HistogramVec) Native() *prometheus.HistogramVec
- func (h *HistogramVec) ObserveDuration(value time.Duration, labels ...string)
- func (h *HistogramVec) ObserveFloat64(value float64, labels ...string)
- func (h *HistogramVec) ObserveInt(value int64, labels ...string)
- func (h *HistogramVec) ObserveInt64(value int64, labels ...string)
- func (h *HistogramVec) ObserveSince(value time.Time, labels ...string)
- func (h *HistogramVec) ObserveUint64(value int64, labels ...string)
- type Metric
- type Option
- type RateCounter
- type Set
- func (s *Set) NewAppReadiness(service string) *AppReadiness
- func (s *Set) NewCounter(name string, helpChunks ...string) *Counter
- func (s *Set) NewCounterVec(name string, labels []string, helpChunks ...string) *CounterVec
- func (s *Set) NewGauge(name string, helpChunks ...string) *Gauge
- func (s *Set) NewGaugeVec(name string, labels []string, helpChunks ...string) *GaugeVec
- func (s *Set) NewHeadBlockNumber(service string) *HeadBlockNum
- func (s *Set) NewHeadTimeDrift(service string) *HeadTimeDrift
- func (s *Set) NewHistogram(name string, helpChunks ...string) *Histogram
- func (s *Set) NewHistogramVec(name string, labels []string, helpChunks ...string) *HistogramVec
- func (s *Set) Register()
- type ValueFromMetric
- type ValuesFromMetric
Constants ¶
This section is empty.
Variables ¶
var InferUnit = time.Duration(0)
var NoOpPrometheusRegister = func(c ...prometheus.Collector) {}
var PrometheusRegister = prometheus.MustRegister
Functions ¶
Types ¶
type AppReadiness ¶
type AppReadiness struct {
// contains filtered or unexported fields
}
func (*AppReadiness) SetNotReady ¶
func (a *AppReadiness) SetNotReady()
func (*AppReadiness) SetReady ¶
func (a *AppReadiness) SetReady()
type AvgCounter ¶
type AvgCounter struct {
// contains filtered or unexported fields
}
func NewAvgCounter ¶
func NewAvgCounter(samplingWindow time.Duration, eventType string) *AvgCounter
NewAvgCounter allows you to get the average of an event over the period of time. For example, if you want to know the average cache hits in a given time
Over 1 second, you will increment the average by the number of cache hits ¶
``` counter := NewAvgCounter(1*time.Second, "cache hits") counter.IncBy(283) counter.IncBy(23) counter.IncBy(192) counter.IncBy(392)
counter.String() == avg 222.5 cache hits (over 1s) [12321 total] ```
func (*AvgCounter) Average ¶
func (c *AvgCounter) Average() float64
func (*AvgCounter) AverageString ¶
func (c *AvgCounter) AverageString() string
func (*AvgCounter) IncBy ¶
func (c *AvgCounter) IncBy(value int64)
IncBy adds multiple events (useful for debouncing event counts)
func (*AvgCounter) String ¶
func (c *AvgCounter) String() string
func (*AvgCounter) Total ¶
func (c *AvgCounter) Total() uint64
type AvgDurationCounter ¶
type AvgDurationCounter struct {
// contains filtered or unexported fields
}
func NewAvgDurationCounter ¶
func NewAvgDurationCounter(samplingWindow time.Duration, unit time.Duration, description string) *AvgDurationCounter
NewAvgDurationCounter allows you to get teh average elapsed time of a given process As an example, if you want to know the average block process time.
Example: if over 1 second you process 3 blocks where the processing time respectively takes 2s, 5s, 300ms. The average will yield a result of 2.43s per block.
``` counter := NewAvgDurationCounter(30*time.Second, time.Second, "per block") counter.AddDuration(2 * time.Second) counter.AddDuration(5 * time.Second) counter.AddDuration(300 * time.Millisecond)
counter.String() == 2.43s per block (avg over 30s) ```
The `unit` parameter can be 0, in which case the unit will be inferred based on the actual duration, e.g. if the average is 1.5s, the unit will be 1s while if the average is 10us, the unit will be 10us.
func (*AvgDurationCounter) AddDuration ¶
func (c *AvgDurationCounter) AddDuration(dur time.Duration)
func (*AvgDurationCounter) AddElapsedTime ¶
func (c *AvgDurationCounter) AddElapsedTime(start time.Time)
func (*AvgDurationCounter) Average ¶
func (c *AvgDurationCounter) Average() time.Duration
func (*AvgDurationCounter) AverageString ¶
func (c *AvgDurationCounter) AverageString() string
func (*AvgDurationCounter) String ¶
func (c *AvgDurationCounter) String() string
func (*AvgDurationCounter) Total ¶
func (c *AvgDurationCounter) Total() time.Duration
type AvgRateCounter ¶
type AvgRateCounter struct {
// contains filtered or unexported fields
}
func MustNewAvgRateCounter ¶
func NewAvgRateCounter ¶
func NewAvgRateCounter(samplingWindow time.Duration, period time.Duration, unit string) (*AvgRateCounter, error)
NewAvgRateCounter Tracks the average rate over a period of time. The rate is computed by accumulating the total count at the <samplingWindow> interval and averaging them our ove the number of <period> defined. Suppose there is a block count that increments as follows
0s to 1s -> 10 blocks 1s to 2s -> 3 blocks 2s to 3s -> 0 blocks 3s to 4s -> 7 blocks
If your samplingWindow = 1s and your period = 4s, the rate will be computed as
(10 + 3 + 0 + 7)/4 = 5 blocks/sec
If your samplingWindow = 1s and your period = 3s, the rate will be computed as
(10 + 3 + 0)/4 = 4.33 blocks/sec
then when the "window moves" you would get
(3 + 0 + 7)/4 = 3.333 blocks/sec
func (*AvgRateCounter) Add ¶
func (a *AvgRateCounter) Add(v uint64)
Add tracks a number of events, to be used to compute the rage
func (AvgRateCounter) RateString ¶
func (c AvgRateCounter) RateString() string
func (*AvgRateCounter) Stop ¶
func (c *AvgRateCounter) Stop()
type AvgRatePromCounter ¶
type AvgRatePromCounter struct {
// contains filtered or unexported fields
}
func MustNewAvgRateFromPromCounter ¶
func MustNewAvgRateFromPromCounter(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) *AvgRatePromCounter
MustNewAvgRateFromPromCounter acts like NewAvgRateFromPromCounter but panics if an error occurs. Refers to NewAvgRateFromPromCounter for more information.
func NewAvgRateFromPromCounter ¶
func NewAvgRateFromPromCounter(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) (*AvgRatePromCounter, error)
NewAvgRateFromPromCounter Extracts the average rate of a Prom Collector over a period of time. The rate is computed by accumulating the total count at the <samplingWindow> interval and averaging them our ove the number of <period> defined. Suppose there is a block count that increments as follows
0s to 1s -> 10 blocks 1s to 2s -> 3 blocks 2s to 3s -> 0 blocks 3s to 4s -> 7 blocks
If your samplingWindow = 1s and your period = 4s, the rate will be computed as
(10 + 3 + 0 + 7)/4 = 5 blocks/sec
If your samplingWindow = 1s and your period = 3s, the rate will be computed as
(10 + 3 + 0)/4 = 4.33 blocks/sec
then when the "window moves" you would get
(3 + 0 + 7)/4 = 3.333 blocks/sec
type AvgRatePromGauge ¶
type AvgRatePromGauge struct {
// contains filtered or unexported fields
}
func MustNewAvgRateFromPromGauge ¶
func MustNewAvgRateFromPromGauge(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) *AvgRatePromGauge
MustNewAvgRateFromPromGauge acts like NewAvgRateFromPromGauge but panics if an error occurs. Refers to NewAvgRateFromPromGauge for more information.
func NewAvgRateFromPromGauge ¶
func NewAvgRateFromPromGauge(promCollector prometheus.Collector, samplingWindow time.Duration, period time.Duration, unit string) (*AvgRatePromGauge, error)
NewAvgRateFromPromGauge extracts the average rate of a Promtheus Gauge over a period of time. The rate is computed by accumulating the total count at the <samplingWindow> interval and averaging them our ove the number of <period> defined. Suppose there is a block count that increments as follows
0s to 1s -> 10 blocks 1s to 2s -> 3 blocks 2s to 3s -> 0 blocks 3s to 4s -> 7 blocks
If your samplingWindow = 1s and your period = 4s, the rate will be computed as
(10 + 3 + 0 + 7)/4 = 5 blocks/sec
If your samplingWindow = 1s and your period = 3s, the rate will be computed as
(10 + 3 + 0)/4 = 4.33 blocks/sec
then when the "window moves" you would get
(3 + 0 + 7)/4 = 3.333 blocks/sec
**Important** Your Gauge should be ever increasing. If it's not, the results will be incorrect.
type CountableFunc ¶
type CountableFunc = func() uint64
type Counter ¶
type Counter struct {
// contains filtered or unexported fields
}
func (*Counter) AddFloat64 ¶
func (*Counter) Collect ¶
func (g *Counter) Collect(in chan<- prometheus.Metric)
func (*Counter) Describe ¶
func (g *Counter) Describe(in chan<- *prometheus.Desc)
func (*Counter) Native ¶
func (c *Counter) Native() prometheus.Counter
type CounterVec ¶
type CounterVec struct {
// contains filtered or unexported fields
}
func (*CounterVec) AddFloat64 ¶
func (c *CounterVec) AddFloat64(value float64, labels ...string)
func (*CounterVec) AddInt ¶
func (c *CounterVec) AddInt(value int, labels ...string)
func (*CounterVec) AddInt64 ¶
func (c *CounterVec) AddInt64(value int64, labels ...string)
func (*CounterVec) AddUint64 ¶
func (c *CounterVec) AddUint64(value uint64, labels ...string)
func (*CounterVec) Collect ¶
func (g *CounterVec) Collect(in chan<- prometheus.Metric)
func (*CounterVec) DeleteLabelValues ¶
func (c *CounterVec) DeleteLabelValues(labels ...string)
func (*CounterVec) Describe ¶
func (g *CounterVec) Describe(in chan<- *prometheus.Desc)
func (*CounterVec) Inc ¶
func (c *CounterVec) Inc(labels ...string)
func (*CounterVec) Native ¶
func (g *CounterVec) Native() *prometheus.CounterVec
type Gauge ¶
type Gauge struct {
// contains filtered or unexported fields
}
func (*Gauge) Collect ¶
func (g *Gauge) Collect(in chan<- prometheus.Metric)
func (*Gauge) Describe ¶
func (g *Gauge) Describe(in chan<- *prometheus.Desc)
func (*Gauge) Native ¶
func (g *Gauge) Native() prometheus.Gauge
func (*Gauge) SetFloat64 ¶
type GaugeVec ¶
type GaugeVec struct {
// contains filtered or unexported fields
}
func (*GaugeVec) Collect ¶
func (g *GaugeVec) Collect(in chan<- prometheus.Metric)
func (*GaugeVec) DeleteLabelValues ¶
func (*GaugeVec) Describe ¶
func (g *GaugeVec) Describe(in chan<- *prometheus.Desc)
func (*GaugeVec) Native ¶
func (g *GaugeVec) Native() *prometheus.GaugeVec
func (*GaugeVec) SetFloat64 ¶
type HeadBlockNum ¶
type HeadBlockNum struct {
// contains filtered or unexported fields
}
func (*HeadBlockNum) SetUint64 ¶
func (h *HeadBlockNum) SetUint64(blockNum uint64)
type HeadTimeDrift ¶
type HeadTimeDrift struct {
// contains filtered or unexported fields
}
func (*HeadTimeDrift) SetBlockTime ¶
func (h *HeadTimeDrift) SetBlockTime(blockTime time.Time)
type Histogram ¶
type Histogram struct {
// contains filtered or unexported fields
}
func (*Histogram) Collect ¶
func (h *Histogram) Collect(in chan<- prometheus.Metric)
func (*Histogram) Describe ¶
func (h *Histogram) Describe(in chan<- *prometheus.Desc)
func (*Histogram) Native ¶
func (h *Histogram) Native() prometheus.Histogram
func (*Histogram) ObserveDuration ¶
func (*Histogram) ObserveFloat64 ¶
func (*Histogram) ObserveInt ¶
func (*Histogram) ObserveInt64 ¶
func (*Histogram) ObserveSince ¶
func (*Histogram) ObserveUint64 ¶
type HistogramVec ¶
type HistogramVec struct {
// contains filtered or unexported fields
}
func (*HistogramVec) Collect ¶
func (h *HistogramVec) Collect(in chan<- prometheus.Metric)
func (*HistogramVec) DeleteLabelValues ¶
func (h *HistogramVec) DeleteLabelValues(labels ...string)
func (*HistogramVec) Describe ¶
func (h *HistogramVec) Describe(in chan<- *prometheus.Desc)
func (*HistogramVec) Native ¶
func (h *HistogramVec) Native() *prometheus.HistogramVec
func (*HistogramVec) ObserveDuration ¶
func (h *HistogramVec) ObserveDuration(value time.Duration, labels ...string)
func (*HistogramVec) ObserveFloat64 ¶
func (h *HistogramVec) ObserveFloat64(value float64, labels ...string)
func (*HistogramVec) ObserveInt ¶
func (h *HistogramVec) ObserveInt(value int64, labels ...string)
func (*HistogramVec) ObserveInt64 ¶
func (h *HistogramVec) ObserveInt64(value int64, labels ...string)
func (*HistogramVec) ObserveSince ¶
func (h *HistogramVec) ObserveSince(value time.Time, labels ...string)
func (*HistogramVec) ObserveUint64 ¶
func (h *HistogramVec) ObserveUint64(value int64, labels ...string)
type Metric ¶
type Metric interface {
prometheus.Collector
}
type Option ¶
type Option func(s *Set)
func PrefixNameWith ¶
PrefixNameWith will prefix all metric of this given set using the following prefix.
type RateCounter ¶
type RateCounter struct {
// contains filtered or unexported fields
}
func NewPerMinuteLocalRateCounter ¶
func NewPerMinuteLocalRateCounter(unit string) *RateCounter
func NewPerSecondLocalRateCounter ¶
func NewPerSecondLocalRateCounter(unit string) *RateCounter
func NewRateCounter ¶
func NewRateCounter(interval time.Duration, unit string) *RateCounter
NewRateCounter allows you to know how many times an event happen over a fixed period of time
For example, if over 1 second you process 20 blocks, then querying the counter within this 1s samplingWindow will yield a result of 20 blocks/s. The rate change as the time moves.
``` counter := NewRateCounter(1*time.Second, "s", "blocks") counter.Incr() counter.Incr() counter.Incr()
counter.String() == 3 blocks/s (over 1s)
IMPORTANT: The rate is calculated by the number of events that occurred in the last sampling window thus if from time 0s to 1s you have 3 events the rate is 3 events/sec; subsequently if you wait another 2 seconds without any event occurring, then query the rate you will get 0 event/sec. If you want an Avg rate over multiple sampling window Use AvgRateCounter ```
func (*RateCounter) IncBy ¶
func (c *RateCounter) IncBy(value int64)
IncrBy adds multiple events inot the RateCounter
func (*RateCounter) Rate ¶
func (c *RateCounter) Rate() int64
func (*RateCounter) RateString ¶
func (c *RateCounter) RateString() string
func (*RateCounter) String ¶
func (c *RateCounter) String() string
func (*RateCounter) Total ¶
func (c *RateCounter) Total() uint64
type Set ¶
type Set struct {
// contains filtered or unexported fields
}
func NewSet ¶
NewSet creates a set of metrics that can then be used to create a varieties of specific metrics (Gauge, Counter, Histogram).
func (*Set) NewAppReadiness ¶
func (s *Set) NewAppReadiness(service string) *AppReadiness
func (*Set) NewCounterVec ¶
func (s *Set) NewCounterVec(name string, labels []string, helpChunks ...string) *CounterVec
func (*Set) NewGaugeVec ¶
func (*Set) NewHeadBlockNumber ¶
func (s *Set) NewHeadBlockNumber(service string) *HeadBlockNum
func (*Set) NewHeadTimeDrift ¶
func (s *Set) NewHeadTimeDrift(service string) *HeadTimeDrift
func (*Set) NewHistogramVec ¶
func (s *Set) NewHistogramVec(name string, labels []string, helpChunks ...string) *HistogramVec
type ValueFromMetric ¶
type ValueFromMetric struct {
// contains filtered or unexported fields
}
ValueFromMetric can be used to extract the value of a Prometheus Metric object.
*Important* This for now does not correctly handles `Vec` like metrics since the actual logic is to return the first ever value encountered while in a `Vec` metric, there is usually N values, one per label.
func NewValueFromMetric ¶
func NewValueFromMetric(metric prometheus.Collector, unit string) *ValueFromMetric
func (*ValueFromMetric) ValueFloat ¶
func (c *ValueFromMetric) ValueFloat() float64
func (*ValueFromMetric) ValueUint ¶
func (c *ValueFromMetric) ValueUint() uint64
type ValuesFromMetric ¶
type ValuesFromMetric struct {
// contains filtered or unexported fields
}
ValuesFromMetric can be used to extract the values of a Prometheus Metric 'Vec' object, i.e. with a 'label' dimension.
func NewValuesFromMetric ¶
func NewValuesFromMetric(metric prometheus.Collector) *ValuesFromMetric
Source Files
Directories