config

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 23, 2023 License: Unlicense Imports: 9 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ConfigFile = "gcp2prom.yaml"
View Source
var Scale = map[string]ScalingFunc{
	"*1024*1024*1024": multiply(1024.0 * 1024.0 * 1024.0),
	"*1024*1024":      multiply(1024.0 * 1024.0),
	"*60*60*24":       multiply(60.0 * 60.0 * 24.0),
	"/100":            divide(100.0),
	"/1000":           divide(1000.0),
	"/1000/1000":      divide(1000.0 * 1000.0),
	"/1000/1000/1000": divide(1000.0 * 1000.0 * 1000.0),
}

Functions

This section is empty.

Types

type Configuration

type Configuration struct {
	// System is the first part of each Prometheus metric name.
	System string

	// Subsystem maps GCP metric path prefixes to the 2nd part of Prom metric
	// names.  Each key should be a prefix of some GCP metric names (paths) up
	// to (and including) a '/' character (for example,
	// "bigquery.googleapis.com/query/").  Metrics that do not match any of
	// these prefixes are silently ignored.
	//
	// The prefix is stripped from the GCP metric path to get the (first
	// iteration) of the last part of each metric name in Prometheus.  Any
	// '/' characters left after the prefix are left as-is until all Suffix
	// replacements are finished (see Suffix item below), at which point any
	// remaining '/' characters become '_' characters.
	//
	// Only the longest matching prefix is applied (per metric).
	//
	Subsystem map[string]string

	// Unit maps a unit name to the name of a predefined scaling factor to
	// convert to base units that are preferred in Prometheus.  The names
	// of the conversion functions look like multiplication or division
	// operations, often with repeated parts.  For example,
	// `"ns": "/1000/1000/1000"` to convert nanoseconds to seconds and
	// `"d": "*60*60*24"` to convert days to seconds.
	//
	// Each key is a string containing a comma-separated list of unit types.
	// If you use the same unit type in multiple entries, then which of those
	// entries that will be applied to a metric will be "random".
	//
	Unit map[string]string

	// Histogram is a list of rules for resampling histogram metrics to reduce
	// the number of buckets or to simply ignore histogram metrics with too
	// many buckets.
	//
	// The rules are evaluated in the order listed and only the first
	// matching rule (for each metric) is applied.
	//
	Histogram []HistogramConf

	// OmitLabel specifies rules for identifying labels to be omitted from
	// the metrics exported to Prometheus.  This is usually used to remove
	// labels that would cause high-cardinality metrics.
	//
	// If a metric matches more than one rule, then any labels mentioned in
	// any of the matching rules will be omitted.
	//
	OmitLabel []OmitLabelConf

	// Suffix is a list of rules for adjusting the last part of Prometheus
	// metric names by replacing a suffix.  Rules are applied in the order
	// listed and each rule that applies will change the Prometheus metric
	// name that will be used for matching subsequent rules.
	//
	Suffix []*SuffixConf
}

The Configuration type specifies what data can be put in the gcp2prom.yaml configuration file to control which GCP metrics can be exported to Prometheus and to configure how each gets converted.

Note that YAML lowercases the letters of keys so, for example, to set MaxBuckets, your YAML would contain something like `maxbuckets: 32`.

func LoadConfig

func LoadConfig(path string) (Configuration, error)

func MustLoadConfig

func MustLoadConfig(path string) Configuration

func (Configuration) GcpPrefixes

func (c Configuration) GcpPrefixes() []string

Returns the list of prefixes to GCP metrics that could be handled.

func (Configuration) MatchMetric

func (c Configuration) MatchMetric(md *sd.MetricDescriptor) *MetricMatcher

Constructs a MetricMatcher for the given GCP MetricDescriptor. If a second argument is passed, it should be a string holding the path to a YAML file containing the configuration to be used for exporting the metric to Prometheus.

Returns `nil` if the metric is not one that can be exported to Prometheus based on the configuration chosen (because no Subsystem has been configured for it).

type HistogramConf

type HistogramConf struct {
	// The For element specifies which metrics match this rule.  If
	// missing (or empty), then the rule applies to all histogram metrics
	// (except those already matched by a prior rule) -- which only makes
	// sense for the last rule listed.
	//
	For Selector

	// MinBuckets specifies the minimum number of buckets needed for
	// resampling to happen.  If there are fewer than MinBuckets buckets,
	// then the buckets are not resampled and are preserved as-is.
	//
	MinBuckets int

	// MinBound specifies the minimum allowed bucket boundary.  If the
	// first bucket boundary is below this value, then the lowest
	// bucket boundary that is at least MinBound becomes the first
	// bucket boundary (and the counts for all prior buckets get
	// combined into the new first bucket).  Since metric values can
	// be negative, a MinBound of 0.0 is not ignored (unless MaxBound
	// is also 0.0).
	//
	MinBound,

	MinRatio,

	MaxBound float64

	// If the number of buckets (after resampling, if any was configured
	// in this rule) is larger than MaxBuckets, then the metric is just
	// ignored and will not be exported to Prometheus.
	//
	MaxBuckets int
}

A HistogramConf is a rule for resampling histogram metrics to reduce the number of buckets or to simply ignore histogram metrics with too many buckets. If MinBound, MinRatio, and MaxBound are all 0.0 (or missing), then they are ignored and only MaxBuckets applies.

If MinBound, MinRatio, and MaxBound are all 0.0 (or missing), then they are all ignored and only MaxBuckets applies.

Note that if the Unit config element specifies a scaling factor for this metric, then that scaling is applied to the bucket boundaries (and not the MinBound nor MaxBound) before any resampling is done.

type LongestFirst

type LongestFirst []string

func (LongestFirst) Len

func (p LongestFirst) Len() int

func (LongestFirst) Less

func (p LongestFirst) Less(i, j int) bool

func (LongestFirst) Swap

func (p LongestFirst) Swap(i, j int)

type MetricMatcher

type MetricMatcher struct {
	MD     *sd.MetricDescriptor
	SubSys string // Middle part of Prom metric name.
	Name   string // Last part of Prom metric name, so far.
	// Metric kind; one of 'C', 'D', or 'G' for cumulative, delta, or gauge.
	Kind mon.MetricKind
	// Metric type; one of 'F', 'I', 'S', 'H', or 'B' for float, int, string,
	// histogram (distribution), or bool.
	Type mon.ValueType
	// MD.Unit but ” becomes '-' and values (or parts of values) like
	// '{Bytes}' are replaced by just '{}'.
	Unit string
	// contains filtered or unexported fields
}

MetricMatcher contains the information about one type of GCP metric. This is compared to each Selector found in the config to determine if that part of the config should apply to this metric.

This is mostly just a GCP MetricDescriptor plus the last part of the metric name to be used in Prometheus (as computed so far). This 'Name' field may be updated as each 'Suffix' config is applied.

func (*MetricMatcher) HistogramLimits

func (mm *MetricMatcher) HistogramLimits() (
	minBuckets int, minBound, minRatio, maxBound float64, maxBuckets int,
)

Returns minBuckets, minBound, minRatio, maxBound, and maxBuckets to use for histogram resampling for the given metric.

func (*MetricMatcher) OmitLabels

func (mm *MetricMatcher) OmitLabels() []string

Returns the label names to be dropped when exporting the passed-in GCP metric to Prometheus.

func (*MetricMatcher) PromName

func (mm *MetricMatcher) PromName() string

Returns the full metric name to use in Prometheus.

func (*MetricMatcher) Scaler

func (mm *MetricMatcher) Scaler() (ScalingFunc, string)

Returns `nil` or a function that scales float64 values from the units used in GCP to the base units that are preferred in Prometheus.

type OmitLabelConf

type OmitLabelConf struct {
	For    Selector // Selects which metrics to check.
	Labels []string // The list of metric labels to ignore.
}

OmitLabelConf specifies a rule for identifying labels to be omitted from the metrics exported to Prometheus. This is usually used to remove labels that would cause high-cardinality metrics.

type ScalingFunc

type ScalingFunc func(float64) float64

type Selector

type Selector struct {
	Prefix []string // Prefix(es) to match against full GCP metric paths.
	Suffix []string // Suffix(es) to match against Prom metric name.
	Only   string   // Required attributes (letters from "CDGHFIBS").
	Not    string   // Disallowed attributes (letters from "CDGHFIBS").
	Unit   string   // Required unit designation(s) (comma-separated).
}

Selector is the type used in each "For" entry, selecting which metrics to apply the associated settings to. A missing or empty "For" entry matches every metric.

For each top-level element in a Selector, specifying more than one value restricts to metrics matching _any_ of the values in that list ("or"). For Not, that means that a metric is excluded if it matches any of the attributes listed ("nor"). But a metric only matches a Selector if it matches every non-empty, top-level element ("and").

The letters used in Only and Not stand for: Cumulative, Delta, Gauge, Histogram, Float, Int, Bool, and String.

For Unit, ” becomes '-' and values (or parts of values) like '{Bytes}' become '{}'.

type SuffixConf

type SuffixConf struct {
	For     Selector
	Replace map[string]string
	// contains filtered or unexported fields
}

SuffixConf is a rule for adjusting the last part of Prometheus metric names by replacing a suffix. The For element determines which metrics this rule applies to.

If a metric matches a rule (at the time it is evaluated), then that rule is applied (in order) to that metric.

If any key in the Replace map is a suffix match for the Prometheus metric name (as computed to this point, possibly based on suffix matches from prior rules), then only the longest such key is selected. Then that suffix is replaced by the corresponding value in Replace.

A '/' character is prepended to the last part of the Prometheus name before comparing it to each Replace key so you can use a key like "/port_usage" to match (and replace) the whole name, not just a suffix.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL