index

package
v0.0.0-...-7028b89 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2017 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package index provides a number of indexes backed by persistent key-value stores. The only supported implementation of a key-value store is currently goleveldb, but other implementations can easily be added.

Index

Constants

View Source
const (
	FingerprintToMetricDir     = "archived_fingerprint_to_metric"
	FingerprintTimeRangeDir    = "archived_fingerprint_to_timerange"
	LabelNameToLabelValuesDir  = "labelname_to_labelvalues"
	LabelPairToFingerprintsDir = "labelpair_to_fingerprints"
)

Directory names for LevelDB indices.

Variables

View Source
var (
	FingerprintMetricCacheSize     = 10 * 1024 * 1024
	FingerprintTimeRangeCacheSize  = 5 * 1024 * 1024
	LabelNameLabelValuesCacheSize  = 10 * 1024 * 1024
	LabelPairFingerprintsCacheSize = 20 * 1024 * 1024
)

LevelDB cache sizes, changeable via flags.

Functions

func DeleteFingerprintTimeRangeIndex

func DeleteFingerprintTimeRangeIndex(basePath string) error

DeleteFingerprintTimeRangeIndex deletes the LevelDB-backed FingerprintTimeRangeIndex. Use only for a not yet opened index.

func DeleteLabelNameLabelValuesIndex

func DeleteLabelNameLabelValuesIndex(basePath string) error

DeleteLabelNameLabelValuesIndex deletes the LevelDB-backed LabelNameLabelValuesIndex. Use only for a not yet opened index.

func DeleteLabelPairFingerprintIndex

func DeleteLabelPairFingerprintIndex(basePath string) error

DeleteLabelPairFingerprintIndex deletes the LevelDB-backed LabelPairFingerprintIndex. Use only for a not yet opened index.

Types

type Batch

type Batch interface {
	Put(key, value encoding.BinaryMarshaler) error
	Delete(key encoding.BinaryMarshaler) error
	Reset()
}

Batch allows KeyValueStore mutations to be pooled and committed together. An implementation does not have to be goroutine-safe. Never modify a Batch concurrently or commit the same batch multiple times concurrently. Marshaling of keys and values is guaranteed to be complete when the Put or Delete methods have returned.

type FingerprintMetricIndex

type FingerprintMetricIndex struct {
	KeyValueStore
}

FingerprintMetricIndex models a database mapping fingerprints to metrics.

func NewFingerprintMetricIndex

func NewFingerprintMetricIndex(basePath string) (*FingerprintMetricIndex, error)

NewFingerprintMetricIndex returns a LevelDB-backed FingerprintMetricIndex ready to use.

func (*FingerprintMetricIndex) IndexBatch

func (i *FingerprintMetricIndex) IndexBatch(mapping FingerprintMetricMapping) error

IndexBatch indexes a batch of mappings from fingerprints to metrics.

This method is goroutine-safe, but note that no specific order of execution can be guaranteed (especially critical if IndexBatch and UnindexBatch are called concurrently for the same fingerprint).

func (*FingerprintMetricIndex) Lookup

func (i *FingerprintMetricIndex) Lookup(fp model.Fingerprint) (metric model.Metric, ok bool, err error)

Lookup looks up a metric by fingerprint. Looking up a non-existing fingerprint is not an error. In that case, (nil, false, nil) is returned.

This method is goroutine-safe.

func (*FingerprintMetricIndex) UnindexBatch

func (i *FingerprintMetricIndex) UnindexBatch(mapping FingerprintMetricMapping) error

UnindexBatch unindexes a batch of mappings from fingerprints to metrics.

This method is goroutine-safe, but note that no specific order of execution can be guaranteed (especially critical if IndexBatch and UnindexBatch are called concurrently for the same fingerprint).

type FingerprintMetricMapping

type FingerprintMetricMapping map[model.Fingerprint]model.Metric

FingerprintMetricMapping is an in-memory map of fingerprints to metrics.

type FingerprintTimeRangeIndex

type FingerprintTimeRangeIndex struct {
	KeyValueStore
}

FingerprintTimeRangeIndex models a database tracking the time ranges of metrics by their fingerprints.

func NewFingerprintTimeRangeIndex

func NewFingerprintTimeRangeIndex(basePath string) (*FingerprintTimeRangeIndex, error)

NewFingerprintTimeRangeIndex returns a LevelDB-backed FingerprintTimeRangeIndex ready to use.

func (*FingerprintTimeRangeIndex) Lookup

func (i *FingerprintTimeRangeIndex) Lookup(fp model.Fingerprint) (firstTime, lastTime model.Time, ok bool, err error)

Lookup returns the time range for the given fingerprint. Looking up a non-existing fingerprint is not an error. In that case, (0, 0, false, nil) is returned.

This method is goroutine-safe.

type KeyValueAccessor

type KeyValueAccessor interface {
	Key(encoding.BinaryUnmarshaler) error
	Value(encoding.BinaryUnmarshaler) error
}

KeyValueAccessor allows access to the key and value of an entry in a KeyValueStore.

type KeyValueStore

type KeyValueStore interface {
	Put(key, value encoding.BinaryMarshaler) error
	// Get unmarshals the result into value. It returns false if no entry
	// could be found for key. If value is nil, Get behaves like Has.
	Get(key encoding.BinaryMarshaler, value encoding.BinaryUnmarshaler) (bool, error)
	Has(key encoding.BinaryMarshaler) (bool, error)
	// Delete returns (false, nil) if key does not exist.
	Delete(key encoding.BinaryMarshaler) (bool, error)

	NewBatch() Batch
	Commit(b Batch) error

	// ForEach iterates through the complete KeyValueStore and calls the
	// supplied function for each mapping.
	ForEach(func(kv KeyValueAccessor) error) error

	Close() error
}

KeyValueStore persists key/value pairs. Implementations must be fundamentally goroutine-safe. However, it is the caller's responsibility that keys and values can be safely marshaled and unmarshaled (via the MarshalBinary and UnmarshalBinary methods of the keys and values). For example, if you call the Put method of a KeyValueStore implementation, but the key or the value are modified concurrently while being marshaled into its binary representation, you obviously have a problem. Methods of KeyValueStore return only after (un)marshaling is complete.

func NewLevelDB

func NewLevelDB(o LevelDBOptions) (KeyValueStore, error)

NewLevelDB returns a newly allocated LevelDB-backed KeyValueStore ready to use.

type LabelNameLabelValuesIndex

type LabelNameLabelValuesIndex struct {
	KeyValueStore
}

LabelNameLabelValuesIndex is a KeyValueStore that maps existing label names to all label values stored for that label name.

func NewLabelNameLabelValuesIndex

func NewLabelNameLabelValuesIndex(basePath string) (*LabelNameLabelValuesIndex, error)

NewLabelNameLabelValuesIndex returns a LevelDB-backed LabelNameLabelValuesIndex ready to use.

func (*LabelNameLabelValuesIndex) IndexBatch

IndexBatch adds a batch of label name to label values mappings to the index. A mapping of a label name to an empty slice of label values results in a deletion of that mapping from the index.

While this method is fundamentally goroutine-safe, note that the order of execution for multiple batches executed concurrently is undefined.

func (*LabelNameLabelValuesIndex) Lookup

func (i *LabelNameLabelValuesIndex) Lookup(l model.LabelName) (values model.LabelValues, ok bool, err error)

Lookup looks up all label values for a given label name and returns them as model.LabelValues (which is a slice). Looking up a non-existing label name is not an error. In that case, (nil, false, nil) is returned.

This method is goroutine-safe.

func (*LabelNameLabelValuesIndex) LookupSet

func (i *LabelNameLabelValuesIndex) LookupSet(l model.LabelName) (values map[model.LabelValue]struct{}, ok bool, err error)

LookupSet looks up all label values for a given label name and returns them as a set. Looking up a non-existing label name is not an error. In that case, (nil, false, nil) is returned.

This method is goroutine-safe.

type LabelNameLabelValuesMapping

type LabelNameLabelValuesMapping map[model.LabelName]codable.LabelValueSet

LabelNameLabelValuesMapping is an in-memory map of label names to label values.

type LabelPairFingerprintIndex

type LabelPairFingerprintIndex struct {
	KeyValueStore
}

LabelPairFingerprintIndex is a KeyValueStore that maps existing label pairs to the fingerprints of all metrics containing those label pairs.

func NewLabelPairFingerprintIndex

func NewLabelPairFingerprintIndex(basePath string) (*LabelPairFingerprintIndex, error)

NewLabelPairFingerprintIndex returns a LevelDB-backed LabelPairFingerprintIndex ready to use.

func (*LabelPairFingerprintIndex) IndexBatch

IndexBatch indexes a batch of mappings from label pairs to fingerprints. A mapping to an empty slice of fingerprints results in deletion of that mapping from the index.

While this method is fundamentally goroutine-safe, note that the order of execution for multiple batches executed concurrently is undefined.

func (*LabelPairFingerprintIndex) Lookup

Lookup looks up all fingerprints for a given label pair. Looking up a non-existing label pair is not an error. In that case, (nil, false, nil) is returned.

This method is goroutine-safe.

func (*LabelPairFingerprintIndex) LookupSet

func (i *LabelPairFingerprintIndex) LookupSet(p model.LabelPair) (fps map[model.Fingerprint]struct{}, ok bool, err error)

LookupSet looks up all fingerprints for a given label pair. Looking up a non-existing label pair is not an error. In that case, (nil, false, nil) is returned.

This method is goroutine-safe.

type LabelPairFingerprintsMapping

type LabelPairFingerprintsMapping map[model.LabelPair]codable.FingerprintSet

LabelPairFingerprintsMapping is an in-memory map of label pairs to fingerprints.

type LevelDB

type LevelDB struct {
	// contains filtered or unexported fields
}

LevelDB is a LevelDB-backed sorted KeyValueStore.

func (*LevelDB) Close

func (l *LevelDB) Close() error

Close implements KeyValueStore.

func (*LevelDB) Commit

func (l *LevelDB) Commit(b Batch) error

Commit implements KeyValueStore.

func (*LevelDB) Delete

func (l *LevelDB) Delete(key encoding.BinaryMarshaler) (bool, error)

Delete implements KeyValueStore.

func (*LevelDB) ForEach

func (l *LevelDB) ForEach(cb func(kv KeyValueAccessor) error) error

ForEach implements KeyValueStore.

func (*LevelDB) Get

Get implements KeyValueStore.

func (*LevelDB) Has

func (l *LevelDB) Has(key encoding.BinaryMarshaler) (has bool, err error)

Has implements KeyValueStore.

func (*LevelDB) NewBatch

func (l *LevelDB) NewBatch() Batch

NewBatch implements KeyValueStore.

func (*LevelDB) Put

func (l *LevelDB) Put(key, value encoding.BinaryMarshaler) error

Put implements KeyValueStore.

type LevelDBBatch

type LevelDBBatch struct {
	// contains filtered or unexported fields
}

LevelDBBatch is a Batch implementation for LevelDB.

func (*LevelDBBatch) Delete

func (b *LevelDBBatch) Delete(key encoding.BinaryMarshaler) error

Delete implements Batch.

func (*LevelDBBatch) Put

func (b *LevelDBBatch) Put(key, value encoding.BinaryMarshaler) error

Put implements Batch.

func (*LevelDBBatch) Reset

func (b *LevelDBBatch) Reset()

Reset implements Batch.

type LevelDBOptions

type LevelDBOptions struct {
	Path           string // Base path to store files.
	CacheSizeBytes int
}

LevelDBOptions provides options for a LevelDB.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL