inmem

package
v1.11.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 23, 2024 License: MIT Imports: 17 Imported by: 10

Documentation

Overview

Package inmem implements a shared, in-memory index for each database.

The in-memory index is the original index implementation and provides fast access to index data. However, it also forces high memory usage for large datasets and can cause OOM errors.

Index is the shared index structure that provides most of the functionality. However, ShardIndex is a light per-shard wrapper that adapts this original shared index format to the new per-shard format.

Index

Constants

View Source 
const IndexName = tsdb.InmemIndexName

IndexName is the name of this index.

Variables

This section is empty.

Functions

func NewShardIndex 

func NewShardIndex(id uint64, seriesIDSet *tsdb.SeriesIDSet, opt tsdb.EngineOptions) tsdb.Index

NewShardIndex returns a new index for a shard.

Types

type FilterExprs 

type FilterExprs map[uint64]influxql.Expr

FilterExprs represents a map of series IDs to filter expressions.

func (FilterExprs) DeleteBoolLiteralTrues 

func (fe FilterExprs) DeleteBoolLiteralTrues()

DeleteBoolLiteralTrues deletes all elements whose filter expression is a boolean literal true.

func (FilterExprs) Len 

func (fe FilterExprs) Len() int

Len returns the number of elements.

type Index 

type Index struct {
	// contains filtered or unexported fields
}

Index is the in memory index of a collection of measurements, time series, and their tags. Exported functions are goroutine safe while un-exported functions assume the caller will use the appropriate locks.

func NewIndex 

func NewIndex(database string, sfile *tsdb.SeriesFile) *Index

NewIndex returns a new initialized Index.

func (*Index) Bytes added in v1.6.0 

func (i *Index) Bytes() int

Bytes estimates the memory footprint of this Index, in bytes.

func (*Index) Close 

func (i *Index) Close() error

func (*Index) CreateMeasurementIndexIfNotExists 

func (i *Index) CreateMeasurementIndexIfNotExists(name []byte) *measurement

CreateMeasurementIndexIfNotExists creates or retrieves an in memory index object for the measurement

func (*Index) CreateSeriesListIfNotExists added in v1.5.0 

func (i *Index) CreateSeriesListIfNotExists(seriesIDSet *tsdb.SeriesIDSet, measurements map[string]int,
	keys, names [][]byte, tagsSlice []models.Tags, opt *tsdb.EngineOptions, ignoreLimits bool, tracker tsdb.StatsTracker) error

CreateSeriesListIfNotExists adds the series for the given measurement to the index and sets its ID or returns the existing series object

func (*Index) Database added in v1.5.0 

func (i *Index) Database() string

Database returns the name of the database the index was initialized with.

func (*Index) DiskSizeBytes added in v1.5.0 

func (i *Index) DiskSizeBytes() int64

DiskSizeBytes always returns zero bytes, since this is an in-memory index.

func (*Index) DropMeasurement 

func (i *Index) DropMeasurement(name []byte) error

DropMeasurement removes the measurement and all of its underlying series from the database index

func (*Index) DropMeasurementIfSeriesNotExist added in v1.5.0 

func (i *Index) DropMeasurementIfSeriesNotExist(name []byte) (bool, error)

DropMeasurementIfSeriesNotExist drops a measurement only if there are no more series for the measurment.

func (*Index) DropSeriesGlobal added in v1.5.0 

func (i *Index) DropSeriesGlobal(key []byte) error

DropSeriesGlobal removes the series key and its tags from the index.

func (*Index) FieldSet added in v1.5.0 

func (i *Index) FieldSet() *tsdb.MeasurementFieldSet

FieldSet returns the assigned fieldset.

func (*Index) ForEachMeasurementName 

func (i *Index) ForEachMeasurementName(fn func(name []byte) error) error

ForEachMeasurementName iterates over each measurement name.

func (*Index) ForEachMeasurementTagKey 

func (i *Index) ForEachMeasurementTagKey(name []byte, fn func(key []byte) error) error

ForEachMeasurementTagKey iterates over all tag keys for a measurement.

func (*Index) HasTagKey 

func (i *Index) HasTagKey(name, key []byte) (bool, error)

HasTagKey returns true if tag key exists.

func (*Index) HasTagValue 

func (i *Index) HasTagValue(name, key, value []byte) (bool, error)

HasTagValue returns true if tag value exists.

func (*Index) Measurement 

func (i *Index) Measurement(name []byte) (*measurement, error)

Measurement returns the measurement object from the index by the name

func (*Index) MeasurementExists 

func (i *Index) MeasurementExists(name []byte) (bool, error)

MeasurementExists returns true if the measurement exists.

func (*Index) MeasurementIterator added in v1.5.0 

func (i *Index) MeasurementIterator() (tsdb.MeasurementIterator, error)

MeasurementIterator returns an iterator over all measurements in the index. MeasurementIterator does not support authorization.

func (*Index) MeasurementNamesByExpr 

func (i *Index) MeasurementNamesByExpr(auth query.FineAuthorizer, expr influxql.Expr) ([][]byte, error)

MeasurementNamesByExpr takes an expression containing only tags and returns a list of matching measurement names.

TODO(edd): Remove authorisation from these methods. There shouldn't need to be any auth passed down into the index.

func (*Index) MeasurementNamesByRegex 

func (i *Index) MeasurementNamesByRegex(re *regexp.Regexp) ([][]byte, error)

MeasurementNamesByRegex returns the measurements that match the regex.

func (*Index) MeasurementSeriesIDIterator added in v1.5.0 

func (i *Index) MeasurementSeriesIDIterator(name []byte) (tsdb.SeriesIDIterator, error)

func (*Index) MeasurementSeriesKeysByExpr 

func (i *Index) MeasurementSeriesKeysByExpr(name []byte, condition influxql.Expr) ([][]byte, error)

func (*Index) MeasurementSeriesKeysByExprIterator added in v1.5.0 

func (i *Index) MeasurementSeriesKeysByExprIterator(name []byte, condition influxql.Expr) (tsdb.SeriesIDIterator, error)

func (*Index) MeasurementTagKeyValuesByExpr added in v1.3.2 

func (i *Index) MeasurementTagKeyValuesByExpr(auth query.FineAuthorizer, name []byte, keys []string, expr influxql.Expr, keysSorted bool) ([][]string, error)

MeasurementTagKeyValuesByExpr returns a set of tag values filtered by an expression.

See tsm1.Engine.MeasurementTagKeyValuesByExpr for a fuller description of this method.

func (*Index) MeasurementTagKeysByExpr 

func (i *Index) MeasurementTagKeysByExpr(name []byte, expr influxql.Expr) (map[string]struct{}, error)

MeasurementTagKeysByExpr returns an ordered set of tag keys filtered by an expression.

func (*Index) MeasurementsByName 

func (i *Index) MeasurementsByName(names [][]byte) ([]*measurement, error)

MeasurementsByName returns a list of measurements.

func (*Index) MeasurementsSketches 

func (i *Index) MeasurementsSketches() (estimator.Sketch, estimator.Sketch, error)

MeasurementsSketches returns the sketches for the measurements.

func (*Index) Open 

func (i *Index) Open() (err error)

func (*Index) Rebuild added in v1.4.0 

func (i *Index) Rebuild()

Rebuild recreates the measurement indexes to allow deleted series to be removed and garbage collected.

func (*Index) Series 

func (i *Index) Series(key []byte) (*series, error)

Series returns a series by key.

func (*Index) SeriesIDIterator added in v1.5.0 

func (i *Index) SeriesIDIterator(opt query.IteratorOptions) (tsdb.SeriesIDIterator, error)

SeriesIDIterator returns an influxql iterator over matching series ids.

func (*Index) SeriesKeys 

func (i *Index) SeriesKeys() []string

func (*Index) SeriesSketches 

func (i *Index) SeriesSketches() (estimator.Sketch, estimator.Sketch, error)

SeriesSketches returns the sketches for the series.

func (*Index) SetFieldName 

func (i *Index) SetFieldName(measurement []byte, name string)

SetFieldName adds a field name to a measurement.

func (*Index) SetFieldSet 

func (i *Index) SetFieldSet(fieldset *tsdb.MeasurementFieldSet)

SetFieldSet sets a shared field set from the engine.

func (*Index) TagKeyCardinality 

func (i *Index) TagKeyCardinality(name, key []byte) int

TagKeyCardinality returns the number of values for a measurement/tag key.

func (*Index) TagKeyHasAuthorizedSeries added in v1.4.3 

func (i *Index) TagKeyHasAuthorizedSeries(auth query.FineAuthorizer, name []byte, key string) bool

TagKeyHasAuthorizedSeries determines if there exists an authorized series for the provided measurement name and tag key.

func (*Index) TagKeyIterator added in v1.5.0 

func (i *Index) TagKeyIterator(name []byte) (tsdb.TagKeyIterator, error)

func (*Index) TagKeySeriesIDIterator added in v1.5.0 

func (i *Index) TagKeySeriesIDIterator(name, key []byte) (tsdb.SeriesIDIterator, error)

func (*Index) TagSets 

func (i *Index) TagSets(shardSeriesIDs *tsdb.SeriesIDSet, name []byte, opt query.IteratorOptions) ([]*query.TagSet, error)

TagSets returns a list of tag sets.

func (*Index) TagValueIterator added in v1.5.0 

func (i *Index) TagValueIterator(name, key []byte) (tsdb.TagValueIterator, error)

TagValueIterator provides an iterator over all the tag values belonging to series with the provided measurement name and tag key.

TagValueIterator does not currently support authorization.

func (*Index) TagValueN 

func (i *Index) TagValueN(name, key []byte) int

TagValueN returns the cardinality of a tag value.

func (*Index) TagValueSeriesIDIterator added in v1.5.0 

func (i *Index) TagValueSeriesIDIterator(name, key, value []byte) (tsdb.SeriesIDIterator, error)

func (*Index) TagsForSeries 

func (i *Index) TagsForSeries(key string) (models.Tags, error)

TagsForSeries returns the tag map for the passed in series

func (*Index) Type 

func (i *Index) Type() string

func (*Index) UniqueReferenceID added in v1.6.0 

func (i *Index) UniqueReferenceID() uintptr

func (*Index) WithLogger 

func (i *Index) WithLogger(*zap.Logger)

type ShardIndex 

type ShardIndex struct {
	*Index // Shared reference to global database-wide index.
	// contains filtered or unexported fields
}

ShardIndex represents a shim between the TSDB index interface and the shared in-memory index. This is required because per-shard in-memory indexes will grow the heap size too large.

func (*ShardIndex) CreateSeriesIfNotExists 

func (idx *ShardIndex) CreateSeriesIfNotExists(key, name []byte, tags models.Tags, tracker tsdb.StatsTracker) error

CreateSeriesIfNotExists creates the provided series on the index if it is not already present.

func (*ShardIndex) CreateSeriesListIfNotExists 

func (idx *ShardIndex) CreateSeriesListIfNotExists(keys, names [][]byte, tagsSlice []models.Tags, tracker tsdb.StatsTracker) error

CreateSeriesListIfNotExists creates a list of series if they doesn't exist in bulk.

func (*ShardIndex) DropMeasurementIfSeriesNotExist added in v1.5.0 

func (idx *ShardIndex) DropMeasurementIfSeriesNotExist(name []byte) (bool, error)

DropMeasurementIfSeriesNotExist drops a measurement only if there are no more series for the measurment.

func (*ShardIndex) DropSeries added in v1.5.0 

func (idx *ShardIndex) DropSeries(seriesID uint64, key []byte, _ bool) error

DropSeries removes the provided series id from the local bitset that tracks series in this shard only.

func (*ShardIndex) DropSeriesList added in v1.8.1 

func (idx *ShardIndex) DropSeriesList(seriesIDs []uint64, keys [][]byte, _ bool) error

DropSeriesList removes the provided series ids from the local bitset that tracks series in this shard only.

func (*ShardIndex) InitializeSeries 

func (idx *ShardIndex) InitializeSeries(keys, names [][]byte, tags []models.Tags) error

InitializeSeries is called during start-up. This works the same as CreateSeriesListIfNotExists except it ignore limit errors.

func (*ShardIndex) SeriesIDSet added in v1.5.0 

func (idx *ShardIndex) SeriesIDSet() *tsdb.SeriesIDSet

SeriesIDSet returns the bitset associated with the series ids.

func (*ShardIndex) SeriesN added in v1.5.0 

func (idx *ShardIndex) SeriesN() int64

SeriesN returns the number of unique non-tombstoned series local to this shard.

func (*ShardIndex) TagSets 

func (idx *ShardIndex) TagSets(name []byte, opt query.IteratorOptions) ([]*query.TagSet, error)

TagSets returns a list of tag sets based on series filtering.

type TagFilter 

type TagFilter struct {
	Op    influxql.Token
	Key   string
	Value string
	Regex *regexp.Regexp
}

TagFilter represents a tag filter when looking up other tags or measurements.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.