Documentation ¶
Overview ¶
Package boltutil provides higher level primitives to work with bolt DB.
Index ¶
- Constants
- type BucketStatsMetric
- type Config
- type DbMetric
- type DbStatsMetric
- type IndexedBucket
- func (ix *IndexedBucket) Delete(keys []string) error
- func (ix *IndexedBucket) DeleteTx(tx *bolt.Tx, keys []string) error
- func (ix *IndexedBucket) Insert(recs []Record) error
- func (ix *IndexedBucket) List(offset, size int) ([]Record, int, error)
- func (ix *IndexedBucket) ReIndex() error
- func (ix *IndexedBucket) Read(keys []string) ([]Record, error)
- func (ix *IndexedBucket) ReadIndex(idx string, keys []string) (map[string][]string, error)
- func (ix *IndexedBucket) ReadIndexTx(tx *bolt.Tx, idx string, keys []string) (map[string][]string, error)
- func (ix *IndexedBucket) ReadRaw(key string) ([]byte, error)
- func (ix *IndexedBucket) ReadTx(tx *bolt.Tx, keys []string) ([]Record, error)
- func (ix *IndexedBucket) Update(inputRecs []Record, writeFn WriteFn) error
- func (ix *IndexedBucket) UpdateTx(tx *bolt.Tx, inputRecs []Record, writeFn WriteFn) error
- type Record
- type TxStatsMetric
- type WriteFn
Constants ¶
const ( // Maximum length of the value for an index. This results from us storing the // length in a two byte uint. MAX_INDEX_VAL_LEN = 65535 // META_DATA_BUCKET_PREFIX is the prefix of the bucket name used for meta data. META_DATA_BUCKET_PREFIX = "_meta_" // META_DATA_KEY is the key under which meta data are stored. META_DATA_KEY = "meta" // BUILD_INDEX_BATCH_SIZE is the batchsize used when building new indices. BUILD_INDEX_BATCH_SIZE = 100 )
const (
DEFAULT_REPORT_FREQUENCY = time.Minute * 10
)
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type BucketStatsMetric ¶
type BucketStatsMetric struct { // Page count statistics. BranchPageN metrics2.Int64Metric // number of logical branch pages BranchOverflowN metrics2.Int64Metric // number of physical branch overflow pages LeafPageN metrics2.Int64Metric // number of logical leaf pages LeafOverflowN metrics2.Int64Metric // number of physical leaf overflow pages // Tree statistics. KeyN metrics2.Int64Metric // number of keys/value pairs Depth metrics2.Int64Metric // number of levels in B+tree // Page size utilization. BranchAlloc metrics2.Int64Metric // bytes allocated for physical branch pages BranchInuse metrics2.Int64Metric // bytes actually used for branch data LeafAlloc metrics2.Int64Metric // bytes allocated for physical leaf pages LeafInuse metrics2.Int64Metric // bytes actually used for leaf data // Bucket statistics BucketN metrics2.Int64Metric // total number of buckets including the top bucket InlineBucketN metrics2.Int64Metric // total number of inlined buckets InlineBucketInuse metrics2.Int64Metric // bytes used for inlined buckets (also accounted for in LeafInuse) }
BucketStatsMetric contains sub-metrics for each field of bolt.BucketStats. Create via NewDbMetric.
func (*BucketStatsMetric) Delete ¶
func (m *BucketStatsMetric) Delete() error
Delete deletes all sub-metrics.
func (*BucketStatsMetric) Update ¶
func (m *BucketStatsMetric) Update(cur bolt.BucketStats)
Update sets all sub-metrics from cur.
type Config ¶
type Config struct { // DB is the boltdb instance to store the indexed bucket. DB *bolt.DB // Name is the primary name of the database. Name string // Indices is the list of indices that should be created for every record stored. Indices []string // Codec is used to serialize and deserialize records. It has to consume // and produce instances that implement the Record interface. Codec util.LRUCodec }
Config contains the configuration values to set up a IndexedBucket instance.
type DbMetric ¶
type DbMetric struct { Liveness metrics2.Liveness DbStatsMetric *DbStatsMetric BucketStatsMetrics map[string]*BucketStatsMetric // contains filtered or unexported fields }
DbMetric gathers and reports a number of statistics about a BoltDB using the metrics2 package.
func NewDbMetric ¶
NewDbMetric initializes a DbMetric and starts a goroutine to periodically update the sub-metrics from the given bolt.DB. Bucket stats are reported only for the given buckets. tags should include "database" and should not include "metric" or "bucket_path". Returns an error if the initial update fails for any reason.
func NewDbMetricWithClient ¶
func NewDbMetricWithClient(c metrics2.Client, d *bolt.DB, bucketNames []string, tags ...map[string]string) (*DbMetric, error)
NewDbMetricWithClient is the same as NewDbMetric, but uses the specified metrics2.Client rather than the default client.
type DbStatsMetric ¶
type DbStatsMetric struct { // Freelist stats FreePageN metrics2.Int64Metric // total number of free pages on the freelist PendingPageN metrics2.Int64Metric // total number of pending pages on the freelist FreeAlloc metrics2.Int64Metric // total bytes allocated in free pages FreelistInuse metrics2.Int64Metric // total bytes used by the freelist // Transaction stats TxN metrics2.Int64Metric // total number of started read transactions OpenTxN metrics2.Int64Metric // number of currently open read transactions TxStatsMetric *TxStatsMetric // global, ongoing stats. }
DbStatsMetric contains sub-metrics for each field of bolt.Stats. Create via NewDbMetric.
func (*DbStatsMetric) Delete ¶
func (m *DbStatsMetric) Delete() error
Delete deletes all sub-metrics.
func (*DbStatsMetric) Update ¶
func (m *DbStatsMetric) Update(cur bolt.Stats)
Update sets sub-metrics based on cur.
type IndexedBucket ¶
type IndexedBucket struct { // BoltDB instance used by this store. DB *bolt.DB // contains filtered or unexported fields }
IndexedBucket embeds a bolt database and adds higher level functions to add and maintain indices. Indices allow to read records by keys other than the primary key. It aims at following the boltdb patterns and allows to still use the underlying bolt DB directly if necessary.
func NewIndexedBucket ¶
func NewIndexedBucket(config *Config) (*IndexedBucket, error)
NewIndexedBucket returns a new instance of IndexedBucket. Since it uses an existing BoltDB instance, it is up to the caller to close it.
func (*IndexedBucket) Delete ¶
func (ix *IndexedBucket) Delete(keys []string) error
Delete deletes the records identified by keys from the database and updates the indices accordingly.
func (*IndexedBucket) DeleteTx ¶
func (ix *IndexedBucket) DeleteTx(tx *bolt.Tx, keys []string) error
DeleteTx does the same as Delete but uses an existing transaction.
func (*IndexedBucket) Insert ¶
func (ix *IndexedBucket) Insert(recs []Record) error
func (*IndexedBucket) List ¶
func (ix *IndexedBucket) List(offset, size int) ([]Record, int, error)
List returns a list of records sorted in ascending order by the primary key starting with record at offset. If size <= 0 all records will be returned. Note: This can be take a long time for large values of offset and/or size. Results might not be consistent across multiple calls since they might be interleaved with write operations.
func (*IndexedBucket) ReIndex ¶
func (ix *IndexedBucket) ReIndex() error
ReIndex rebuilds the indices that were specified when the instance was created. This will block until all indices are rebuild.
func (*IndexedBucket) Read ¶
func (ix *IndexedBucket) Read(keys []string) ([]Record, error)
Read reads the records identified by keys. The returned slice has the exact same number of elements as keys. If a record could not be found the corresponding entry in the return value will be nil.
func (*IndexedBucket) ReadIndex ¶
ReadIndex returns the record keys for the given index name and list of index values. The return value maps from index values (secondary key values) to a list of primary keys.
func (*IndexedBucket) ReadIndexTx ¶
func (ix *IndexedBucket) ReadIndexTx(tx *bolt.Tx, idx string, keys []string) (map[string][]string, error)
ReadIndexTx does the same as ReadIndex but uses an existing transaction.
func (*IndexedBucket) ReadRaw ¶
func (ix *IndexedBucket) ReadRaw(key string) ([]byte, error)
ReadRaw reads the bytes for a given key. If the key does not exist the returned byte slice is nil.
func (*IndexedBucket) Update ¶
func (ix *IndexedBucket) Update(inputRecs []Record, writeFn WriteFn) error
Update allows to update records in the database by following these steps:
- read existing records from the database, identified by the Key() value of inputRecs.
- call the writeFn to allow the client to update the records. If it does not want a record to be updated it will set its entry to nil.
type Record ¶
type Record interface { // Key returns the primary key of the record. The return value does not have // to be a human readable string it can be any sequence of bytes. The // returned string has to uniquely identify this instance. Key() string // IndexValues returns the values of the type for the attributes we want to // index. The return value maps from the index names to the list of values for // that index (= secondary key values): map[index_name][]{val_1, val_2, val_n}. // Any key in the returned map must also be given in Indices field of the // Config instance passed to NewStore(...). // Note: Individual values for each index cannot be longer than 65535. IndexValues() map[string][]string }
Record is the interface that has to be implemented by a client to store instance of a type in a IndexedBucket instance. It allows to extract relevant key information about each record.
type TxStatsMetric ¶
type TxStatsMetric struct { // Page statistics. PageCount metrics2.Int64Metric // number of page allocations PageAlloc metrics2.Int64Metric // total bytes allocated // Cursor statistics. CursorCount metrics2.Int64Metric // number of cursors created // Node statistics NodeCount metrics2.Int64Metric // number of node allocations NodeDeref metrics2.Int64Metric // number of node dereferences // Rebalance statistics. Rebalance metrics2.Int64Metric // number of node rebalances RebalanceTime metrics2.Int64Metric // total time spent rebalancing // Split/Spill statistics. Split metrics2.Int64Metric // number of nodes split Spill metrics2.Int64Metric // number of nodes spilled SpillTime metrics2.Int64Metric // total time spent spilling // Write statistics. Write metrics2.Int64Metric // number of writes performed WriteTime metrics2.Int64Metric // total time spent writing to disk }
TxStatsMetric contains sub-metrics for each field of the bolt.TxStats from bolt.DB.Stats(). Create via NewDbMetric.
TxStatsMetric does not use aggregating metrics, so it's unlikely to work well for per-Tx TxStats.
func (*TxStatsMetric) Delete ¶
func (m *TxStatsMetric) Delete() error
Delete deletes all sub-metrics.
func (*TxStatsMetric) Update ¶
func (m *TxStatsMetric) Update(cur bolt.TxStats)
Update sets all sub-metrics from cur.
type WriteFn ¶
WriteFn is the type of the callback function that can be handed to the Write method to update existing records before they are overwritten with a new instance. The passed records are the current records in the database (before new values are written). The WriteFn can then modify records in place and they will be written to disk. To prevent a record from being written back into the database it should set the entry to nil.