storage

package
v0.0.0-...-298395c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 25, 2019 License: MIT Imports: 19 Imported by: 0

Documentation

Overview

The storage package provides a key/value based interface for storing Kapacitor metadata. All services wishing to store data should use this interface.

The usage patterns for this storage layer are typical create/replace/delete/get/list operations. Typically objects are serialized and stored as the value. As a result, updates to a single field of an object can incur the cost to retrieve the entire object and store it again. In most cases this is acceptable since modifications are rare and object size is small.

A BoltDB backed implementation is also provided.

Index

Constants

View Source
const (
	DefaultIDIndex = "id"
)

Variables

View Source
var (
	ErrObjectExists   = errors.New("object already exists")
	ErrNoObjectExists = errors.New("no object exists")
)
View Source
var (
	ErrNoKeyExists = errors.New("no key exists")
)

Common errors that can be returned

Functions

func DoListFunc

func DoListFunc(list []*KeyValue, match func(value []byte) bool, offset, limit int) []string

Return a list of values from a list of KeyValues using an offset/limit bound and a match function.

func DoUpdate

func DoUpdate(o TxOperator, f func(Tx) error) error

DoUpdate provides a complete implementation of Interface.Update for a TxOperator.

func DoView

func DoView(o TxOperator, f func(ReadOnlyTx) error) error

View manages a read only transaction.

func ImpossibleTypeErr

func ImpossibleTypeErr(exp interface{}, got interface{}) error

func VersionEasyJSONDecode

func VersionEasyJSONDecode(data []byte, decF func(version int, dec *jlexer.Lexer) error) error

VersionEasyJSONDecode decodes and object that was encoded using VersionJSONEncode.

func VersionJSONDecode

func VersionJSONDecode(data []byte, decF func(version int, dec *json.Decoder) error) error

VersionJSONDecode decodes and object that was encoded using VersionJSONEncode.

func VersionJSONEncode

func VersionJSONEncode(version int, o interface{}) ([]byte, error)

VersionJSONEncode encodes an object as json wrapping it in a VersionWrapper struct.

Types

type APIServer

type APIServer struct {
	Registrar StoreActionerRegistrar
	DB        *bolt.DB

	HTTPDService interface {
		AddRoutes([]httpd.Route) error
		DelRoutes([]httpd.Route)
	}
	// contains filtered or unexported fields
}

func (*APIServer) Close

func (s *APIServer) Close() error

func (*APIServer) Open

func (s *APIServer) Open() error

type BinaryObject

type BinaryObject interface {
	encoding.BinaryMarshaler
	encoding.BinaryUnmarshaler
	ObjectID() string
}

type Bolt

type Bolt struct {
	// contains filtered or unexported fields
}

Bolt implementation of Store

func NewBolt

func NewBolt(db *bolt.DB, bucket string) *Bolt

func (*Bolt) BeginReadOnlyTx

func (b *Bolt) BeginReadOnlyTx() (ReadOnlyTx, error)

func (*Bolt) BeginTx

func (b *Bolt) BeginTx() (Tx, error)

func (*Bolt) Delete

func (b *Bolt) Delete(key string) error

func (*Bolt) Exists

func (b *Bolt) Exists(key string) (exists bool, err error)

func (*Bolt) Get

func (b *Bolt) Get(key string) (kv *KeyValue, err error)

func (*Bolt) List

func (b *Bolt) List(prefix string) (kvs []*KeyValue, err error)

func (*Bolt) Put

func (b *Bolt) Put(key string, value []byte) error

func (*Bolt) Update

func (b *Bolt) Update(f func(tx Tx) error) error

func (*Bolt) View

func (b *Bolt) View(f func(tx ReadOnlyTx) error) error

type Config

type Config struct {
	// Path to a boltdb database file.
	BoltDBPath string `toml:"boltdb"`
}

func NewConfig

func NewConfig() Config

func (Config) Validate

func (c Config) Validate() error

type Diagnostic

type Diagnostic interface {
	Error(msg string, err error)
}

type Index

type Index struct {
	Name      string
	ValueFunc ValueFunc
	Unique    bool
}

func (Index) ValueOf

func (idx Index) ValueOf(o BinaryObject) (string, error)

type IndexedStore

type IndexedStore struct {
	// contains filtered or unexported fields
}

Indexed provides basic CRUD operations and maintains indexes.

func NewIndexedStore

func NewIndexedStore(store Interface, c IndexedStoreConfig) (*IndexedStore, error)

func (*IndexedStore) Create

func (s *IndexedStore) Create(o BinaryObject) error

func (*IndexedStore) CreateTx

func (s *IndexedStore) CreateTx(tx Tx, o BinaryObject) error

func (*IndexedStore) Delete

func (s *IndexedStore) Delete(id string) error

func (*IndexedStore) DeleteTx

func (s *IndexedStore) DeleteTx(tx Tx, id string) error

func (*IndexedStore) Get

func (s *IndexedStore) Get(id string) (o BinaryObject, err error)

func (*IndexedStore) GetTx

func (s *IndexedStore) GetTx(tx ReadOnlyTx, id string) (BinaryObject, error)

func (*IndexedStore) List

func (s *IndexedStore) List(index, pattern string, offset, limit int) (objects []BinaryObject, err error)

List returns a list of objects that match a given pattern. If limit < 0, then no limit is enforced.

func (*IndexedStore) ListTx

func (s *IndexedStore) ListTx(tx ReadOnlyTx, index, pattern string, offset, limit int) ([]BinaryObject, error)

func (*IndexedStore) Put

func (s *IndexedStore) Put(o BinaryObject) error

func (*IndexedStore) PutTx

func (s *IndexedStore) PutTx(tx Tx, o BinaryObject) error

func (*IndexedStore) Rebuild

func (s *IndexedStore) Rebuild() error

Rebuild completely rebuilds all indexes for the store.

func (*IndexedStore) RebuildTx

func (s *IndexedStore) RebuildTx(tx Tx) error

Rebuild completely rebuilds all indexes for the store using the provided transaction.

func (*IndexedStore) Replace

func (s *IndexedStore) Replace(o BinaryObject) error

func (*IndexedStore) ReplaceTx

func (s *IndexedStore) ReplaceTx(tx Tx, o BinaryObject) error

func (*IndexedStore) ReverseList

func (s *IndexedStore) ReverseList(index, pattern string, offset, limit int) (objects []BinaryObject, err error)

ReverseList returns a list of objects that match a given pattern, using reverse sort. If limit < 0, then no limit is enforced.

func (*IndexedStore) ReverseListTx

func (s *IndexedStore) ReverseListTx(tx ReadOnlyTx, index, pattern string, offset, limit int) ([]BinaryObject, error)

type IndexedStoreConfig

type IndexedStoreConfig struct {
	Prefix        string
	DataPrefix    string
	IndexesPrefix string
	NewObject     NewObjectF
	Indexes       []Index
}

func DefaultIndexedStoreConfig

func DefaultIndexedStoreConfig(prefix string, newObject NewObjectF) IndexedStoreConfig

func (IndexedStoreConfig) Validate

func (c IndexedStoreConfig) Validate() error

type Interface

type Interface interface {

	// View creates a new read only transaction and always rolls it back.
	View(func(ReadOnlyTx) error) error

	// Update creates a new read-write transaction and always rolls it back.
	// If the function returns a nil error the transaction is committed, otherwise the error is returned.
	Update(func(Tx) error) error
}

Common interface for interacting with a simple Key/Value storage

type KeyValue

type KeyValue struct {
	Key   string
	Value []byte
}

type MemStore

type MemStore struct {
	Name string
	// contains filtered or unexported fields
}

MemStore is an in memory only implementation of the storage.Interface. This is intend to be used for testing use cases only.

func NewMemStore

func NewMemStore(name string) *MemStore

func (*MemStore) BeginReadOnlyTx

func (s *MemStore) BeginReadOnlyTx() (ReadOnlyTx, error)

func (*MemStore) BeginTx

func (s *MemStore) BeginTx() (Tx, error)

func (*MemStore) Delete

func (s *MemStore) Delete(key string) error

func (*MemStore) Exists

func (s *MemStore) Exists(key string) (bool, error)

func (*MemStore) Get

func (s *MemStore) Get(key string) (*KeyValue, error)

func (*MemStore) List

func (s *MemStore) List(prefix string) ([]*KeyValue, error)

func (*MemStore) Put

func (s *MemStore) Put(key string, value []byte) error

func (*MemStore) Update

func (s *MemStore) Update(f func(tx Tx) error) error

func (*MemStore) View

func (s *MemStore) View(f func(tx ReadOnlyTx) error) error

type NewObjectF

type NewObjectF func() BinaryObject

type ReadOnlyTx

type ReadOnlyTx interface {
	ReadOperator

	// Rollback signals that the transaction is complete.
	// If the transaction was not committed, then all changes are reverted.
	// Rollback must always be called for every transaction.
	Rollback() error
}

ReadOnlyTx provides an interface for performing read operations in a single transaction.

type ReadOperator

type ReadOperator interface {
	// Retrieve a value.
	Get(key string) (*KeyValue, error)
	// Check if a key exists>
	Exists(key string) (bool, error)
	// List all values with given prefix.
	List(prefix string) ([]*KeyValue, error)
}

ReadOperator provides an interface for performing read operations.

type Service

type Service struct {
	HTTPDService interface {
		AddRoutes([]httpd.Route) error
		DelRoutes([]httpd.Route)
	}
	// contains filtered or unexported fields
}

func NewService

func NewService(conf Config, d Diagnostic) *Service

func (*Service) Close

func (s *Service) Close() error

func (*Service) Open

func (s *Service) Open() error

func (*Service) Register

func (s *Service) Register(name string, store StoreActioner)

func (*Service) Store

func (s *Service) Store(name string) Interface

Return a namespaced store. Calling Store with the same namespace returns the same Store.

func (*Service) Versions

func (s *Service) Versions() Versions

type StoreActioner

type StoreActioner interface {
	// Rebuild the entire store, this should be considered to be an expensive action.
	Rebuild() error
}

StoreActioner exposes and interface for various actions that can be performed on a store.

type StoreActionerRegistrar

type StoreActionerRegistrar interface {
	List() []string
	Register(name string, store StoreActioner)
	Get(name string) (StoreActioner, bool)
}

func NewStorageResitrar

func NewStorageResitrar() StoreActionerRegistrar

type Tx

type Tx interface {
	ReadOnlyTx
	WriteOperator

	// Commit finalizes the transaction.
	// Once a transaction is committed, rolling back the transaction has no effect.
	Commit() error
}

Tx provides an interface for performing read and write storage operations in a single transaction.

type TxOperator

type TxOperator interface {
	// BeginReadOnlyTx starts a new read only transaction. The transaction must be rolledback.
	// Leaving a transaction open can block other operations and otherwise
	// significantly degrade the performance of the storage backend.
	// A single go routine should only have one transaction open at a time.
	BeginReadOnlyTx() (ReadOnlyTx, error)
	// BeginTx starts a new transaction for reads and writes. The transaction must be committed or rolledback.
	// Leaving a transaction open can block other operations and otherwise
	// significantly degrade the performance of the storage backend.
	// A single go routine should only have one transaction open at a time.
	BeginTx() (Tx, error)
}

type ValueFunc

type ValueFunc func(BinaryObject) (string, error)

type VersionWrapper

type VersionWrapper struct {
	Version int              `json:"version"`
	Value   *json.RawMessage `json:"value"`
}

VersionWrapper wraps a structure with a version so that changes to the structure can be properly decoded.

func (VersionWrapper) MarshalEasyJSON

func (v VersionWrapper) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (VersionWrapper) MarshalJSON

func (v VersionWrapper) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (*VersionWrapper) UnmarshalEasyJSON

func (v *VersionWrapper) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*VersionWrapper) UnmarshalJSON

func (v *VersionWrapper) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

type Versions

type Versions interface {
	Get(id string) (string, error)
	Set(id, version string) error
}

func NewVersions

func NewVersions(store Interface) Versions

type WriteOperator

type WriteOperator interface {
	// Store a value.
	Put(key string, value []byte) error
	// Delete a key.
	// Deleting a non-existent key is not an error.
	Delete(key string) error
}

WriteOperator provides an interface for performing write operations.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL