db

package module
v0.4.1-stable Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 18, 2020 License: Apache-2.0 Imports: 14 Imported by: 0

README

Tendermint DB

Test Lint

Common database interface for various database backends. Primarily meant for applications built on Tendermint, such as the Cosmos SDK, but can be used independently of these as well.

Minimum Go Version

Go 1.13+

Supported Database Backends

  • GoLevelDB [stable]: A pure Go implementation of LevelDB (see below). Currently the default on-disk database used in the Cosmos SDK.

  • MemDB [stable]: An in-memory database using Google's B-tree package. Has very high performance both for reads, writes, and range scans, but is not durable and will lose all data on process exit. Does not support transactions. Suitable for e.g. caches, working sets, and tests. Used for IAVL working sets when the pruning strategy allows it.

  • LevelDB [experimental]: A Go wrapper around LevelDB. Uses LSM-trees for on-disk storage, which have good performance for write-heavy workloads, particularly on spinning disks, but requires periodic compaction to maintain decent read performance and reclaim disk space. Does not support transactions.

  • BoltDB [experimental]: A fork of BoltDB. Uses B+trees for on-disk storage, which have good performance for read-heavy workloads and range scans. Supports serializable ACID transactions.

  • RocksDB [experimental]: A Go wrapper around RocksDB. Similarly to LevelDB (above) it uses LSM-trees for on-disk storage, but is optimized for fast storage media such as SSDs and memory. Supports atomic transactions, but not full ACID transactions.

Meta-databases

  • PrefixDB [stable]: A database which wraps another database and uses a static prefix for all keys. This allows multiple logical databases to be stored in a common underlying databases by using different namespaces. Used by the Cosmos SDK to give different modules their own namespaced database in a single application database.

  • RemoteDB [experimental]: A database that connects to distributed Tendermint db instances via gRPC. This can help with detaching difficult deployments such as LevelDB, and can also ease dependency management for Tendermint developers.

Tests

To test common databases, run make test. If all databases are available on the local machine, use make test-all to test them all.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FileExists

func FileExists(filePath string) bool

func IsKeyInDomain

func IsKeyInDomain(key, start, end []byte) bool

See DB interface documentation for more information.

Types

type BackendType

type BackendType string
const (
	// GoLevelDBBackend represents goleveldb (github.com/syndtr/goleveldb - most
	// popular implementation)
	//   - pure go
	//   - stable
	GoLevelDBBackend BackendType = "goleveldb"
	// CLevelDBBackend represents cleveldb (uses levigo wrapper)
	//   - fast
	//   - requires gcc
	//   - use cleveldb build tag (go build -tags cleveldb)
	CLevelDBBackend BackendType = "cleveldb"
	// MemDBBackend represents in-memory key value store, which is mostly used
	// for testing.
	MemDBBackend BackendType = "memdb"
	// BoltDBBackend represents bolt (uses etcd's fork of bolt -
	// github.com/etcd-io/bbolt)
	//   - EXPERIMENTAL
	//   - may be faster is some use-cases (random reads - indexer)
	//   - use boltdb build tag (go build -tags boltdb)
	BoltDBBackend BackendType = "boltdb"
	// RocksDBBackend represents rocksdb (uses github.com/tecbot/gorocksdb)
	//   - EXPERIMENTAL
	//   - requires gcc
	//   - use rocksdb build tag (go build -tags rocksdb)
	RocksDBBackend BackendType = "rocksdb"
)

These are valid backend types.

type Batch

type Batch interface {
	SetDeleter

	// Write writes the batch, possibly without flushing to disk. Only Close() can be called after,
	// other methods will panic.
	Write() error

	// WriteSync writes the batch and flushes it to disk. Only Close() can be called after, other
	// methods will panic.
	WriteSync() error

	// Close closes the batch. It is idempotent, but any other calls afterwards will panic.
	Close()
}

Batch Close must be called when the program no longer needs the object.

type DB

type DB interface {

	// Get returns nil if key doesn't exist.
	// A nil key is interpreted as an empty byteslice.
	// CONTRACT: key, value readonly []byte
	Get([]byte) ([]byte, error)

	// Has checks if a key exists.
	// A nil key is interpreted as an empty byteslice.
	// CONTRACT: key, value readonly []byte
	Has(key []byte) (bool, error)

	// Set sets the key.
	// A nil key is interpreted as an empty byteslice.
	// CONTRACT: key, value readonly []byte
	Set([]byte, []byte) error
	SetSync([]byte, []byte) error

	// Delete deletes the key.
	// A nil key is interpreted as an empty byteslice.
	// CONTRACT: key readonly []byte
	Delete([]byte) error
	DeleteSync([]byte) error

	// Iterate over a domain of keys in ascending order. End is exclusive.
	// Start must be less than end, or the Iterator is invalid.
	// A nil start is interpreted as an empty byteslice.
	// If end is nil, iterates up to the last item (inclusive).
	// CONTRACT: No writes may happen within a domain while an iterator exists over it.
	// CONTRACT: start, end readonly []byte
	Iterator(start, end []byte) (Iterator, error)

	// Iterate over a domain of keys in descending order. End is exclusive.
	// Start must be less than end, or the Iterator is invalid.
	// If start is nil, iterates up to the first/least item (inclusive).
	// If end is nil, iterates from the last/greatest item (inclusive).
	// CONTRACT: No writes may happen within a domain while an iterator exists over it.
	// CONTRACT: start, end readonly []byte
	ReverseIterator(start, end []byte) (Iterator, error)

	// Closes the connection.
	Close() error

	// Creates a batch for atomic updates. The caller must call Batch.Close.
	NewBatch() Batch

	// For debugging
	Print() error

	// Stats returns a map of property values for all keys and the size of the cache.
	Stats() map[string]string
}

DBs are goroutine safe.

func NewDB

func NewDB(name string, backend BackendType, dir string) DB

NewDB creates a new database of type backend with the given name. NOTE: function panics if:

  • backend is unknown (not registered)
  • creator function, provided during registration, returns error

type GoLevelDB

type GoLevelDB struct {
	// contains filtered or unexported fields
}

func NewGoLevelDB

func NewGoLevelDB(name string, dir string) (*GoLevelDB, error)

func NewGoLevelDBWithOpts

func NewGoLevelDBWithOpts(name string, dir string, o *opt.Options) (*GoLevelDB, error)

func (*GoLevelDB) Close

func (db *GoLevelDB) Close() error

Close implements DB.

func (*GoLevelDB) DB

func (db *GoLevelDB) DB() *leveldb.DB

func (*GoLevelDB) Delete

func (db *GoLevelDB) Delete(key []byte) error

Delete implements DB.

func (*GoLevelDB) DeleteSync

func (db *GoLevelDB) DeleteSync(key []byte) error

DeleteSync implements DB.

func (*GoLevelDB) Get

func (db *GoLevelDB) Get(key []byte) ([]byte, error)

Get implements DB.

func (*GoLevelDB) Has

func (db *GoLevelDB) Has(key []byte) (bool, error)

Has implements DB.

func (*GoLevelDB) Iterator

func (db *GoLevelDB) Iterator(start, end []byte) (Iterator, error)

Iterator implements DB.

func (*GoLevelDB) NewBatch

func (db *GoLevelDB) NewBatch() Batch

NewBatch implements DB.

func (*GoLevelDB) Print

func (db *GoLevelDB) Print() error

Print implements DB.

func (*GoLevelDB) ReverseIterator

func (db *GoLevelDB) ReverseIterator(start, end []byte) (Iterator, error)

ReverseIterator implements DB.

func (*GoLevelDB) Set

func (db *GoLevelDB) Set(key []byte, value []byte) error

Set implements DB.

func (*GoLevelDB) SetSync

func (db *GoLevelDB) SetSync(key []byte, value []byte) error

SetSync implements DB.

func (*GoLevelDB) Stats

func (db *GoLevelDB) Stats() map[string]string

Stats implements DB.

type Iterator

type Iterator interface {

	// The start & end (exclusive) limits to iterate over.
	// If end < start, then the Iterator goes in reverse order.
	//
	// A domain of ([]byte{12, 13}, []byte{12, 14}) will iterate
	// over anything with the prefix []byte{12, 13}.
	//
	// The smallest key is the empty byte array []byte{} - see BeginningKey().
	// The largest key is the nil byte array []byte(nil) - see EndingKey().
	// CONTRACT: start, end readonly []byte
	Domain() (start []byte, end []byte)

	// Valid returns whether the current position is valid.
	// Once invalid, an Iterator is forever invalid.
	Valid() bool

	// Next moves the iterator to the next sequential key in the database, as
	// defined by order of iteration.
	// If Valid returns false, this method will panic.
	Next()

	// Key returns the key of the cursor.
	// If Valid returns false, this method will panic.
	// CONTRACT: key readonly []byte
	Key() (key []byte)

	// Value returns the value of the cursor.
	// If Valid returns false, this method will panic.
	// CONTRACT: value readonly []byte
	Value() (value []byte)

	Error() error

	// Close releases the Iterator.
	Close()
}

Usage:

var itr Iterator = ... defer itr.Close()

for ; itr.Valid(); itr.Next() {
	k, v := itr.Key(); itr.Value()
	...
}

func IteratePrefix

func IteratePrefix(db DB, prefix []byte) (Iterator, error)

IteratePrefix is a convenience function for iterating over a key domain restricted by prefix.

type MemDB

type MemDB struct {
	// contains filtered or unexported fields
}

MemDB is an in-memory database backend using a B-tree for storage.

func NewMemDB

func NewMemDB() *MemDB

NewMemDB creates a new in-memory database.

func (*MemDB) Close

func (db *MemDB) Close() error

Close implements DB.

func (*MemDB) Delete

func (db *MemDB) Delete(key []byte) error

Delete implements DB.

func (*MemDB) DeleteSync

func (db *MemDB) DeleteSync(key []byte) error

DeleteSync implements DB.

func (*MemDB) Get

func (db *MemDB) Get(key []byte) ([]byte, error)

Get implements DB.

func (*MemDB) Has

func (db *MemDB) Has(key []byte) (bool, error)

Has implements DB.

func (*MemDB) Iterator

func (db *MemDB) Iterator(start, end []byte) (Iterator, error)

Iterator implements DB. Takes out a read-lock on the database until the iterator is closed.

func (*MemDB) NewBatch

func (db *MemDB) NewBatch() Batch

NewBatch implements DB.

func (*MemDB) Print

func (db *MemDB) Print() error

Print implements DB.

func (*MemDB) ReverseIterator

func (db *MemDB) ReverseIterator(start, end []byte) (Iterator, error)

ReverseIterator implements DB. Takes out a read-lock on the database until the iterator is closed.

func (*MemDB) Set

func (db *MemDB) Set(key []byte, value []byte) error

Set implements DB.

func (*MemDB) SetSync

func (db *MemDB) SetSync(key []byte, value []byte) error

SetSync implements DB.

func (*MemDB) Stats

func (db *MemDB) Stats() map[string]string

Stats implements DB.

type PrefixDB

type PrefixDB struct {
	// contains filtered or unexported fields
}

PrefixDB wraps a namespace of another database as a logical database.

func NewPrefixDB

func NewPrefixDB(db DB, prefix []byte) *PrefixDB

NewPrefixDB lets you namespace multiple DBs within a single DB.

func (*PrefixDB) Close

func (pdb *PrefixDB) Close() error

Close implements DB.

func (*PrefixDB) Delete

func (pdb *PrefixDB) Delete(key []byte) error

Delete implements DB.

func (*PrefixDB) DeleteSync

func (pdb *PrefixDB) DeleteSync(key []byte) error

DeleteSync implements DB.

func (*PrefixDB) Get

func (pdb *PrefixDB) Get(key []byte) ([]byte, error)

Get implements DB.

func (*PrefixDB) Has

func (pdb *PrefixDB) Has(key []byte) (bool, error)

Has implements DB.

func (*PrefixDB) Iterator

func (pdb *PrefixDB) Iterator(start, end []byte) (Iterator, error)

Iterator implements DB.

func (*PrefixDB) NewBatch

func (pdb *PrefixDB) NewBatch() Batch

NewBatch implements DB.

func (*PrefixDB) Print

func (pdb *PrefixDB) Print() error

Print implements DB.

func (*PrefixDB) ReverseIterator

func (pdb *PrefixDB) ReverseIterator(start, end []byte) (Iterator, error)

ReverseIterator implements DB.

func (*PrefixDB) Set

func (pdb *PrefixDB) Set(key []byte, value []byte) error

Set implements DB.

func (*PrefixDB) SetSync

func (pdb *PrefixDB) SetSync(key []byte, value []byte) error

SetSync implements DB.

func (*PrefixDB) Stats

func (pdb *PrefixDB) Stats() map[string]string

Stats implements DB.

type SetDeleter

type SetDeleter interface {
	// Set sets a key/value pair.
	// CONTRACT: key, value readonly []byte
	Set(key, value []byte)

	// Delete deletes a key/value pair.
	// CONTRACT: key readonly []byte
	Delete(key []byte)
}

Directories

Path Synopsis
remotedb is a package for connecting to distributed Tendermint db.DB instances.
remotedb is a package for connecting to distributed Tendermint db.DB instances.
grpcdb
grpcdb is the distribution of Tendermint's db.DB instances using the gRPC transport to decouple local db.DB usages from applications, to using them over a network in a highly performant manner.
grpcdb is the distribution of Tendermint's db.DB instances using the gRPC transport to decouple local db.DB usages from applications, to using them over a network in a highly performant manner.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL