db

package
v0.0.0-...-259d925 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 7, 2017 License: BSD-3-Clause Imports: 5 Imported by: 57

Documentation

Overview

Package db defines the interfaces for a key/value store.

A DB's basic operations (Get, Set, Delete) should be self-explanatory. Get and Delete will return ErrNotFound if the requested key is not in the store. Callers are free to ignore this error.

A DB also allows for iterating over the key/value pairs in key order. If d is a DB, the code below prints all key/value pairs whose keys are 'greater than or equal to' k:

iter := d.Find(k, readOptions)
for iter.Next() {
	fmt.Printf("key=%q value=%q\n", iter.Key(), iter.Value())
}
return iter.Close()

Other leveldb packages provide implementations of these interfaces. The Options struct in this package holds the optional parameters for these implementations, including a Comparer to define a 'less than' relationship over keys. It is always valid to pass a nil *Options, which means to use the default parameter values. Any zero field of a non-nil *Options also means to use the default value for that parameter. Thus, the code below uses a custom Comparer, but the default values for every other parameter:

db := memdb.New(&db.Options{
	Comparer: myComparer,
})

Index

Constants

This section is empty.

Variables

View Source
var ErrNotFound = errors.New("leveldb/db: not found")

ErrNotFound means that a get or delete call did not find the requested key.

Functions

func SharedPrefixLen

func SharedPrefixLen(a, b []byte) int

SharedPrefixLen returns the largest i such that a[:i] equals b[:i]. This function can be useful in implementing the Comparer interface.

Types

type Comparer

type Comparer interface {
	// Compare returns -1, 0, or +1 depending on whether a is 'less than',
	// 'equal to' or 'greater than' b. The two arguments can only be 'equal'
	// if their contents are exactly equal. Furthermore, the empty slice
	// must be 'less than' any non-empty slice.
	Compare(a, b []byte) int

	// Name returns the name of the comparer.
	//
	// The Level-DB on-disk format stores the comparer name, and opening a
	// database with a different comparer from the one it was created with
	// will result in an error.
	Name() string

	// AppendSeparator appends a sequence of bytes x to dst such that
	// a <= x && x < b, where 'less than' is consistent with Compare.
	// It returns the enlarged slice, like the built-in append function.
	//
	// Precondition: either a is 'less than' b, or b is an empty slice.
	// In the latter case, empty means 'positive infinity', and appending any
	// x such that a <= x will be valid.
	//
	// An implementation may simply be "return append(dst, a...)" but appending
	// fewer bytes will result in smaller tables.
	//
	// For example, if dst, a and b are the []byte equivalents of the strings
	// "aqua", "black" and "blue", then the result may be "aquablb".
	// Similarly, if the arguments were "aqua", "green" and "", then the result
	// may be "aquah".
	AppendSeparator(dst, a, b []byte) []byte
}

Comparer defines a total ordering over the space of []byte keys: a 'less than' relationship.

var DefaultComparer Comparer = defCmp{}

DefaultComparer is the default implementation of the Comparer interface. It uses the natural ordering, consistent with bytes.Compare.

type Compression

type Compression int

Compression is the per-block compression algorithm to use.

const (
	DefaultCompression Compression = iota
	NoCompression
	SnappyCompression
)

type DB

type DB interface {
	// Get gets the value for the given key. It returns ErrNotFound if the DB
	// does not contain the key.
	//
	// The caller should not modify the contents of the returned slice, but
	// it is safe to modify the contents of the argument after Get returns.
	Get(key []byte, o *ReadOptions) (value []byte, err error)

	// Set sets the value for the given key. It overwrites any previous value
	// for that key; a DB is not a multi-map.
	//
	// It is safe to modify the contents of the arguments after Set returns.
	Set(key, value []byte, o *WriteOptions) error

	// Delete deletes the value for the given key. It returns ErrNotFound if
	// the DB does not contain the key.
	//
	// It is safe to modify the contents of the arguments after Delete returns.
	Delete(key []byte, o *WriteOptions) error

	// Find returns an iterator positioned before the first key/value pair
	// whose key is 'greater than or equal to' the given key. There may be no
	// such pair, in which case the iterator will return false on Next.
	//
	// Any error encountered will be implicitly returned via the iterator. An
	// error-iterator will yield no key/value pairs and closing that iterator
	// will return that error.
	//
	// It is safe to modify the contents of the argument after Find returns.
	Find(key []byte, o *ReadOptions) Iterator

	// Close closes the DB. It may or may not close any underlying io.Reader
	// or io.Writer, depending on how the DB was created.
	//
	// It is not safe to close a DB until all outstanding iterators are closed.
	// It is valid to call Close multiple times. Other methods should not be
	// called after the DB has been closed.
	Close() error
}

DB is a key/value store.

It is safe to call Get and Find from concurrent goroutines. It is not necessarily safe to do so for Set and Delete.

Some implementations may impose additional restrictions. For example:

  • Set calls may need to be in increasing key order.
  • a DB may be read-only or write-only.

type File

type File interface {
	io.Closer
	io.Reader
	io.ReaderAt
	io.Writer
	Stat() (os.FileInfo, error)
	Sync() error
}

File is a readable, writable sequence of bytes.

Typically, it will be an *os.File, but test code may choose to substitute memory-backed implementations.

type FileSystem

type FileSystem interface {
	// Create creates the named file for writing, truncating it if it already
	// exists.
	Create(name string) (File, error)

	// Open opens the named file for reading.
	Open(name string) (File, error)

	// Remove removes the named file or directory.
	Remove(name string) error

	// Rename renames a file. It overwrites the file at newname if one exists,
	// the same as os.Rename.
	Rename(oldname, newname string) error

	// MkdirAll creates a directory and all necessary parents. The permission
	// bits perm have the same semantics as in os.MkdirAll. If the directory
	// already exists, MkdirAll does nothing and returns nil.
	MkdirAll(dir string, perm os.FileMode) error

	// Lock locks the given file, creating the file if necessary, and
	// truncating the file if it already exists. The lock is an exclusive lock
	// (a write lock), but locked files should neither be read from nor written
	// to. Such files should have zero size and only exist to co-ordinate
	// ownership across processes.
	//
	// A nil Closer is returned if an error occurred. Otherwise, close that
	// Closer to release the lock.
	//
	// On Linux and OSX, a lock has the same semantics as fcntl(2)'s advisory
	// locks. In particular, closing any other file descriptor for the same
	// file will release the lock prematurely.
	//
	// Attempting to lock a file that is already locked by the current process
	// has undefined behavior.
	//
	// Lock is not yet implemented on other operating systems, and calling it
	// will return an error.
	Lock(name string) (io.Closer, error)

	// List returns a listing of the given directory. The names returned are
	// relative to dir.
	List(dir string) ([]string, error)

	// Stat returns an os.FileInfo describing the named file.
	Stat(name string) (os.FileInfo, error)
}

FileSystem is a namespace for files.

The names are filepath names: they may be / separated or \ separated, depending on the underlying operating system.

var DefaultFileSystem FileSystem = defFS{}

DefaultFileSystem is a FileSystem implementation backed by the underlying operating system's file system.

type FilterPolicy

type FilterPolicy interface {
	// Name names the filter policy.
	Name() string

	// AppendFilter appends to dst an encoded filter that holds a set of []byte
	// keys.
	AppendFilter(dst []byte, keys [][]byte) []byte

	// MayContain returns whether the encoded filter may contain given key.
	// False positives are possible, where it returns true for keys not in the
	// original set.
	MayContain(filter, key []byte) bool
}

FilterPolicy is an algorithm for probabilistically encoding a set of keys. The canonical implementation is a Bloom filter.

Every FilterPolicy has a name. This names the algorithm itself, not any one particular instance. Aspects specific to a particular instance, such as the set of keys or any other parameters, will be encoded in the []byte filter returned by NewFilter.

The name may be written to files on disk, along with the filter data. To use these filters, the FilterPolicy name at the time of writing must equal the name at the time of reading. If they do not match, the filters will be ignored, which will not affect correctness but may affect performance.

type Iterator

type Iterator interface {
	// Next moves the iterator to the next key/value pair.
	// It returns whether the iterator is exhausted.
	Next() bool

	// Key returns the key of the current key/value pair, or nil if done.
	// The caller should not modify the contents of the returned slice, and
	// its contents may change on the next call to Next.
	Key() []byte

	// Value returns the value of the current key/value pair, or nil if done.
	// The caller should not modify the contents of the returned slice, and
	// its contents may change on the next call to Next.
	Value() []byte

	// Close closes the iterator and returns any accumulated error. Exhausting
	// all the key/value pairs in a table is not considered to be an error.
	// It is valid to call Close multiple times. Other methods should not be
	// called after the iterator has been closed.
	Close() error
}

Iterator iterates over a DB's key/value pairs in key order.

An iterator must be closed after use, but it is not necessary to read an iterator until exhaustion.

An iterator is not necessarily goroutine-safe, but it is safe to use multiple iterators concurrently, with each in a dedicated goroutine.

It is also safe to use an iterator concurrently with modifying its underlying DB, if that DB permits modification. However, the resultant key/value pairs are not guaranteed to be a consistent snapshot of that DB at a particular point in time.

func NewConcatenatingIterator

func NewConcatenatingIterator(iters ...Iterator) Iterator

NewConcatenatingIterator returns an iterator that concatenates its input. Walking the resultant iterator will walk each input iterator in turn, exhausting each input before moving on to the next.

The sequence of the combined inputs' keys are assumed to be in strictly increasing order: iters[i]'s last key is less than iters[i+1]'s first key.

None of the iters may be nil.

func NewMergingIterator

func NewMergingIterator(cmp Comparer, iters ...Iterator) Iterator

NewMergingIterator returns an iterator that merges its input. Walking the resultant iterator will return all key/value pairs of all input iterators in strictly increasing key order, as defined by cmp.

The input's key ranges may overlap, but there are assumed to be no duplicate keys: if iters[i] contains a key k then iters[j] will not contain that key k.

None of the iters may be nil.

type Options

type Options struct {
	// BlockRestartInterval is the number of keys between restart points
	// for delta encoding of keys.
	//
	// The default value is 16.
	BlockRestartInterval int

	// BlockSize is the minimum uncompressed size in bytes of each table block.
	//
	// The default value is 4096.
	BlockSize int

	// Comparer defines a total ordering over the space of []byte keys: a 'less
	// than' relationship. The same comparison algorithm must be used for reads
	// and writes over the lifetime of the DB.
	//
	// The default value uses the same ordering as bytes.Compare.
	Comparer Comparer

	// Compression defines the per-block compression to use.
	//
	// The default value (DefaultCompression) uses snappy compression.
	Compression Compression

	// ErrorIfDBExists is whether it is an error if the database already exists.
	//
	// The default value is false.
	ErrorIfDBExists bool

	// FileSystem maps file names to byte storage.
	//
	// The default value uses the underlying operating system's file system.
	FileSystem FileSystem

	// FilterPolicy defines a filter algorithm (such as a Bloom filter) that
	// can reduce disk reads for Get calls.
	//
	// One such implementation is bloom.FilterPolicy(10) from the leveldb/bloom
	// package.
	//
	// The default value means to use no filter.
	FilterPolicy FilterPolicy

	// MaxOpenFiles is a soft limit on the number of open files that can be
	// used by the DB.
	//
	// The default value is 1000.
	MaxOpenFiles int

	// WriteBufferSize is the amount of data to build up in memory (backed by
	// an unsorted log on disk) before converting to a sorted on-disk file.
	//
	// Larger values increase performance, especially during bulk loads. Up to
	// two write buffers may be held in memory at the same time, so you may
	// wish to adjust this parameter to control memory usage. Also, a larger
	// write buffer will result in a longer recovery time the next time the
	// database is opened.
	//
	// The default value is 4MiB.
	WriteBufferSize int

	// VerifyChecksums is whether to verify the per-block checksums in a DB.
	//
	// The default value is false.
	VerifyChecksums bool
}

Options holds the optional parameters for leveldb's DB implementations. These options apply to the DB at large; per-query options are defined by the ReadOptions and WriteOptions types.

Options are typically passed to a constructor function as a struct literal. The GetXxx methods are used inside the DB implementations; they return the default parameter value if the *Options receiver is nil or the field value is zero.

Read/Write options:

  • Comparer
  • FileSystem
  • FilterPolicy
  • MaxOpenFiles

Read options:

  • VerifyChecksums

Write options:

  • BlockRestartInterval
  • BlockSize
  • Compression
  • ErrorIfDBExists
  • WriteBufferSize

func (*Options) GetBlockRestartInterval

func (o *Options) GetBlockRestartInterval() int

func (*Options) GetBlockSize

func (o *Options) GetBlockSize() int

func (*Options) GetComparer

func (o *Options) GetComparer() Comparer

func (*Options) GetCompression

func (o *Options) GetCompression() Compression

func (*Options) GetErrorIfDBExists

func (o *Options) GetErrorIfDBExists() bool

func (*Options) GetFileSystem

func (o *Options) GetFileSystem() FileSystem

func (*Options) GetFilterPolicy

func (o *Options) GetFilterPolicy() FilterPolicy

func (*Options) GetMaxOpenFiles

func (o *Options) GetMaxOpenFiles() int

func (*Options) GetVerifyChecksums

func (o *Options) GetVerifyChecksums() bool

func (*Options) GetWriteBufferSize

func (o *Options) GetWriteBufferSize() int

type ReadOptions

type ReadOptions struct {
}

ReadOptions hold the optional per-query parameters for Get and Find operations.

Like Options, a nil *ReadOptions is valid and means to use the default values.

type WriteOptions

type WriteOptions struct {
	// Sync is whether to sync underlying writes from the OS buffer cache
	// through to actual disk, if applicable. Setting Sync can result in
	// slower writes.
	//
	// If false, and the machine crashes, then some recent writes may be lost.
	// Note that if it is just the process that crashes (and the machine does
	// not) then no writes will be lost.
	//
	// In other words, Sync being false has the same semantics as a write
	// system call. Sync being true means write followed by fsync.
	//
	// The default value is false.
	Sync bool
}

WriteOptions hold the optional per-query parameters for Set and Delete operations.

Like Options, a nil *WriteOptions is valid and means to use the default values.

func (*WriteOptions) GetSync

func (o *WriteOptions) GetSync() bool

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL