Documentation ¶
Index ¶
- Variables
- func DiskUsage(ctx context.Context, d Datastore) (uint64, error)
- func GetBackedHas(ctx context.Context, ds Read, key key.Key) (bool, error)
- func GetBackedSize(ctx context.Context, ds Read, key key.Key) (int, error)
- type Batch
- type Batching
- type CheckedDatastore
- type Datastore
- type GCDatastore
- type LogBatch
- type LogDatastore
- func (d *LogDatastore) Batch(ctx context.Context) (Batch, error)
- func (d *LogDatastore) Check(ctx context.Context) error
- func (d *LogDatastore) Children() []Datastore
- func (d *LogDatastore) Close() error
- func (d *LogDatastore) CollectGarbage(ctx context.Context) error
- func (d *LogDatastore) Delete(ctx context.Context, key key.Key) (err error)
- func (d *LogDatastore) DiskUsage(ctx context.Context) (uint64, error)
- func (d *LogDatastore) Get(ctx context.Context, key key.Key) (value []byte, err error)
- func (d *LogDatastore) GetSize(ctx context.Context, key key.Key) (size int, err error)
- func (d *LogDatastore) Has(ctx context.Context, key key.Key) (exists bool, err error)
- func (d *LogDatastore) Put(ctx context.Context, key key.Key, value []byte) (err error)
- func (d *LogDatastore) Query(ctx context.Context, q dsq.Query) (dsq.Results, error)
- func (d *LogDatastore) Scrub(ctx context.Context) error
- func (d *LogDatastore) Sync(ctx context.Context, prefix key.Key) error
- type MapDatastore
- func (d *MapDatastore) Batch(ctx context.Context) (Batch, error)
- func (d *MapDatastore) Close() error
- func (d *MapDatastore) Delete(ctx context.Context, key key.Key) (err error)
- func (d *MapDatastore) Get(ctx context.Context, key key.Key) (value []byte, err error)
- func (d *MapDatastore) GetSize(ctx context.Context, key key.Key) (size int, err error)
- func (d *MapDatastore) Has(ctx context.Context, key key.Key) (exists bool, err error)
- func (d *MapDatastore) Put(ctx context.Context, key key.Key, value []byte) (err error)
- func (d *MapDatastore) Query(ctx context.Context, q dsq.Query) (dsq.Results, error)
- func (d *MapDatastore) Sync(ctx context.Context, prefix key.Key) error
- type NullDatastore
- func (d *NullDatastore) Batch() (Batch, error)
- func (d *NullDatastore) Close() error
- func (d *NullDatastore) Delete(ctx context.Context, key key.Key) (err error)
- func (d *NullDatastore) Get(ctx context.Context, key key.Key) (value []byte, err error)
- func (d *NullDatastore) GetSize(ctx context.Context, key key.Key) (size int, err error)
- func (d *NullDatastore) Has(ctx context.Context, key key.Key) (exists bool, err error)
- func (d *NullDatastore) Put(ctx context.Context, key key.Key, value []byte) (err error)
- func (d *NullDatastore) Query(ctx context.Context, q dsq.Query) (dsq.Results, error)
- func (d *NullDatastore) Sync(ctx context.Context, prefix key.Key) error
- type PersistentDatastore
- type Read
- type ScrubbedDatastore
- type Shim
- type TTL
- type TTLDatastore
- type Txn
- type TxnDatastore
- type Write
Constants ¶
This section is empty.
Variables ¶
var ErrBatchUnsupported = errors.New("this datastore does not support batching")
ErrBatchUnsupported is returned if the by Batch if the Datastore doesn't actually support batching.
var ErrNotFound error = &dsError{error: errors.New("datastore: key not found"), isNotFound: true}
ErrNotFound is returned by Get and GetSize when a datastore does not map the given key to a value.
Functions ¶
func DiskUsage ¶
DiskUsage checks if a Datastore is a PersistentDatastore and returns its DiskUsage(), otherwise returns 0.
func GetBackedHas ¶
GetBackedHas provides a default Datastore.Has implementation. It exists so Datastore.Has implementations can use it, like so:
func (*d SomeDatastore) Has(key Key) (exists bool, err error) { return GetBackedHas(d, key) }
func GetBackedSize ¶
GetBackedSize provides a default Datastore.GetSize implementation. It exists so Datastore.GetSize implementations can use it, like so:
func (*d SomeDatastore) GetSize(key Key) (size int, err error) { return GetBackedSize(d, key) }
Types ¶
type Batch ¶
func NewBasicBatch ¶
type Batching ¶
Batching datastores support deferred, grouped updates to the database. `Batch`es do NOT have transactional semantics: updates to the underlying datastore are not guaranteed to occur in the same iota of time. Similarly, batched updates will not be flushed to the underlying datastore until `Commit` has been called. `Txn`s from a `TxnDatastore` have all the capabilities of a `Batch`, but the reverse is NOT true.
type CheckedDatastore ¶
CheckedDatastore is an interface that should be implemented by datastores which may need checking on-disk data integrity.
type Datastore ¶
type Datastore interface { Read Write // Sync guarantees that any Put or Delete calls under prefix that returned // before Sync(prefix) was called will be observed after Sync(prefix) // returns, even if the program crashes. If Put/Delete operations already // satisfy these requirements then Sync may be a no-op. // // If the prefix fails to Sync this method returns an error. Sync(ctx context.Context, prefix key.Key) error io.Closer }
Datastore represents storage for any key-value pair.
Datastores are general enough to be backed by all kinds of different storage: in-memory caches, databases, a remote datastore, flat files on disk, etc.
The general idea is to wrap a more complicated storage facility in a simple, uniform interface, keeping the freedom of using the right tools for the job. In particular, a Datastore can aggregate other datastores in interesting ways, like sharded (to distribute load) or tiered access (caches before databases).
While Datastores should be written general enough to accept all sorts of values, some implementations will undoubtedly have to be specific (e.g. SQL databases where fields should be decomposed into columns), particularly to support queries efficiently. Moreover, certain datastores may enforce certain types of values (e.g. requiring an io.Reader, a specific struct, etc) or serialization formats (JSON, Protobufs, etc).
IMPORTANT: No Datastore should ever Panic! This is a cross-module interface, and thus it should behave predictably and handle exceptional conditions with proper error reporting. Thus, all Datastore calls may return errors, which should be checked by callers.
type GCDatastore ¶
GCDatastore is an interface that should be implemented by datastores which don't free disk space by just removing data from them.
type LogBatch ¶
type LogBatch struct { Name string // contains filtered or unexported fields }
LogBatch logs all accesses through the batch.
type LogDatastore ¶
type LogDatastore struct { Name string // contains filtered or unexported fields }
LogDatastore logs all accesses through the datastore.
func NewLogDatastore ¶
func NewLogDatastore(ds Datastore, name string) *LogDatastore
NewLogDatastore constructs a log datastore.
func (*LogDatastore) Children ¶
func (d *LogDatastore) Children() []Datastore
Children implements Shim
func (*LogDatastore) Close ¶
func (d *LogDatastore) Close() error
func (*LogDatastore) CollectGarbage ¶
func (d *LogDatastore) CollectGarbage(ctx context.Context) error
func (*LogDatastore) DiskUsage ¶
func (d *LogDatastore) DiskUsage(ctx context.Context) (uint64, error)
DiskUsage implements the PersistentDatastore interface.
type MapDatastore ¶
type MapDatastore struct {
// contains filtered or unexported fields
}
MapDatastore uses a standard Go map for internal storage.
func NewMapDatastore ¶
func NewMapDatastore(ktype key.KeyType) (d *MapDatastore, err error)
NewMapDatastore constructs a MapDatastore. It is _not_ thread-safe by default, wrap using sync.MutexWrap if you need thread safety (the answer here is usually yes).
func (*MapDatastore) Close ¶
func (d *MapDatastore) Close() error
type NullDatastore ¶
type NullDatastore struct { }
NullDatastore stores nothing, but conforms to the API. Useful to test with.
func NewNullDatastore ¶
func NewNullDatastore() *NullDatastore
NewNullDatastore constructs a null datastoe
func (*NullDatastore) Batch ¶
func (d *NullDatastore) Batch() (Batch, error)
func (*NullDatastore) Close ¶
func (d *NullDatastore) Close() error
type PersistentDatastore ¶
type PersistentDatastore interface { Datastore // DiskUsage returns the space used by a datastore, in bytes. DiskUsage(ctx context.Context) (uint64, error) }
PersistentDatastore is an interface that should be implemented by datastores which can report disk usage.
type Read ¶
type Read interface { // Get retrieves the object `value` named by `key`. // Get will return ErrNotFound if the key is not mapped to a value. Get(ctx context.Context, key key.Key) (value []byte, err error) // Has returns whether the `key` is mapped to a `value`. // In some contexts, it may be much cheaper only to check for existence of // a value, rather than retrieving the value itself. (e.g. HTTP HEAD). // The default implementation is found in `GetBackedHas`. Has(ctx context.Context, key key.Key) (exists bool, err error) // GetSize returns the size of the `value` named by `key`. // In some contexts, it may be much cheaper to only get the size of the // value rather than retrieving the value itself. GetSize(ctx context.Context, key key.Key) (size int, err error) // Query searches the datastore and returns a query result. This function // may return before the query actually runs. To wait for the query: // // result, _ := ds.Query(q) // // // use the channel interface; result may come in at different times // for entry := range result.Next() { ... } // // // or wait for the query to be completely done // entries, _ := result.Rest() // for entry := range entries { ... } // Query(ctx context.Context, q query.Query) (query.Results, error) }
Read is the read-side of the Datastore interface.
type ScrubbedDatastore ¶
ScrubbedDatastore is an interface that should be implemented by datastores which want to provide a mechanism to check data integrity and/or error correction.
type TTL ¶
type TTL interface { PutWithTTL(ctx context.Context, key key.Key, value []byte, ttl time.Duration) error SetTTL(ctx context.Context, key key.Key, ttl time.Duration) error GetExpiration(ctx context.Context, key key.Key) (time.Time, error) }
TTL encapulates the methods that deal with entries with time-to-live.
type TTLDatastore ¶
TTLDatastore is an interface that should be implemented by datastores that support expiring entries.
type Txn ¶
type Txn interface { Read Write // Commit finalizes a transaction, attempting to commit it to the Datastore. // May return an error if the transaction has gone stale. The presence of an // error is an indication that the data was not committed to the Datastore. Commit(ctx context.Context) error // Discard throws away changes recorded in a transaction without committing // them to the underlying Datastore. Any calls made to Discard after Commit // has been successfully called will have no effect on the transaction and // state of the Datastore, making it safe to defer. Discard(ctx context.Context) }
Txn extends the Datastore type. Txns allow users to batch queries and mutations to the Datastore into atomic groups, or transactions. Actions performed on a transaction will not take hold until a successful call to Commit has been made. Likewise, transactions can be aborted by calling Discard before a successful Commit has been made.
type TxnDatastore ¶
type TxnDatastore interface { Datastore NewTransaction(ctx context.Context, readOnly bool) (Txn, error) }
TxnDatastore is an interface that should be implemented by datastores that support transactions.
type Write ¶
type Write interface { // Put stores the object `value` named by `key`. // // The generalized Datastore interface does not impose a value type, // allowing various datastore middleware implementations (which do not // handle the values directly) to be composed together. // // Ultimately, the lowest-level datastore will need to do some value checking // or risk getting incorrect values. It may also be useful to expose a more // type-safe interface to your application, and do the checking up-front. Put(ctx context.Context, key key.Key, value []byte) error // Delete removes the value for given `key`. If the key is not in the // datastore, this method returns no error. Delete(ctx context.Context, key key.Key) error }
Write is the write-side of the Datastore interface.
Directories ¶
Path | Synopsis |
---|---|
Package autobatch provides a go-datastore implementation that automatically batches together writes by holding puts in memory until a certain threshold is met.
|
Package autobatch provides a go-datastore implementation that automatically batches together writes by holding puts in memory until a certain threshold is met. |
Package delayed wraps a datastore allowing to artificially delay all operations.
|
Package delayed wraps a datastore allowing to artificially delay all operations. |
Package fs is a simple Datastore implementation that stores keys as directories and files, mirroring the key.
|
Package fs is a simple Datastore implementation that stores keys as directories and files, mirroring the key. |
Package failstore implements a datastore which can produce custom failures on operations by calling a user-provided error function.
|
Package failstore implements a datastore which can produce custom failures on operations by calling a user-provided error function. |
fuzz
module
|
|
Package key provides the Key interface and the KeySlice type, along with some utility functions around key.
|
Package key provides the Key interface and the KeySlice type, along with some utility functions around key. |
Package keytransform introduces a Datastore Shim that transforms keys before passing them to its child.
|
Package keytransform introduces a Datastore Shim that transforms keys before passing them to its child. |
Package mount provides a Datastore that has other Datastores mounted at various key prefixes and is threadsafe
|
Package mount provides a Datastore that has other Datastores mounted at various key prefixes and is threadsafe |
Package namespace introduces a namespace Datastore Shim, which basically mounts the entire child datastore under a prefix.
|
Package namespace introduces a namespace Datastore Shim, which basically mounts the entire child datastore under a prefix. |
Package retrystore provides a datastore wrapper which allows to retry operations.
|
Package retrystore provides a datastore wrapper which allows to retry operations. |