core: go.gazette.dev/core/keyspace Index | Files

package keyspace

import "go.gazette.dev/core/keyspace"

Package keyspace implements an efficient mechanism to mirror a decoded Etcd key/value space into a local KeySpace, which may be kept updated via a long- lived Watch operation. Each key & value of a KeySpace is decoded with a user provided decoder, and validated. Clients of a KeySpace are thus ensured that only validated keys & values are captured, while the KeySpace maintains consistency of the key/value set despite the potential for validation errors.

KeySpace instances may be "observed", which allows additional states to be derived from and updated by the KeySpace while being protected by the KeySpace Mutex. For example, if a value `foo` is mutated by a function of KeySpace.Observers, then any reader which properly synchronizes over KeySpace.Mu is guaranteed to see values of `foo` which reflect the current KeySpace state. Formally, readers are assured atomicity of a combined update to the KeySpace and the derived value.

KeySpace scales efficiently to Watches over 100's of thousands of keys by amortizing updates with a short Nagle-like delay, while providing fast range and point queries powered by its packed, sorted ordering.

Index

Package Files

doc.go key_space.go key_values.go

type KeySpace Uses

type KeySpace struct {
    // Key prefix which roots this KeySpace.
    Root string
    // Header is the last Etcd operation header which updated this KeySpace.
    Header etcdserverpb.ResponseHeader
    // KeyValues is a complete and decoded mirror of the (prefixed) Etcd key/value space.
    KeyValues
    // Observers called upon each mutation of the KeySpace. Observer calls occur
    // in-order, after the KeySpace itself has been updated, and while a write-
    // lock over the KeySpace is still held (which Observers must not release).
    // Observers is intended to simplify atomicity of stateful representations
    // deriving themselves from the KeySpace: any mutations performed by Observers
    // will appear synchronously with changes to the KeySpace itself, from the
    // perspective of a client appropriately utilizing a read-lock.
    Observers []func()
    // WatchApplyDelay is the duration for which KeySpace should allow Etcd
    // WatchResponses to queue before applying all responses to the KeySpace.
    // This Nagle-like mechanism amortizes the cost of applying many
    // WatchResponses arriving in close succession. Default is 30ms.
    WatchApplyDelay time.Duration
    // Mu guards Header, KeyValues, and Observers. It must be locked before any are accessed.
    Mu  sync.RWMutex
    // contains filtered or unexported fields
}

A KeySpace is a local mirror of a decoded portion of the Etcd key/value space, which is kept in sync with Etcd via long-lived Watch operations. KeySpace must be read-locked before access, to guard against concurrent updates.

func NewKeySpace Uses

func NewKeySpace(prefix string, decoder KeyValueDecoder) *KeySpace

NewKeySpace returns a KeySpace with the configured key |prefix| and |decoder|. |prefix| must be a "Clean" path, as defined by path.Clean, or NewKeySpace panics. This check limits the space of possible prefixes somewhat, but guards against many common and unintentional errors (eg, mixed use of trailing slashes).

func (*KeySpace) Apply Uses

func (ks *KeySpace) Apply(responses ...clientv3.WatchResponse) error

Apply one or more Etcd WatchResponses to the KeySpace. Apply returns only unrecoverable Application errors; inconsistencies in the updates themselves are instead logged. Apply is exported principally in support of testing fixtures; most clients should instead use Watch. Clients must ensure concurrent calls to Apply are not made.

func (*KeySpace) Load Uses

func (ks *KeySpace) Load(ctx context.Context, client *clientv3.Client, rev int64) error

Load loads a snapshot of the prefixed KeySpace at revision |rev|, or if |rev| is zero, at the current revision.

func (*KeySpace) WaitForRevision Uses

func (ks *KeySpace) WaitForRevision(ctx context.Context, revision int64) error

WaitForRevision blocks until the KeySpace Revision is at least |revision|, or until the context is done. A read lock of the KeySpace Mutex must be held at invocation, and will be re-acquired before WaitForRevision returns.

func (*KeySpace) Watch Uses

func (ks *KeySpace) Watch(ctx context.Context, client clientv3.Watcher) error

Watch a loaded KeySpace and apply updates as they are received.

type KeyValue Uses

type KeyValue struct {
    Raw     mvccpb.KeyValue
    Decoded interface{}
}

KeyValue composes a "raw" Etcd KeyValue with its user-defined, decoded representation.

type KeyValueDecoder Uses

type KeyValueDecoder func(raw *mvccpb.KeyValue) (interface{}, error)

A KeyValueDecoder decodes raw KeyValue instances into a user-defined representation. KeyValueDecoder returns an error if the KeyValue cannot be decoded. KeySpace will log decoding errors and not incorporate them into the current KeyValues, but will also treat them as recoverable and in all cases seek to bring the KeyValues representation to consistency with Etcd. In practice, this means bad values written to Etcd which fail to decode may be later corrected with valid representations: KeySpace will ignore the bad update and then reflect the corrected one once available.

type KeyValues Uses

type KeyValues []KeyValue

KeyValues is a collection of KeyValue naturally ordered on keys.

func (KeyValues) Copy Uses

func (kv KeyValues) Copy() KeyValues

Copy returns a deep-copy of the KeyValues.

func (KeyValues) EqualKeyRevisions Uses

func (kv KeyValues) EqualKeyRevisions(other KeyValues) bool

EqualKeyRevisions returns true if keys and revisions of |other| matches this KeyValues.

func (KeyValues) Prefixed Uses

func (kv KeyValues) Prefixed(prefix string) KeyValues

Prefixed returns the sub-slice of KeyValues prefixed by |prefix|.

func (KeyValues) Range Uses

func (kv KeyValues) Range(from, to string) KeyValues

Range returns the sub-slice of KeyValues spanning range [from, to).

func (KeyValues) Search Uses

func (kv KeyValues) Search(key string) (ind int, found bool)

Search returns the index at which |key| is found to be present, or should be inserted to maintain ordering.

Package keyspace imports 14 packages (graph) and is imported by 20 packages. Updated 2020-05-01. Refresh now. Tools for package owners.