copydb

package module
v0.0.0-...-17277e5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 11, 2019 License: MIT Imports: 12 Imported by: 0

README

CopyDB

Build Status codecov Go Report Card GoDoc

CopyDB is in-memory database works over Redis pub/sub approach. All writes are synced with Redis and then replicated across all CopyDB instances.

CopyDB is an embedded database and it gives you an ability to organize local cache inside your application.

CopyDB provides eventual consistency for data that updates frequently.

How it works

CopyDB downloads all available items from Redis after start. Then the database subscribes on items updates.

Each item contains a version to guarantee a consistent update on a particular instance. An item will be fully reloaded from Redis if a new version is not greater than existed by 1.

The database supports simple operations: SET, UNSET and REMOVE. It's described by Item interface:

type Item interface {
	Set(name string, data []byte)
	Unset(name string)
	Remove()
}

Read queries should be performed using embedded Query* functions:

  • QueryByID
  • QueryAll
  • QueryPool
  • QueryStats

Queries execution blocks items updates. Take the note you must write optimized queries.

Monitoring

CopyDB exposes essential metrics to get health status.

type Stats struct {
	ItemsApplied           int
	ItemsFailed            int
	ItemsEvicted           int
	ItemsReplicated        int
	VersionConfictDetected int
	DBScanned              int
}

Example

Please, checkout an example provided with the repository to figure out how to run your first application based on CopyDB.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrVersionConflict = errors.New("version conflict")
	ErrVersionNotFound = errors.New("version not found")
	ErrItemNotFound    = errors.New("item not found")
	ErrZeroVersion     = errors.New("zero version")
)

Functions

This section is empty.

Types

type DB

type DB struct {
	// contains filtered or unexported fields
}

DB implements all subscriptions and updates.

func New

func New(r Redis, opts ...DBOpt) (*DB, error)

New creates a new DB.

func (*DB) QueriesIn

func (db *DB) QueriesIn() chan<- Query

QueriesIn returns a channel to accept queries.

func (*DB) Query

func (db *DB) Query(ctx context.Context, q Query) error

Query runs a query and waits for scan completion.

func (*DB) Serve

func (db *DB) Serve() error

Serve subscribes to the channel updates.

func (*DB) Stop

func (db *DB) Stop(ctx context.Context) error

Stop terminates serve.

type DBOpt

type DBOpt func(*DB)

DBOpt configures DB.

func WithBufferedQueries

func WithBufferedQueries(c int) DBOpt

WithBufferedQueries configures a capacity of queries queue.

func WithCapacity

func WithCapacity(c int) DBOpt

WithCapacity configures items capacity.

func WithChannelKey

func WithChannelKey(s string) DBOpt

WithChannelKey configures channel key.

func WithItemKeyPattern

func WithItemKeyPattern(s string) DBOpt

WithItemKeyPattern configures item key pattern.

func WithListKey

func WithListKey(s string) DBOpt

WithListKey configures list key.

func WithLogger

func WithLogger(logger *log.Logger) DBOpt

WithLogger configures a logger.

func WithPool

func WithPool(pool Pool) DBOpt

WithPool configures items pool.

func WithPubSubOpts

func WithPubSubOpts(opts PubSubOpts) DBOpt

WithPubSubOpts configures pubsub.

func WithTTL

func WithTTL(d time.Duration) DBOpt

WithTTL configures a cache TTL.

type Item

type Item interface {
	Set(name string, data []byte)
	Unset(name string)
	Remove()
}

Item contains functions for data manipulation.

type ItemField

type ItemField []byte

ItemField contains a data of a field.

func (ItemField) String

func (f ItemField) String() string

type Pool

type Pool interface {
	Get() Item
	Put(Item)
}

Pool maintains items lifecycle.

type PubSubOpts

type PubSubOpts struct {
	ReceiveTimeout time.Duration
	ChannelSize    int
}

PubSubOpts contains parameters for pubcub configuration.

type Query

type Query interface {
	// contains filtered or unexported methods
}

Query scans items to perform a request.

func QueryAll

func QueryAll(resolve QueryResolve, reject QueryReject) Query

QueryAll creates a query to scan all items.

func QueryByID

func QueryByID(id string, unix int64, resolve QueryResolve, reject QueryReject) Query

QueryByID creates a query to get an item by identifier. The func accepts unix timestamp if you want to query the item not older than it.

func QueryPool

func QueryPool(scan func(Pool)) Query

QueryPool creates a query to scan the pool directly.

func QueryStats

func QueryStats(scan func(Stats)) Query

QueryStats creates a query to scan db statistics.

type QueryReject

type QueryReject func(error)

QueryReject is a callback with an error.

type QueryResolve

type QueryResolve func(Item)

QueryResolve is a callback with found item.

type Redis

type Redis interface {
	Pipeline() redis.Pipeliner
	HGetAll(key string) *redis.StringStringMapCmd
	ZRangeByScoreWithScores(key string, opt redis.ZRangeBy) *redis.ZSliceCmd
	Subscribe(channels ...string) *redis.PubSub

	Eval(script string, keys []string, args ...interface{}) *redis.Cmd
	EvalSha(sha1 string, keys []string, args ...interface{}) *redis.Cmd
	ScriptExists(hashes ...string) *redis.BoolSliceCmd
	ScriptLoad(script string) *redis.StringCmd
}

type SimpleItem

type SimpleItem map[string]ItemField

SimpleItem implements Item.

func (SimpleItem) IsEmpty

func (i SimpleItem) IsEmpty() bool

IsEmpty checks that the item has no fields.

func (SimpleItem) Remove

func (i SimpleItem) Remove()

Remove resets all fields.

func (SimpleItem) Set

func (i SimpleItem) Set(name string, data []byte)

Set updates a field by name.

func (SimpleItem) Unset

func (i SimpleItem) Unset(name string)

Unset removes a filed by name.

type SimplePool

type SimplePool struct {
	New func() Item
	// contains filtered or unexported fields
}

SimplePool implements Pool.

func (*SimplePool) Get

func (p *SimplePool) Get() Item

Get returns a new Item.

func (*SimplePool) Put

func (p *SimplePool) Put(i Item)

Put saves an item in the pool.

type Statement

type Statement struct {
	// contains filtered or unexported fields
}

Statement wraps an update and execs it.

func NewStatement

func NewStatement(id string) Statement

NewStatement creates a new statement update.

func (*Statement) Exec

func (s *Statement) Exec(ctx context.Context, db *DB, currtime time.Time) error

Exec runs a statement.

func (*Statement) Remove

func (s *Statement) Remove()

Remove erases an item.

func (*Statement) Set

func (s *Statement) Set(name string, data []byte)

Set updates a field.

func (*Statement) SetString

func (s *Statement) SetString(name, data string)

SetString updates a field within string.

func (*Statement) Unset

func (s *Statement) Unset(name string)

Unset removes a field.

type Stats

type Stats struct {
	ItemsApplied           int
	ItemsFailed            int
	ItemsEvicted           int
	ItemsReplicated        int
	VersionConfictDetected int
	DBScanned              int
}

Stats contains basic metrics of the database.

Directories

Path Synopsis
examples
indexes
internal
model
Package model is a generated protocol buffer package.
Package model is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL