sharding

package module
v0.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 2, 2022 License: MIT Imports: 9 Imported by: 3

README

sharding test Coverage Status CodeQL report godoc

⚡ Super light and extremely flexible sharding library written in pure go.

Features

  • 🆓 Dependency free - uses only standard go packages but can be extended with customs if required.
  • 🌐 Generic - unlike most libraries, not limited to sql.DB or any other interface, you define the client.
  • 🤵 Elegant - modern looking API, generic, lambdas and other cool stuff.
  • 💯 Tested - 100% test coverage (and planning to keep it this way).

Examples

Pitfalls

  • ❌ Key types are currently limited to string, int64, uint64 or []byte.
  • ❌ There's no re-sharding support. It is too specific for each use case, thus pretty hard to implement.

License

The sharding package is open-sourced software licensed under the MIT license.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Cluster

type Cluster[KeyType ID, ConnType any] interface {

	// All returns all shards.
	All() []Shard[ConnType]

	// One returns Shard by key.
	One(key KeyType) Shard[ConnType]

	// Each runs fn on each shard within cluster.
	Each(fn func(s Shard[ConnType]) error) error

	// Map takes a list of identifiers and returns a map[] where the key is the corresponding
	// shard and the value is a slice of ids that belong to shard.
	Map(ids []KeyType) map[Shard[ConnType]][]KeyType

	// ByKeys executes fn on each result of Map func.
	ByKeys(ids []KeyType, fn func([]KeyType, Shard[ConnType]) error) error
}

Cluster interface.

func Connect

func Connect[KeyType ID, ConnType any](cfg Config[KeyType, ConnType]) (Cluster[KeyType, ConnType], error)

Connect to database using configs.

type Config

type Config[KeyType ID, ConnType any] struct {
	Connect  ConnectFunc[ConnType]       // required. connection func
	Shards   []ShardConfig               // required. shards config.
	Context  context.Context             // optional. defaults to context.Background()
	Strategy Strategy[KeyType, ConnType] // optional. defaults to defaultStrategy.
}

Config struct.

type ConnectFunc

type ConnectFunc[ConnType any] func(ctx context.Context, addr string) (ConnType, error)

ConnectFunc wraps connection func.

type Hash

type Hash[KeyType ID] interface {
	Sum(key KeyType) uint64
}

Hash interface.

func NewDefaultHash

func NewDefaultHash[KeyType ID]() Hash[KeyType]

type ID

type ID interface {
	string | []byte | int64 | uint64
}

ID type definition.

type Shard

type Shard[ConnType any] interface {
	ID() int64
	Conn() ConnType
}

Shard interface.

type ShardConfig

type ShardConfig struct {
	ID   int64  `json:"id"`
	Addr string `json:"dsn"`
}

ShardConfig type include constant connection id and dsn.

func ShardsConfigFromEnv

func ShardsConfigFromEnv(prefix ...string) []ShardConfig

ShardsConfigFromEnv loads parses environment variables and searches for variables called [prefix_]SHARD_ADDRESS_n, where prefix is optional and n is an increment number.

type Strategy

type Strategy[KeyType ID, ConnType any] interface {

	// Find shard by key.
	Find(key KeyType, shards []Shard[ConnType]) Shard[ConnType]
}

Strategy interface.

func NewDefaultStrategy

func NewDefaultStrategy[KeyType ID, ConnType any](
	hash Hash[KeyType],
) Strategy[KeyType, ConnType]

Directories

Path Synopsis
examples
memcache Module
sql Module

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL