shardis

package module
v0.0.0-...-8cb1862 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 8, 2017 License: MIT Imports: 14 Imported by: 0

README

Shardis

A simple Redis sharding client. Given a key, if the key contains the {} characters, instead of hashing the whole string to obtain the instance ID, hash the string inside {} to map it to the correct instance where the value is stored. For example, key "foo" will be hashed as SHA1("foo"), the key "bar{zap}" will be hashed just as SHA1("zap"), inspired by http://oldblog.antirez.com/post/redis-presharding.html

Installation

$ go get github.com/wang502/shardis

Usage

Start local Redis server
$ git clone git@github.com:antirez/redis.git
$ cd redis
$ ./src/redis-server
Configuration

Add a config.json in your project folder

{
"servers": [
      {"name": "name1",
       "host": "127.0.0.1:6379",
       "port": "",
       "db": "",
       "block_timeout": 0
      },
      {"name": "name2",
       "host": "",
       "port": "",
       "db": "",
       "block_timeout": 0
      }],
"hash": "md5",
"replicas": 10
}
  • servers: an array of servers involved in sharding. Each server json contains server name (needs to be unique), server host, port number, Redis db name, threshold of timeout for Redis blocking commands
  • hash: hashing method you prefer to use for sharding. (sha1, md5 and crc32)

Configuration

configPath := flag.String("c", "config.json", "path to configuration file")
flag.Parse()
config, err := shardis.InitConfig(*configPath)

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	//HashMethods maps the name to implementation of a hashing method
	HashMethods = map[string]interface{}{
		"crc32": Crc,
		"md5":   MdFive,
		"sha1":  ShaOne,
	}
)
View Source
var (
	// RegexFormat is a regex string used to extract substring enclosed by "{" and "}"
	RegexFormat = "{.*}"
)

Functions

func Crc

func Crc(key string) string

Crc uses crc32 to hash given string

func MdFive

func MdFive(key string) string

MdFive uses md5 to hash given string

func ShaOne

func ShaOne(key string) string

ShaOne uses sha1 to hash given string

Types

type Config

type Config struct {
	Servers    []map[string]interface{} `json:"servers"`
	HashMethod string                   `json:"hash"`
	Replicas   int                      `json:"replicas"`
}

Config represents the configuration of Shardis

func InitConfig

func InitConfig(confPath string) (*Config, error)

InitConfig reads config.json and create corresponding Config object

type Connection

type Connection struct {
	Host         string
	Pool         *redis.Pool
	BlockTimeout int
}

Connection represents the connection to a single Redis instance

func NewConnection

func NewConnection(host string, password string, blockTimeout int) *Connection

NewConnection creats a new Connection instance

func (*Connection) Blpop

func (conn *Connection) Blpop(keys ...string) (interface{}, error)

Blpop for calling Redis SET command on a single instance

func (*Connection) Get

func (conn *Connection) Get(key string) (interface{}, error)

Get for calling Redis SET command on a single instance

func (*Connection) Lpop

func (conn *Connection) Lpop(key string) (interface{}, error)

Lpop for calling Redis SET command on a single instance

func (*Connection) Lpush

func (conn *Connection) Lpush(key string, value interface{}) error

Lpush for calling Redis SET command on a single instance

func (*Connection) Rpop

func (conn *Connection) Rpop(key string) (interface{}, error)

Rpop for calling Redis SET command on a single instance

func (*Connection) Rpush

func (conn *Connection) Rpush(key string, value interface{}) error

Rpush for calling Redis SET command on a single instance

func (*Connection) Set

func (conn *Connection) Set(key string, value interface{}) error

Set for calling Redis SET command on a single instance

type HashRing

type HashRing struct {
	HashMethod string
	Nodes      []string
	Replicas   int
	Ring       map[string]string
	SortedKeys []string
}

HashRing represents a ring of Redis instance involved in Shardis

func NewHashRing

func NewHashRing(nodes []string, hashMethod string, replicas int) (*HashRing, error)

NewHashRing initilizes a new HashRing object

func (*HashRing) AddNode

func (hashRing *HashRing) AddNode(node string) error

AddNode add a new node to the HashRing given the node name

func (*HashRing) GetNode

func (hashRing *HashRing) GetNode(key string) string

GetNode gets the

func (*HashRing) RemoveNode

func (hashRing *HashRing) RemoveNode(node string) error

RemoveNode removes the given node from HashRing

type Shardis

type Shardis struct {
	Nodes       []string
	Connections map[string]*Connection
	Ring        *HashRing
}

Shardis represents a sharis client

func New

func New(config *Config) (*Shardis, error)

New Create a Shardis instance

func (*Shardis) Blpop

func (shard *Shardis) Blpop(key string) (interface{}, error)

Blpop is for Redis BLPOP

func (*Shardis) Get

func (shard *Shardis) Get(key string) (interface{}, error)

Get is for Redis GET

func (*Shardis) GetServer

func (shard *Shardis) GetServer(key string) *Connection

GetServer gets connection of the server that given key is mapped to

func (*Shardis) GetServerName

func (shard *Shardis) GetServerName(key string) string

GetServerName gets name of the server that given key is hashed to

func (*Shardis) Lpop

func (shard *Shardis) Lpop(key string) (interface{}, error)

Lpop is for Redis LPOP

func (*Shardis) Lpush

func (shard *Shardis) Lpush(key string, value interface{}) error

Lpush is for Redis LPUSH

func (*Shardis) Rpop

func (shard *Shardis) Rpop(key string) (interface{}, error)

Rpop is for Redis LPOP

func (*Shardis) Rpush

func (shard *Shardis) Rpush(key string, value interface{}) error

Rpush is for Redis RPUSH

func (*Shardis) Set

func (shard *Shardis) Set(key string, value interface{}) error

Set is for Redis SET command

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL