consistent: github.com/buraksezer/consistent Index | Files | Directories

package consistent

import "github.com/buraksezer/consistent"

Package consistent provides a consistent hashing function with bounded loads. For more information about the underlying algorithm, please take a look at https://research.googleblog.com/2017/04/consistent-hashing-with-bounded-loads.html

Example Use:

	cfg := consistent.Config{
		PartitionCount:    71,
		ReplicationFactor: 20,
		Load:              1.25,
		Hasher:            hasher{},
	}

     // Create a new consistent object
     // You may call this with a list of members
     // instead of adding them one by one.
	c := consistent.New(members, cfg)

     // myMember struct just needs to implement a String method.
     // New/Add/Remove distributes partitions among membes using the algorithm
     // defined on Google Research Blog.
	c.Add(myMember)

	key := []byte("my-key")
     // LocateKey hashes the key and calculates partition ID with
     // this modulo operation: MOD(hash result, partition count)
     // The owner of the partition is already calculated by New/Add/Remove.
     // LocateKey just returns the member which's responsible for the key.
	member := c.LocateKey(key)

Index

Package Files

consistent.go

Variables

var (
    //ErrInsufficientMemberCount represents an error which means there are not enough members to complete the task.
    ErrInsufficientMemberCount = errors.New("insufficient member count")

    // ErrMemberNotFound represents an error which means requested member could not be found in consistent hash ring.
    ErrMemberNotFound = errors.New("member could not be found in ring")
)

type Config Uses

type Config struct {
    // Hasher is responsible for generating unsigned, 64 bit hash of provided byte slice.
    Hasher Hasher

    // Keys are distributed among partitions. Prime numbers are good to
    // distribute keys uniformly. Select a big PartitionCount if you have
    // too many keys.
    PartitionCount int

    // Members are replicated on consistent hash ring. This number means that a member
    // how many times replicated on the ring.
    ReplicationFactor int

    // Load is used to calculate average load. See the code, the paper and Google's blog post to learn about it.
    Load float64
}

Config represents a structure to control consistent package.

type Consistent Uses

type Consistent struct {
    // contains filtered or unexported fields
}

Consistent holds the information about the members of the consistent hash circle.

func New Uses

func New(members []Member, config Config) *Consistent

New creates and returns a new Consistent object.

func (*Consistent) Add Uses

func (c *Consistent) Add(member Member)

Add adds a new member to the consistent hash circle.

func (*Consistent) AverageLoad Uses

func (c *Consistent) AverageLoad() float64

AverageLoad exposes the current average load.

func (*Consistent) FindPartitionID Uses

func (c *Consistent) FindPartitionID(key []byte) int

FindPartitionID returns partition id for given key.

func (*Consistent) GetClosestN Uses

func (c *Consistent) GetClosestN(key []byte, count int) ([]Member, error)

GetClosestN returns the closest N member to a key in the hash ring. This may be useful to find members for replication.

func (*Consistent) GetClosestNForPartition Uses

func (c *Consistent) GetClosestNForPartition(partID, count int) ([]Member, error)

GetClosestNForPartition returns the closest N member for given partition. This may be useful to find members for replication.

func (*Consistent) GetMembers Uses

func (c *Consistent) GetMembers() []Member

GetMembers returns a thread-safe copy of members.

func (*Consistent) GetPartitionOwner Uses

func (c *Consistent) GetPartitionOwner(partID int) Member

GetPartitionOwner returns the owner of the given partition.

func (*Consistent) LoadDistribution Uses

func (c *Consistent) LoadDistribution() map[string]float64

LoadDistribution exposes load distribution of members.

func (*Consistent) LocateKey Uses

func (c *Consistent) LocateKey(key []byte) Member

LocateKey finds a home for given key

func (*Consistent) Remove Uses

func (c *Consistent) Remove(name string)

Remove removes a member from the consistent hash circle.

type Hasher Uses

type Hasher interface {
    Sum64([]byte) uint64
}

Hasher is responsible for generating unsigned, 64 bit hash of provided byte slice. Hasher should minimize collisions (generating same hash for different byte slice) and while performance is also important fast functions are preferable (i.e. you can use FarmHash family).

type Member Uses

type Member interface {
    String() string
}

Member interface represents a member in consistent hash ring.

Directories

PathSynopsis
_examples

Package consistent imports 6 packages (graph) and is imported by 1 packages. Updated 2018-04-21. Refresh now. Tools for package owners.