kelips

package module
v0.0.0-...-efce6da Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 20, 2018 License: MPL-2.0 Imports: 17 Imported by: 1

README

go-kelips

go-kelips is a go implementation of the kelips DHT. It provides a simple UDP transport. The gossip layer is not included, instead methods have been provided to update the DHT from downstream gossip events.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type AffinityGroup 

type AffinityGroup interface {
	Nodes() []kelipspb.Node
}

AffinityGroup is an interface used for group lookups. This is used to get nodes in a group

type AffinityGroupRPC 

type AffinityGroupRPC interface {
	// Lookup nodes returning the min amount
	LookupNodes(key []byte, min int) ([]*kelipspb.Node, error)

	// Return all nodes for key group
	LookupGroupNodes(key []byte) ([]*kelipspb.Node, error)

	// Lookup nodes from the local view
	Lookup(key []byte) ([]*kelipspb.Node, error)

	// Insert to local group
	Insert(key []byte, tuple TupleHost, propogate bool) error

	// Delete from local group
	Delete(key []byte, tuple TupleHost, propogate bool) error
}

AffinityGroupRPC implements an interface for local rpc's used by the transport

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client implements a kelips client

func NewClient 

func NewClient(peers ...string) (*Client, error)

NewClient inits a new client using exising peers

func (*Client) Delete 

func (c *Client) Delete(key []byte, tuple TupleHost) error

Delete sends a delete request to delete a key

func (*Client) Insert 

func (c *Client) Insert(key []byte, tuple TupleHost) error

Insert sends an insert request for key-tuple mapping to a peer

func (*Client) Lookup 

func (c *Client) Lookup(key []byte) ([]*kelipspb.Node, error)

Lookup returns nodes holding the key

func (*Client) LookupGroupNodes 

func (c *Client) LookupGroupNodes(key []byte) ([]*kelipspb.Node, error)

LookupGroupNodes requests group nodes for a key from a peer

func (*Client) LookupNodes 

func (c *Client) LookupNodes(key []byte, min int) ([]*kelipspb.Node, error)

LookupNodes request nodes for key returning atleast min number of nodes

type Config 

type Config struct {
	// AdvertiseHost used to compute the hash.  This may be different from the
	// transport address
	AdvertiseHost string

	// Number of affinity groups
	K int

	// Setting this to true will cause writes to be propogated to all nodes in
	// in the group.  The default is false relying on the underlying gossip
	// transport to provide propogation
	EnablePropogation bool

	// Hash function to use
	HashFunc func() hash.Hash

	// Tuple store. Defaults in an in-mem one if not specified
	TupleStore TupleStore

	Region string
	Sector string
	Zone   string

	// Meta is serialized to binary and made part of the node object
	Meta map[string]string
}

Config holds the kelips config to initialize the dht

func DefaultConfig 

func DefaultConfig(host string) *Config

DefaultConfig returns a minimum required config

type InmemTuples 

type InmemTuples struct {
	// contains filtered or unexported fields
}

InmemTuples implements an in-memory TupleStore

func NewInmemTuples 

func NewInmemTuples() *InmemTuples

NewInmemTuples instantiates an in-memory tuple store

func (*InmemTuples) Count 

func (ft *InmemTuples) Count() int

Count returns the total number of keys in the store

func (*InmemTuples) Delete 

func (ft *InmemTuples) Delete(key []byte) error

Delete deletes a key removing all associated TupleHosts

func (*InmemTuples) DeleteKeyHost 

func (ft *InmemTuples) DeleteKeyHost(key []byte, h TupleHost) bool

DeleteKeyHost deletes a host associated to the name returning true if it was deleted

func (*InmemTuples) ExpireHost 

func (ft *InmemTuples) ExpireHost(tuple TupleHost) bool

ExpireHost removes a host from all keys referring to it

func (*InmemTuples) Get 

func (ft *InmemTuples) Get(key []byte) ([]TupleHost, error)

Get returns a list of hosts for a key. It returns nil if the name is not found

func (*InmemTuples) Insert 

func (ft *InmemTuples) Insert(key []byte, h TupleHost) error

Insert adds a new host for a name if it does not already exist. It returns true if the host was added

func (*InmemTuples) Iter 

func (ft *InmemTuples) Iter(f func(key []byte, hosts []TupleHost) bool)

Iter iterates over all the tuples. If the callback returns false, iteration is terminated

type Kelips 

type Kelips struct {
	// contains filtered or unexported fields
}

Kelips is the core engine of the dht. It provides functions to operate against it. The gossip layer is not included as part of the implementation but rather methods have been provided such that they can be called from a gossip interface

func Create 

func Create(conf *Config, remote Transport) *Kelips

Create instantiates kelips and registers the local group to the transport. It inits an in-memory tuple store

func (*Kelips) AddNode 

func (kelips *Kelips) AddNode(node *kelipspb.Node, force bool) error

AddNode adds a node to the DHT. It calculates the node id and adds it to the group it belongs to

func (*Kelips) Delete 

func (kelips *Kelips) Delete(key []byte, tuple TupleHost) (err error)

Delete deletes a key and all assoicated tuples. If the key belongs to a foreign group the delete is forwarded to a node in that group and its response is returned

func (*Kelips) Insert 

func (kelips *Kelips) Insert(key []byte, tuple TupleHost) error

Insert inserts a key and associated tuple. If the key belongs to a foreign group, the insert is forwarded to a node in that group and its response is returned. If the TupleHost is not known it will not be returned in a lookup though will still be in the tuple store. Once the node is known/alive it will be returned in lookups

func (*Kelips) Join 

func (kelips *Kelips) Join(peers []string) error

Join adds the given peers to their respective groups

func (*Kelips) LocalNode 

func (kelips *Kelips) LocalNode() kelipspb.Node

LocalNode returns the local node by performing a lookup

func (*Kelips) Lookup 

func (kelips *Kelips) Lookup(key []byte) ([]*kelipspb.Node, error)

Lookup hashes the key and finds its affinity group. If the group is local it returns all local known nodes for the key otherwise it makes a Lookup call on the owning foreign group and returns its nodes

func (*Kelips) LookupGroupNodes 

func (kelips *Kelips) LookupGroupNodes(key []byte) ([]*kelipspb.Node, error)

LookupGroupNodes returns all nodes in a group for the key

func (*Kelips) LookupNodes 

func (kelips *Kelips) LookupNodes(key []byte, min int) ([]*kelipspb.Node, error)

LookupNodes returns a minimum of n nodes that a key maps to

func (*Kelips) PingNode 

func (kelips *Kelips) PingNode(hostname string, coord *vivaldi.Coordinate, rtt time.Duration) error

PingNode sets the coords and rtt on a node and updates the heartbeat count

func (*Kelips) RemoveNode 

func (kelips *Kelips) RemoveNode(hostname string) error

RemoveNode removes a node from the DHT. This will remove the node from the local nodes view as well as remove all references in the tuples

func (*Kelips) Seed 

func (kelips *Kelips) Seed(snapshot *kelipspb.Snapshot) error

Seed seeds the local groups with the given snapshot

func (*Kelips) Snapshot 

func (kelips *Kelips) Snapshot() *kelipspb.Snapshot

Snapshot returns a Snapshot of all tuples and nodes known to the node.

type Transport 

type Transport interface {
	LookupGroupNodes(host string, key []byte) ([]*kelipspb.Node, error)
	Lookup(host string, key []byte) ([]*kelipspb.Node, error)
	Insert(host string, key []byte, tuple TupleHost, propogate bool) error
	Delete(host string, key []byte, tuple TupleHost, propogate bool) error
	// Register a local affinity group
	Register(AffinityGroupRPC)
}

Transport implements RPC's needed by kelips

type TupleHost 

type TupleHost []byte

TupleHost is the host port of a key tuple

func NewTupleFromIPPort 

func NewTupleFromIPPort(ip net.IP, port int) TupleHost

NewTupleFromIPPort creates a new tuple with an ip an port

func NewTupleHost 

func NewTupleHost(hostport string) TupleHost

NewTupleHost creates a tuple from a host:port string

func NewTupleHostFromHostPort 

func NewTupleHostFromHostPort(host string, port int) TupleHost

NewTupleHostFromHostPort creates a tuple from a ip string and port

func (TupleHost) Copy 

func (host TupleHost) Copy() TupleHost

Copy returns a copy of the tuple host

func (TupleHost) ID 

func (host TupleHost) ID(h hash.Hash) []byte

ID returns the hash of the host port bytes

func (TupleHost) IPAddress 

func (host TupleHost) IPAddress() net.IP

IPAddress returns the v4 or v6 address

func (TupleHost) Port 

func (host TupleHost) Port() uint16

Port returns the port number

func (TupleHost) String 

func (host TupleHost) String() string

type TupleStore 

type TupleStore interface {
	// Iter iterates over all the tuples.  If the callback returns false, iteration
	// is terminated
	Iter(f func(key []byte, hosts []TupleHost) bool)

	// Count returns the total number of keys in the store
	Count() int

	// Insert adds a new host for a name if it does not already exist.  It returns true
	// if the host was added
	Insert(key []byte, h TupleHost) error

	// Delete deletes a key removing all associated TupleHosts
	Delete(key []byte) error

	// Get returns a list of hosts for a key.  It returns nil if the name is not
	// found
	Get(key []byte) ([]TupleHost, error)

	// DeleteKeyHost deletes a host associated to the name returning true if it was deleted
	DeleteKeyHost(key []byte, h TupleHost) bool

	// ExpireHost removes a host from all keys referring to it
	ExpireHost(tuple TupleHost) bool
}

TupleStore is used to store and interact with tuples

type UDPTransport 

type UDPTransport struct {
	// contains filtered or unexported fields
}

UDPTransport is a udp based transport for kelips. It is well suited due to the small message size and reliance on gossip. It only implements rpc's and not the fault-tolerance. This is primarily used for direct inserts, lookups and deletes

func NewUDPTransport 

func NewUDPTransport(ln *net.UDPConn) *UDPTransport

NewUDPTransport inits a new UDPTransport using the given server connection. Nil can be supplied if the transport is only used as a client and is not a cluster member

func (*UDPTransport) Delete 

func (trans *UDPTransport) Delete(host string, key []byte, tuple TupleHost, propogate bool) error

Delete a key on the the host removing all node mappings for the key

func (*UDPTransport) Insert 

func (trans *UDPTransport) Insert(host string, key []byte, tuple TupleHost, propogate bool) error

Insert inserts a key to node mapping on a remote host

func (*UDPTransport) Lookup 

func (trans *UDPTransport) Lookup(host string, key []byte) ([]*kelipspb.Node, error)

Lookup performs a lookup request on a host for a key

func (*UDPTransport) LookupGroupNodes 

func (trans *UDPTransport) LookupGroupNodes(host string, key []byte) ([]*kelipspb.Node, error)

LookupGroupNodes looksup the group nodes for a key on a remote host

func (*UDPTransport) LookupNodes 

func (trans *UDPTransport) LookupNodes(host string, key []byte, min int) ([]*kelipspb.Node, error)

LookupNodes performs a lookup request on a host returning at least min nodes

func (*UDPTransport) Register 

func (trans *UDPTransport) Register(group AffinityGroupRPC)

Register registers the local group to serve rpcs from and starts accepting connections

Source Files

View all Source files

Directories

Path Synopsis
Package kelipspb is a generated protocol buffer package.
 Package kelipspb is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.