kademlia

package module
v0.0.0-...-6cbe866 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 7, 2019 License: MIT Imports: 26 Imported by: 0

README

kademlia

GoDoc Go Report Card Build Status MIT licensed

This is a Go implementation of a vanilla Kademlia DHT. The implementation is based off of a combination of the original Kademlia whitepaper and the xlattice design specification. It does not attempt to conform to BEP-5, or any other BitTorrent-specific design.

This project has not been heavily battle-tested, and I would not recommend using it in any production environment at this time.

Implementation characteristics

  • uses uTP for all network communication
  • supports IPv4/IPv6

TODO

  • Implement STUN for public address discovery
  • Load testing/Benchmarks
  • More testing around message validation
  • More testing of bad/malicious message handling
  • Banning/throttling of malicious messages/nodes
  • Implement UDP hole punching for NAT traversal
  • Use loose parallelism for iterative lookups
  • Consider breaking store into two messages and transfer bulk of data over TCP
  • Implement republishing according to the xlattice design document
  • Better cleanup of unanswered expected messages
  • Logging support

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ContentAddressable

func ContentAddressable(data []byte) []byte

ContentAddressable returns the key of the provided data. uses sha1.

func FromNetworkNode

func FromNetworkNode(n NetworkNode) *protocol.Node

FromNetworkNode converts internal node structure to protocol node structure.

func FromNetworkNodes

func FromNetworkNodes(ns ...NetworkNode) (out []*protocol.Node)

FromNetworkNodes convience function for converting multiple NetworkNode instances.

func GatewayFingerprint

func GatewayFingerprint(ip net.IP, port int) []byte

GatewayFingerprint generate a fingerprint a IP/port combination.

func MustNewID

func MustNewID() []byte

MustNewID panics if it fails to generate a new random ID.

func NewID

func NewID() ([]byte, error)

NewID generates a new random ID

func UDPAddressFromNode

func UDPAddressFromNode(n NetworkNode) (net.Addr, error)

UDPAddressFromNode convert a NetworkNode into its corresponding udp address.

func WithUDPNodeDialer

func WithUDPNodeDialer(d dialer) grpc.DialOption

WithUDPNodeDialer creates a DialOption from a dialer

Types

type DHT

type DHT struct {

	// Seconds after which an otherwise unaccessed bucket must be refreshed,
	// also used to reap dead nodes from the DHT. if a node hasnt been seen in 4x
	// the refresh period then its dead.
	TRefresh time.Duration

	// The maximum time to wait for a response from a node before discarding
	// it from the bucket
	TPingMax time.Duration

	// The maximum time to wait to locate a key before timing out.
	TLocateTimeout time.Duration
	// contains filtered or unexported fields
}

DHT represents the state of the local node in the distributed hash table

func NewDHT

func NewDHT(s Socket, options ...Option) *DHT

NewDHT initializes a new DHT node. A store and options struct must be provided.

func (*DHT) Bind

func (dht *DHT) Bind(s *grpc.Server) error

Bind registers the kademlia protocol to the grpc.Server

func (*DHT) Bootstrap

func (dht *DHT) Bootstrap(nodes ...NetworkNode) (err error)

Bootstrap attempts to bootstrap the network using the BootstrapNodes provided to the Options struct. This will trigger an iterativeFindNode to the provided BootstrapNodes.

func (*DHT) Dial

func (dht *DHT) Dial(ctx context.Context, n NetworkNode) (*grpc.ClientConn, error)

Dial a peer in the DHT.

func (*DHT) Disconnect

func (dht *DHT) Disconnect() error

Disconnect will trigger a disconnect from the network. All underlying sockets will be closed.

func (*DHT) GetSelf

func (dht *DHT) GetSelf() NetworkNode

GetSelf returns the node information of the local node.

func (*DHT) GetSelfID

func (dht *DHT) GetSelfID() []byte

GetSelfID returns the identifier of the local node

func (*DHT) Locate

func (dht *DHT) Locate(key []byte) (_none []NetworkNode, err error)

Locate does an iterative search through the network based on key. used to locate the nodes to interact with for a given key.

func (*DHT) Nodes

func (dht *DHT) Nodes() []NetworkNode

Nodes returns the nodes themselves sotred in the routing table.

func (*DHT) NumNodes

func (dht *DHT) NumNodes() int

NumNodes returns the total number of nodes stored in the local routing table

type MemoryStore

type MemoryStore struct {
	// contains filtered or unexported fields
}

MemoryStore is a simple in-memory key/value store used for unit testing, and the CLI example

func NewMemoryStore

func NewMemoryStore(dht *DHT) *MemoryStore

NewMemoryStore create a properly initialized memory store.

func (*MemoryStore) Delete

func (ms *MemoryStore) Delete(key []byte)

Delete deletes a key/value pair from the MemoryStore

func (*MemoryStore) ExpireKeys

func (ms *MemoryStore) ExpireKeys()

ExpireKeys should expire all key/values due for expiration.

func (*MemoryStore) GetAllKeysForReplication

func (ms *MemoryStore) GetAllKeysForReplication() [][]byte

GetAllKeysForReplication should return the keys of all data to be replicated across the network. Typically all data should be replicated every tReplicate seconds.

func (*MemoryStore) Retrieve

func (ms *MemoryStore) Retrieve(key []byte) (data []byte, found bool)

Retrieve will return the local key/value if it exists

func (*MemoryStore) Store

func (ms *MemoryStore) Store(key []byte, data []byte, replication time.Time, expiration time.Time, publisher bool) error

Store will store a key/value pair for the local node with the given replication and expiration times.

type Message

type Message struct {
	Sender     *NetworkNode
	Receiver   *NetworkNode
	ID         int64
	Type       int
	IsResponse bool
	Data       interface{}
}

Message sent and received between nodes.

type MessageOption

type MessageOption func(*Message)

MessageOption ...

func MessageOptionSender

func MessageOptionSender(n *NetworkNode) MessageOption

MessageOptionSender ...

type NetworkNode

type NetworkNode struct {
	// ID is a 20 byte unique identifier
	ID []byte

	// IP is the public address of the node
	IP net.IP

	// Port is the public port of the node
	Port int

	// LastSeen when was this node last considered seen by the DHT
	LastSeen time.Time
}

NetworkNode is the over-the-wire representation of a node

type Option

type Option func(*DHT)

Option for a distributed hash table.

func OptionNodeID

func OptionNodeID(id []byte) Option

OptionNodeID set node id, overriding the fingerprint. only should be used in tests.

func OptionNodeIDChecksum

func OptionNodeIDChecksum(c nodeChecksum) Option

OptionNodeIDChecksum set the function that forces node IDs to conform to a checksum. prevents nodes from having multiple IDs for a single address/port.

func OptionRefresh

func OptionRefresh(d time.Duration) Option

OptionRefresh see DHT.TRefresh

func OptionTLSConfig

func OptionTLSConfig(c *tls.Config) Option

OptionTLSConfig ...

func OptionTimeout

func OptionTimeout(d time.Duration) Option

OptionTimeout - timeout to wait beforing timing out when locating a key.

type Socket

type Socket struct {
	Gateway net.IP
	Port    int
	// contains filtered or unexported fields
}

Socket network connection with public IP information.

func NewSocket

func NewSocket(addr string, options ...SocketOption) (s Socket, err error)

NewSocket public ip of the socket.

func (Socket) Accept

func (t Socket) Accept() (net.Conn, error)

Accept waits for and returns the next connection to the listener.

func (Socket) Addr

func (t Socket) Addr() net.Addr

Addr returns the listener's network address.

func (Socket) Close

func (t Socket) Close() error

Close closes the listener. Any blocked Accept operations will be unblocked and return errors.

func (Socket) Dial

func (t Socket) Dial(ctx context.Context, addr net.Addr) (conn net.Conn, err error)

Dial the given net.Addr

func (Socket) LocalAddr

func (t Socket) LocalAddr() net.Addr

LocalAddr returns the listener's network address

func (Socket) LocalNode

func (t Socket) LocalNode() NetworkNode

LocalNode ...

func (Socket) Merge

func (t Socket) Merge(options ...SocketOption) Socket

Merge options into the socket.

func (Socket) NewNode

func (t Socket) NewNode() NetworkNode

NewNode create a node from the current socket and the given id.

type SocketOption

type SocketOption func(*Socket)

SocketOption option for the utp socket.

func SocketOptionGateway

func SocketOptionGateway(gateway net.IP) SocketOption

SocketOptionGateway public IP for the socket.

func SocketOptionPort

func SocketOptionPort(port int) SocketOption

SocketOptionPort public for for the socket.

Directories

Path Synopsis
Package protocol is a generated protocol buffer package.
Package protocol is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL