Documentation
¶
Overview ¶
Package xneo provides base and common functionality for package neo.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Dial ¶
func Dial(ctx context.Context, typ proto.NodeType, net xnet.Networker, addr string, reqID *proto.RequestIdentification) (_ *neonet.NodeLink, _ *proto.AcceptIdentification, err error)
Dial connects to another node in the cluster.
It handshakes, requests identification and checks peer type. If successful returned are:
- established link
- accept identification reply
Dial does not update NodeTab or its node entries in any way. For establishing links to peers present in NodeTab use PeerNode.Dial.
Types ¶
type Cell ¶
Cell describes one storage in a pid entry in partition table.
type ClusterState ¶
type ClusterState struct {
NodeTab *NodeTable // information about nodes in the cluster
PartTab *PartitionTable // information about data distribution in the cluster
Code proto.ClusterState // master idea about cluster state
}
ClusterState represent state of a cluster.
func (*ClusterState) IsOperational ¶
func (cs *ClusterState) IsOperational() bool
func (*ClusterState) Snapshot ¶
func (cs *ClusterState) Snapshot() *ClusterStateSnapshot
type ClusterStateSnapshot ¶
type ClusterStateSnapshot struct {
NodeTab proto.NotifyNodeInformation
PartTab proto.SendPartitionTable
Code proto.ClusterState
}
ClusterStateSnapshot is snapshot of ClusterState. XXX place
type Listener ¶
type Listener interface {
// from LinkListener:
Close() error
Addr() net.Addr
// Accept accepts incoming connection from a peer.
//
// On success the link was handshaked and peer sent us RequestIdentification
// packet which we did not yet answer.
//
// On success returned are:
// - original peer request that carried identification
// - requested identification packet
//
// After successful accept it is the caller responsibility to reply the request.
//
// NOTE established link is Request.Link().
Accept(ctx context.Context) (*neonet.Request, *proto.RequestIdentification, error)
}
Listener is LinkListener adapted to return NodeLink with requested identification on Accept.
func NewListener ¶
func NewListener(inner neonet.LinkListener) Listener
NewListener wraps inner LinkListener into Listener.
type Node ¶
type Node struct {
MyInfo proto.NodeInfo // type, laddr, nid, state, idtime
ClusterName string
Net xnet.Networker // network AP we are sending/receiving on
MasterAddrSlice []string // address of all known masters TODO -> masterRegistry
// XXX reconsider not using State and have just .NodeTab, .PartTab, .ClusterState
// StateMu sync.RWMutex // <- XXX unexport ? XXX not used -> move to MasteredNode ?
State ClusterState // nodeTab/partTab/stateCode XXX unexport?
}
Node provides base functionality underlying implementation of any NEO node.
XXX vvv was moved to to _MasteredNode Every node knows how to talk to master and receives master idea about:
- current node table (all nodes in the cluster), - current partition table (how data is split around storage nodes), - current cluster state.
type NodeTable ¶
type NodeTable struct {
// contains filtered or unexported fields
}
NodeTable represents known peer nodes in a cluster.
It is
NID -> *PeerNode ; = (.laddr, .state, ...) + .link
mapping listing known nodes and associating their node ID with information about a peer.
Master maintains such table and provides it to its peers to know each other:
- Primary Master view of cluster.
- M tracks changes to nodeTab as nodes appear (connected to M) and go (disconnected from M).
- M regularly broadcasts nodeTab content updates(?) to all nodes. This way all nodes stay informed about their peers in cluster.
Usage examples:
- C needs to connect/talk to a storage by nid (the nid itself is obtained from PartitionTable by oid).
- S pulls from other S.
NOTE once a node was added to NodeTable its entry is never deleted: if e.g. a connection to node is lost associated entry is marked as having DOWN (XXX or UNKNOWN ?) node state.
func (*NodeTable) All ¶
All returns all entries in the table as one slice.
func (*NodeTable) Get ¶
Get finds node by node ID.
func (*NodeTable) StorageList ¶
StorageList returns list of all storages in the node table.
type PartitionTable ¶
type PartitionTable struct {
PTid proto.PTid // ↑ for versioning
// contains filtered or unexported fields
}
PartitionTable represents object space partitioning in a cluster.
It is
oid -> []nid
mapping associating object ID with list of storage nodes on where data for this oid should be written-to/loaded-from. This mapping is organized as follows:
Oid space is divided (partitioned) into Np parts via
pid(oid) = oid % Np
rule. The `oid % Np` is known as partition identifier of oid.
There is also externally set "replication factor" R which describes the minimum amount of separate nodes a particular object data should be stored into at the same time.
Given Np, R and []Storage PartitionTable tries to organize
pid -> []Storage
mapping so that
- replication level set by R is met
- storages associated with adjacent pids are different
when such organization is reached, and enough storage nodes are in "running" state, the partition table is called operational and non-operational otherwise.
The PartitionTable can auto-adjust to storage nodes coming and going. Let's consider cluster change from Ns1 to Ns2 storage nodes: if we pick
Np = LCM(Ns1, Ns2)
XXX ^^^ and vvv R is not taken into account.
then it is possible to rebalance data with minimal amount of data movement
~ min(Ns1, Ns2)·size(partition)
Examples:
[S1, S2] + S3: LCM(2,3) = 6
S1 S1 S1
S2 ~ S2 S2
S1 → S3
S2 S2
S1 S1
S2 → S3
[S1, S2, S3] + S4: LCM(3,4) = 12
S1 S1 S1
S2 S2 S2
S3 S3 S3
S2 S2 → S4
S1 S1 S1
S3 ~ S3 S3
S1 → S4
S2 S2
S3 S3
S2 S2
S1 S1
S3 → S4
[S1, S2, S3] - S3: LCM(3,2) = 6
S1 S1 S1
S2 S2 S2
S3 ~ S3 → S1
S1 S1
S2 S2
S3 → S2
Np thus is always multiple of Ns and with further reorderings (if needed) could be reduced directly to Ns.
Usually Master maintains partition table, plans partition updates and tells storages to executed them, and broadcasts partition table updates to all nodes in the cluster.
PartitionTable zero value is valid empty partition table.
func MakePartTab ¶
func MakePartTab(np int, nreplica int, nodev []*PeerNode) *PartitionTable
MakePartTab creates new partition with uniformly distributed nodes. The partition table created will be of len=np
func PartTabFromMsg ¶
func PartTabFromMsg(msg *proto.SendPartitionTable) *PartitionTable
PartTabFromMsg creates partition table with state obtained from message.
func (*PartitionTable) Get ¶
func (pt *PartitionTable) Get(oid zodb.Oid) []Cell
Get returns cells oid is associated with.
func (*PartitionTable) OperationalWith ¶
func (pt *PartitionTable) OperationalWith(nt *NodeTable) bool
OperationalWith checks whether all object space is covered by at least some ready-to-serve nodes.
for all partitions it checks both: - whether there are up-to-date entries in the partition table, and - whether there are corresponding storage nodes that are up
information about nodes being up or down is obtained from supplied NodeTable
func (*PartitionTable) R ¶
func (pt *PartitionTable) R() int
R returns n(replica) for partition table.
func (*PartitionTable) String ¶
func (pt *PartitionTable) String() string
func (*PartitionTable) ToMsg ¶
func (pt *PartitionTable) ToMsg() *proto.SendPartitionTable
ToMsg creates message that represents partition table state.
type PeerNode ¶
type PeerNode struct {
proto.NodeInfo // (.type, .laddr, .nid, .state, .idtime) XXX also protect by mu?
// contains filtered or unexported fields
}
PeerNode represents a peer node in the cluster.
It provides:
- peer's node info (nid, laddr, state, ...), and
- link to the peer.
func (*PeerNode) Dial ¶
Dial establishes link to the peer node.
If the link was not yet established Dial dials the peer appropriately, handshakes, requests identification and checks that identification reply is as expected.
Several Dial calls may be done in parallel - in any case only 1 link-level dial will be made and others will share established link.
In case Dial returns an error - future Dial will attempt to reconnect with "don't reconnect too fast" throttling.
func (*PeerNode) Link ¶
Link returns current link to peer node.
If the link is not yet established - Link returns nil.
See also: Dial.
func (*PeerNode) ResetLink ¶
ResetLink closes link to peer and sets it to nil.
Source Files