Documentation ¶
Overview ¶
memberlist is a library that manages cluster membership and member failure detection using a gossip based protocol.
The use cases for such a library are far-reaching: all distributed systems require membership, and memberlist is a re-usable solution to managing cluster membership and node failure detection.
memberlist is eventually consistent but converges quickly on average. The speed at which it converges can be heavily tuned via various knobs on the protocol. Node failures are detected and network partitions are partially tolerated by attempting to communicate to potentially dead nodes through multiple routes.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Broadcast ¶
type Broadcast interface { // Invalidates checks if enqueuing the current broadcast // invalidates a previous broadcast Invalidates(b Broadcast) bool // Returns a byte form of the message Message() []byte // Finished is invoked when the message will no longer // be broadcast, either due to invalidation or to the // transmit limit being reached Finished() }
Broadcast is something that can be broadcasted via gossip to the memberlist cluster.
type ChannelEventDelegate ¶
type ChannelEventDelegate struct {
Ch chan<- NodeEvent
}
ChannelEventDelegate is used to enable an application to receive events about joins and leaves over a channel instead of a direct function call.
Care must be taken that events are processed in a timely manner from the channel, since this delegate will block until an event can be sent.
func (*ChannelEventDelegate) NotifyJoin ¶
func (c *ChannelEventDelegate) NotifyJoin(n *Node)
func (*ChannelEventDelegate) NotifyLeave ¶
func (c *ChannelEventDelegate) NotifyLeave(n *Node)
type Config ¶
type Config struct { // The name of this node. This must be unique in the cluster. Name string // Configuration related to what address to bind to and ports to // listen on. The ports used must match every node in the cluster, // since they'll be used for connecting as well as listening. BindAddr string UDPPort int TCPPort int // TCPTimeout is the timeout for establishing a TCP connection with // a remote node for a full state sync. TCPTimeout time.Duration // IndirectChecks is the number of nodes that will be asked to perform // an indirect probe of a node in the case a direct probe fails. Memberlist // waits for an ack from any single indirect node, so increasing this // number will increase the likelihood that an indirect probe will succeed // at the expense of bandwidth. IndirectChecks int // RetransmitMult is the multiplier for the number of retransmissions // that are attempted for messages broadcasted over gossip. The actual // count of retransmissions is calculated using the formula: // // Retransmits = RetransmitMult * log(N+1) // // This allows the retransmits to scale properly with cluster size. The // higher the multiplier, the more likely a failed broadcast is to converge // at the expense of increased bandwidth. RetransmitMult int // SuspicionMult is the multiplier for determining the time an // inaccessible node is considered suspect before declaring it dead. // The actual timeout is calculated using the formula: // // SuspicionTimeout = SuspicionMult * log(N+1) * ProbeInterval // // This allows the timeout to scale properly with expected propagation // delay with a larger cluster size. The higher the multiplier, the longer // an inaccessible node is considered part of the cluster before declaring // it dead, giving that suspect node more time to refute if it is indeed // still alive. SuspicionMult int // PushPullInterval is the interval between complete state syncs. // Complete state syncs are done with a single node over TCP and are // quite expensive relative to standard gossiped messages. Setting this // to zero will disable state push/pull syncs completely. // // Setting this interval lower (more frequent) will increase convergence // speeds across larger clusters at the expense of increased bandwidth // usage. PushPullInterval time.Duration // ProbeInterval and ProbeTimeout are used to configure probing // behavior for memberlist. // // ProbeInterval is the interval between random node probes. Setting // this lower (more frequent) will cause the memberlist cluster to detect // failed nodes more quickly at the expense of increased bandwidth usage. // // ProbeTimeout is the timeout to wait for an ack from a probed node // before assuming it is unhealthy. This should be set to 99-percentile // of RTT (round-trip time) on your network. ProbeInterval time.Duration ProbeTimeout time.Duration // GossipInterval and GossipNodes are used to configure the gossip // behavior of memberlist. // // GossipInterval is the interval between sending messages that need // to be gossiped that haven't been able to piggyback on probing messages. // If this is set to zero, non-piggyback gossip is disabled. By lowering // this value (more frequent) gossip messages are propagated across // the cluster more quickly at the expense of increased bandwidth. // // GossipNodes is the number of random nodes to send gossip messages to // per GossipInterval. Increasing this number causes the gossip messages // to propagate across the cluster more quickly at the expense of // increased bandwidth. GossipInterval time.Duration GossipNodes int // Delegate and Events are delegates for receiving and providing // data to memberlist via callback mechanisms. For Delegate, see // the Delegate interface. For Events, see the EventDelegate interface. Delegate Delegate Events EventDelegate // LogOutput is the writer where logs should be sent. If this is not // set, logging will go to stderr by default. LogOutput io.Writer }
func DefaultConfig ¶
func DefaultConfig() *Config
DefaultConfig returns a sane set of configurations for Memberlist. It uses the hostname as the node name, and otherwise sets very conservative values that are sane for most LAN environments. The default configuration errs on the side on the side of caution, choosing values that are optimized for higher convergence at the cost of higher bandwidth usage. Regardless, these values are a good starting point when getting started with memberlist.
type Delegate ¶
type Delegate interface { // NodeMeta is used to retrieve meta-data about the current node // when broadcasting an alive message. It's length is limited to // the given byte size. This metadata is available in the Node structure. NodeMeta(limit int) []byte // NotifyMsg is called when a user-data message is received. // Care should be taken that this method does not block, since doing // so would block the entire UDP packet receive loop. Additionally, the byte // slice may be modified after the call returns, so it should be copied if needed. NotifyMsg([]byte) // GetBroadcasts is called when user data messages can be broadcast. // It can return a list of buffers to send. Each buffer should assume an // overhead as provided with a limit on the total byte size allowed. // The total byte size of the resulting data to send must not exceed // the limit. GetBroadcasts(overhead, limit int) [][]byte // LocalState is used for a TCP Push/Pull. This is sent to // the remote side in addition to the membership information. Any // data can be sent here. See MergeRemoteState as well. LocalState() []byte // MergeRemoteState is invoked after a TCP Push/Pull. This is the // state received from the remote side and is the result of the // remote side's LocalState call. MergeRemoteState([]byte) }
Delegate is the interface that clients must implement if they want to hook into the gossip layer of Memberlist. All the methods must be thread-safe, as they can and generally will be called concurrently.
type EventDelegate ¶
type EventDelegate interface { // NotifyJoin is invoked when a node is detected to have joined. // The Node argument must not be modified. NotifyJoin(*Node) // NotifyLeave is invoked when a node is detected to have left. // The Node argument must not be modified. NotifyLeave(*Node) }
EventDelegate is a simpler delegate that is used only to receive notifications about members joining and leaving. The methods in this delegate may be called by multiple goroutines, but never concurrently. This allows you to reason about ordering.
type Memberlist ¶
type Memberlist struct {
// contains filtered or unexported fields
}
func Create ¶
func Create(conf *Config) (*Memberlist, error)
Create will create a new Memberlist using the given configuration. This will not connect to any other node (see Join) yet, but will start all the listeners to allow other nodes to join this memberlist. After creating a Memberlist, the configuration given should not be modified by the user anymore.
func (*Memberlist) Join ¶
func (m *Memberlist) Join(existing []string) (int, error)
Join is used to take an existing Memberlist and attempt to join a cluster by contacting all the given hosts and performing a state sync. Initially, the Memberlist only contains our own state, so doing this will cause remote nodes to become aware of the existence of this node, effectively joining the cluster.
This returns the number of hosts successfully contacted and an error if none could be reached. If an error is returned, the node did not successfully join the cluster.
func (*Memberlist) Leave ¶
func (m *Memberlist) Leave(timeout time.Duration) error
Leave will broadcast a leave message but will not shutdown the background listeners, meaning the node will continue participating in gossip and state updates.
This will block until the leave message is successfully broadcasted to a member of the cluster, if any exist or until a specified timeout is reached.
This method is safe to call multiple times, but must not be called after the cluster is already shut down.
func (*Memberlist) Members ¶
func (m *Memberlist) Members() []*Node
Members returns a list of all known live nodes. The node structures returned must not be modified. If you wish to modify a Node, make a copy first.
func (*Memberlist) NumMembers ¶
func (m *Memberlist) NumMembers() (alive int)
NumMembers returns the number of alive nodes currently known. Between the time of calling this and calling Members, the number of alive nodes may have changed, so this shouldn't be used to determine how many members will be returned by Members.
func (*Memberlist) Shutdown ¶
func (m *Memberlist) Shutdown() error
Shutdown will stop any background maintanence of network activity for this memberlist, causing it to appear "dead". A leave message will not be broadcasted prior, so the cluster being left will have to detect this node's shutdown using probing. If you wish to more gracefully exit the cluster, call Leave prior to shutting down.
This method is safe to call multiple times.
type Node ¶
type Node struct { Name string Addr net.IP Meta []byte // Metadata from the delegate for this node. }
Node represents a node in the cluster.
type NodeEvent ¶
type NodeEvent struct { Event NodeEventType Node *Node }
NodeEvent is a single event related to node activity in the memberlist. The Node member of this struct must not be directly modified. It is passed as a pointer to avoid unnecessary copies. If you wish to modify the node, make a copy first.
type NodeEventType ¶
type NodeEventType int
NodeEventType are the types of events that can be sent from the ChannelEventDelegate.
const ( NodeJoin NodeEventType = iota NodeLeave )
type TransmitLimitedQueue ¶
type TransmitLimitedQueue struct { // NumNodes returns the number of nodes in the cluster. This is // used to determine the retransmit count, which is calculated // based on the log of this. NumNodes func() int // RetransmitMult is the multiplier used to determine the maximum // number of retransmissions attempted. RetransmitMult int sync.Mutex // contains filtered or unexported fields }
TransmitLimitedQueue is used to queue messages to broadcast to the cluster (via gossip) but limits the number of transmits per message. It also prioritizes messages with lower transmit counts (hence newer messages).
func (*TransmitLimitedQueue) GetBroadcasts ¶
func (q *TransmitLimitedQueue) GetBroadcasts(overhead, limit int) [][]byte
GetBroadcasts is used to get a number of broadcasts, up to a byte limit and applying a per-message overhead as provided.
func (*TransmitLimitedQueue) NumQueued ¶
func (q *TransmitLimitedQueue) NumQueued() int
NumQueued returns the number of queued messages
func (*TransmitLimitedQueue) QueueBroadcast ¶
func (q *TransmitLimitedQueue) QueueBroadcast(b Broadcast)
QueueBroadcast is used to enqueue a broadcast
func (*TransmitLimitedQueue) Reset ¶
func (q *TransmitLimitedQueue) Reset()
Reset clears all the queued messages