cerebrum

package module
v0.0.0-...-cc25aa6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 5, 2019 License: Apache-2.0 Imports: 22 Imported by: 0

README

cerebrum

Cerebrum is a toolkit for distributed systems

Deprecated and unfinished.

Documentation

Index

Constants

View Source
const (
	CerebrumServiceID   = "cerebrum"
	CerebrumRole        = "cerebrum-server"
	CerebrumEventPrefix = CerebrumServiceID + ":"
	CerebrumLeaderEvent = CerebrumEventPrefix + "new-leader"
)

Constants

View Source
const (
	SerfSnapshotDir = "serf/local.snapshot"
	RaftStateDir    = "raft/"

	SnapshotsRetained = 2
)
View Source
const (
	// StatusReap is used to update the status of a node if we
	// are handling a EventMemberReap
	StatusReap = serf.MemberStatus(-1)
)

Variables

View Source
var ErrNoLeader = errors.New("No cluster leader")
View Source
var ErrUnknownConnType = errors.New("Unknown connection type")

Functions

func GetFullEventName

func GetFullEventName(name string) string

GetFullEventName computes the full name of a event

func GetRawEventName

func GetRawEventName(name string) string

GetRawEventName is used to get the raw event name

func IsCerebrumEvent

func IsCerebrumEvent(name string) bool

IsCerebrumEvent checks if a serf event is a cerebrum event

func NewFSM

func NewFSM(path string, userFSM raft.FSM, logOutput io.Writer) (raft.FSM, error)

NewFSM is used to construct a new FSM with a blank state

func ValidateNode

func ValidateNode(member serf.Member) (ok bool, role, dc string)

ValidateNode determines whether a node is a known server and returns

its data center and role.

Types

type Applier

type Applier interface {

	// Apply performs the Raft or forward operation depending on the node's
	// leader status.
	Apply(namedtuple.Tuple) error
}

Applier applies tuples to the Raft log if the node is the leader, otherwise the tuple will be forwarded to the leader.

func NewApplier

func NewApplier(r RaftApplier, f Forwarder, l log.Logger, timeout time.Duration) Applier

type Cerebrum

type Cerebrum interface {
	Start() error
	Stop()
}

func New

func New(c *Config) (cer Cerebrum, err error)

type Config

type Config struct {

	// Bootstrap allows for a single node to become the leader for Raft.
	Bootstrap bool

	// NodeID should be unique across all nodes in the cluster.
	NodeID string

	// NodeName is the name of this node.
	NodeName string

	// DataCenter is the name of the data center for this node.
	DataCenter string

	// ServicePrefix is used to filter out unknown events.
	ServicePrefix string

	// DataPath is where all the data is stored
	DataPath string

	// LeaderElectionHandler processes leader election events.
	LeaderElectionHandler serfer.LeaderElectionHandler

	// UserEvent processes known, non-leader election events.
	UserEvent serfer.UserEventHandler

	// UnknownEventHandler processes unkown events.
	UnknownEventHandler serfer.UnknownEventHandler

	// Called when a Member joins the cluster.
	NodeJoined serfer.MemberJoinHandler

	// Called when a Member leaves the cluster by sending a leave message.
	NodeLeft serfer.MemberLeaveHandler

	// Called when a Member has been detected as failed.
	NodeFailed serfer.MemberFailureHandler

	// Called when a Member has been Readed from the cluster.
	NodeReaped serfer.MemberReapHandler

	// Called when a Member has been updated.
	NodeUpdated serfer.MemberUpdateHandler

	// Called when a serf.Query is received.
	QueryHandler serfer.QueryEventHandler

	// GossipBindAddr is the address of the Serf server.
	GossipBindAddr string

	// GossipBindPort is the port for the Serf server.
	GossipBindPort int

	// GossipAdvertiseAddr is the advertising address for the Serf server.
	GossipAdvertiseAddr string

	// GossipAdvertisePort is the advertising port for the Serf server.
	GossipAdvertisePort int

	// LogOutput is the output for all logs.
	LogOutput io.Writer

	// RaftConfig configures the Raft server.
	RaftConfig *raft.Config

	// SnapshotsRetained is the number of snapshots kept for Raft
	SnapshotsRetained int

	// LogCacheSize is the number of log entries to keep in memory.
	LogCacheSize int

	// TLSConfig is the config for Raft over TLS
	TLSConfig *tls.Config

	// RaftBindAddr is the bind address and port for the Raft TLS server.
	RaftBindAddr string

	// ReconcileInterval is the interval at which Raft makes sure the FSM has caught up.
	ReconcileInterval time.Duration

	// ConnectionDeadline is the maximum the TLS server will wait for connections.
	// This deadline also applies to the ammount of time to wait for the server to shutdown.
	ConnectionDeadline time.Duration

	// EnqueueTimeout is the maximum amount of time a Raft submission will wait
	// before timing out.
	EnqueueTimeout time.Duration

	// EstablishLeadership is called when a node becomes the leader of the
	// Raft cluster. This function can be called multiple times if it returns an
	// error.
	EstablishLeadership func() error

	// RevokeLeadership is called when a node loses the leadership of the
	// Raft cluster.
	RevokeLeadership func() error

	// Services is an array of services running on top of Cerebrum.
	Services []Service

	// ExistingNodes is an array of nodes already in the cluster.
	ExistingNodes []string
}

type Conn

type Conn struct {
	// contains filtered or unexported fields
}

Conn is a pooled connection to a Cerebrum server

func (*Conn) Close

func (c *Conn) Close() error

type ConnPool

type ConnPool struct {
	sync.Mutex
	// contains filtered or unexported fields
}

ConnPool is used to maintain a connection pool to other Nomad servers. This is used to reduce the latency of RPC requests between servers. It is only used to pool connections in the rpc mode. Raft connections are pooled separately.

func NewPool

func NewPool(logOutput io.Writer, maxTime time.Duration, config *tls.Config) *ConnPool

NewPool is used to make a new connection pool Maintain at most one connection per host, for up to maxTime. Set maxTime to 0 to disable reaping. If TLS settings are provided outgoing connections use TLS.

func (*ConnPool) Shutdown

func (p *ConnPool) Shutdown() error

Shutdown is used to close the connection pool

type Context

type Context struct {
	Context context.Context
	Serf    *serf.Serf
	Raft    *raft.Raft
}

type Dialer

type Dialer interface {
	Dial(t yamuxer.StreamType, addr string, timeout time.Duration) (net.Conn, error)
	Shutdown() error
}

Dialer dials an address and initializes a connection of the given type. If the timeout occurs before the dial completes, an error will be returned.

func NewDialer

func NewDialer(p *ConnPool) Dialer

NewDialer creates a new Dialer implementation with a connection pool.

type Forwarder

type Forwarder interface {
	Forward([]byte) error
}

Forwarder forwards data to the cluster leader. If the leader is unknown, an error will be returned. If the leader cannot be contacted, an error returned. If the data cannot be written to the leader, an error will be returned.

func NewForwarder

func NewForwarder(r RaftApplier, d Dialer, l log.Logger) Forwarder

type ForwardingHandler

type ForwardingHandler struct {
	// contains filtered or unexported fields
}

func (*ForwardingHandler) Handle

func (f *ForwardingHandler) Handle(c context.Context, conn net.Conn)

type NodeDetails

type NodeDetails struct {
	Bootstrap  bool
	ID         string
	Name       string
	Role       string
	DataCenter string
	Addr       net.IP
	Port       int
	Services   []NodeService
	Tags       map[string]string
	Status     serf.MemberStatus
}

NodeDetails stores details about a single serf.Member

func GetNodeDetails

func GetNodeDetails(m serf.Member) (n *NodeDetails, err error)

GetNodeDetails should validate all the Serf tags for the given member and return NodeDetails or any error that occurred.

func (NodeDetails) String

func (n NodeDetails) String() (s string)

type NodeService

type NodeService struct {
	Name string
	Port int
}

type NodeStatus

type NodeStatus uint8
const (
	StatusAlive NodeStatus = iota
	StatusFailed
	StatusLeft
	StatusReaped
)

type RaftApplier

type RaftApplier interface {
	Apply(cmd []byte, timeout time.Duration) raft.ApplyFuture
	State() raft.RaftState
	Leader() string
}

RaftApplier covers a few of the raft.Raft methods to make testing easier.

type RaftLayer

type RaftLayer struct {
	// contains filtered or unexported fields
}

RaftLayer implements the raft.StreamLayer interface, so that we can use a single RPC layer for Raft and Cerebrum

func NewRaftLayer

func NewRaftLayer(dialer Dialer, addr net.Addr, config *tls.Config) *RaftLayer

NewRaftLayer is used to initialize a new RaftLayer which can be used as a StreamLayer for Raft. If a tlsConfig is provided, then the connection will use TLS.

func (*RaftLayer) Accept

func (l *RaftLayer) Accept() (net.Conn, error)

Accept is used to return connection which are dialed to be used with the Raft layer

func (*RaftLayer) Addr

func (l *RaftLayer) Addr() net.Addr

Addr is used to return the address of the listener

func (*RaftLayer) Close

func (l *RaftLayer) Close() error

Close is used to stop listening for Raft connections

func (*RaftLayer) Dial

func (l *RaftLayer) Dial(address string, timeout time.Duration) (net.Conn, error)

Dial is used to create a new outgoing connection

func (*RaftLayer) Handle

func (l *RaftLayer) Handle(c context.Context, conn net.Conn)

func (*RaftLayer) Handoff

func (l *RaftLayer) Handoff(c net.Conn) error

Handoff is used to hand off a connection to the RaftLayer. This allows it to be Accept()'ed

type Reconciler

type Reconciler struct {
	ReconcileCh chan serf.Member
	IsLeader    func() bool
}

Reconciler dispatches membership changes to Raft.

func (*Reconciler) Reconcile

func (r *Reconciler) Reconcile(m serf.Member)

Reconcile is used to reconcile Serf events with the strongly consistent store if we are the current leader

type Service

type Service interface {
	Name() string
	Start(*Context) error
	Stop()
}

type TLSStreamLayer

type TLSStreamLayer struct {
	// contains filtered or unexported fields
}

TLSStreamLayer implements raft.StreamLayer interface over TLS.

func NewTLSStreamLayer

func NewTLSStreamLayer(bindAddr string, logOutput io.Writer, config *tls.Config) (*TLSStreamLayer, error)

NewTLSStreamLayer returns a raft.StreamLayer that is built on top of a TLS streaming transport layer.

func (*TLSStreamLayer) Accept

func (t *TLSStreamLayer) Accept() (c net.Conn, err error)

Accept implements the net.Listener interface.

func (*TLSStreamLayer) Addr

func (t *TLSStreamLayer) Addr() net.Addr

Addr implements the net.Listener interface.

func (*TLSStreamLayer) Close

func (t *TLSStreamLayer) Close() (err error)

Close implements the net.Listener interface.

func (*TLSStreamLayer) Dial

func (t *TLSStreamLayer) Dial(address string, timeout time.Duration) (net.Conn, error)

Dial implements the StreamLayer interface.

Directories

Path Synopsis
Godeps
+build go1.3
_workspace/src/github.com/blacklabeldata/serfer
Serfer is a simple library which provides interfaces for handling various Serf events.
Serfer is a simple library which provides interfaces for handling various Serf events.
_workspace/src/github.com/boltdb/bolt
Package bolt implements a low-level key/value store in pure Go.
Package bolt implements a low-level key/value store in pure Go.
_workspace/src/github.com/hashicorp/go-msgpack/codec
High Performance, Feature-Rich Idiomatic Go encoding library for msgpack and binc .
High Performance, Feature-Rich Idiomatic Go encoding library for msgpack and binc .
_workspace/src/github.com/hashicorp/memberlist
memberlist is a library that manages cluster membership and member failure detection using a gossip based protocol.
memberlist is a library that manages cluster membership and member failure detection using a gossip based protocol.
_workspace/src/github.com/mattn/go-isatty
Package isatty implements interface to isatty
Package isatty implements interface to isatty
_workspace/src/github.com/mgutz/ansi
Package ansi is a small, fast library to create ANSI colored strings and codes.
Package ansi is a small, fast library to create ANSI colored strings and codes.
_workspace/src/github.com/stretchr/objx
objx - Go package for dealing with maps, slices, JSON and other data.
objx - Go package for dealing with maps, slices, JSON and other data.
_workspace/src/github.com/stretchr/testify/assert
A set of comprehensive testing tools for use with the normal Go testing system.
A set of comprehensive testing tools for use with the normal Go testing system.
_workspace/src/github.com/stretchr/testify/mock
Provides a system by which it is possible to mock your objects and verify calls are happening as expected.
Provides a system by which it is possible to mock your objects and verify calls are happening as expected.
_workspace/src/golang.org/x/net/context
Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes.
Package context defines the Context type, which carries deadlines, cancelation signals, and other request-scoped values across API boundaries and between processes.
_workspace/src/gopkg.in/tomb.v2
The tomb package handles clean goroutine tracking and termination.
The tomb package handles clean goroutine tracking and termination.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL