raft

package module
v0.0.0-...-234e2d1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 3, 2020 License: MIT Imports: 16 Imported by: 0

README

Raft

Proof of concept implementation of the Raft consensus algorithm.

Raft is a consensus algorithm that is designed to be easy to understand. It's equivalent to Paxos in fault-tolerance and performance. Raft has already been implemented many times over and with multiple extensions and optimizations. Nevertheless, I decided to write one from the ground up to learn its internals.

Tooling dependencies

cd $GOPATH/src
go get -u github.com/gogo/protobuf/protoc-gen-gofast
github.com/gogo/protobuf/install-protobuf.sh  # it you don't already have protoc

Example KV Store

Run a pre-configured 3-node cluster

cd examples/kvstore/server
source ./utils.sh
start3cluster
# eventually call stop3cluster

Or run the cluster yourself manually

cd examples/kvstore/server
go build

# in one process
./server \
  -addr :3001 \
  -id 1 \
  -addresses "1,tcp://localhost:8081|2,tcp://localhost:8082|3,tcp://localhost:8083"

# in another process
./server \
  -addr :3002 \
  -id 2 \
  -addresses "1,tcp://localhost:8081|2,tcp://localhost:8082|3,tcp://localhost:8083"

# in yet another process
./server \
  -addr :3003 \
  -id 3 \
  -addresses "1,tcp://localhost:8081|2,tcp://localhost:8082|3,tcp://localhost:8083"

Interact with the cluster!

You can perform set & get operations against /store, e.g.

curl -XPUT "http://localhost:3001/store?k=hi&v=bye"
curl -XGET "http://localhost:3002/store?k=hi"

You can also check out the current Raft node state as well as its point of views of all of the Raft cluster members:

curl -XGET "http://localhost:3003/state"
curl -XGET "http://localhost:3003/members"

Finally, there's always the trusty /debug/pprof/ endpoint.

Building Protobuf files

# make sure you're in repo base
protoc \
  -I pb/ \
  -I $GOPATH/src \
  pb/raft.proto \
  --gogofaster_out=plugins=grpc:pb

protoc \
  -I examples/kvstore/kvpb/ \
  -I $GOPATH/src \
  examples/kvstore/kvpb/kv.proto \
  --gogofaster_out=plugins=grpc:examples/kvstore/kvpb

Tests

$ go test -v -timeout=5m -race . -args -test.disableLogging
=== RUN   Test_3Node_StartAndStop
--- PASS: Test_3Node_StartAndStop (2.50s)
=== RUN   Test_1Node_ConsistencyLease_RoundRobin
--- PASS: Test_1Node_ConsistencyLease_RoundRobin (3.21s)
=== RUN   Test_1Node_ConsistencyStrict_RoundRobin
--- PASS: Test_1Node_ConsistencyStrict_RoundRobin (3.20s)
=== RUN   Test_1Node_ConsistencyStale_RoundRobin
--- PASS: Test_1Node_ConsistencyStale_RoundRobin (3.20s)
=== RUN   Test_3Node_ConsistencyLease_RoundRobin
--- PASS: Test_3Node_ConsistencyLease_RoundRobin (5.40s)
=== RUN   Test_3Node_ConsistencyStrict_RoundRobin
--- PASS: Test_3Node_ConsistencyStrict_RoundRobin (5.20s)
=== RUN   Test_3Node_ConsistencyStale_RoundRobin
--- PASS: Test_3Node_ConsistencyStale_RoundRobin (5.10s)
=== RUN   Test_heartbeatTicker
--- PASS: Test_heartbeatTicker (0.01s)
=== RUN   Test_electionTicker
--- PASS: Test_electionTicker (0.02s)
=== RUN   ExampleApplication_register
--- PASS: ExampleApplication_register (2.20s)
PASS
ok  	github.com/ulysseses/raft	31.229s

Benchmarks

Benchmarks were performed on a relatively-noisy test environment (i.e. background apps were running). Tested on a Macbook Mac OS X 3.1 GHz Quad-Core i7 with 16 GB RAM.

Note: Go's builtin benchmark tooling may not accurately reflect the actual runtime. For example, reads performed on the leader of a 3-node cluster with ConsistencyStrict (i.e. reads require a quorum of heartbeat acks) take ~450ms on average, according to the benchmark shown below. However, the same read but on a follower node takes only ~12us. This is a huge discrepancy.

There may be a variety of factors of why this is the case, ranging from benchmark internals, cache contention, goroutine scheduler, GC, etc. I'm investigating the reasons why and may follow up with a solution or a blog post detailing my findings.

In the meantime, we can still rely on the go-wrk benchmarks, which paint a more accurate picture of the throughput and latency.

go-wrk benchmarks

go-wrk is a heavy-duty but small small http benchmark utility. Here we show benchmarks for a Strict-Serializable (ConsistencyStrict) write and read to the leader. You can expect better performance if tuning to ConsistencyLease (lease-based consistency) or ConsistencyStale (eventual consistency). Sending reads/writes to followers also work, but expect extra latency due to RTT: these requests are proxied to the leader.

$ cd examples/kvstore/server
$ source ./utils.sh
$ start3cluster  # :3002 is the leader, ConsistencyStrict mode
[2] 26631
[3] 26632
[4] 26633
{"level":"info","ts":1580078483.289747,"caller":"server/main.go:207","msg":"starting http kv store backed by raft","addr":":3001"}
{"level":"info","ts":1580078486.270493,"caller":"server/main.go:207","msg":"starting http kv store backed by raft","addr":":3002"}
{"level":"info","ts":1580078486.280907,"caller":"server/main.go:207","msg":"starting http kv store backed by raft","addr":":3003"}

$ go-wrk -c 1 -n 1000 -m PUT "http://localhost:3002/store?k=hi&v=bye"
==========================BENCHMARK==========================
URL:				http://localhost:3002/store?k=hi&v=bye

Used Connections:		1
Used Threads:			1
Total number of calls:		1000

===========================TIMINGS===========================
Total time passed:		0.89s
Avg time per request:		0.89ms
Requests per second:		1117.82
Median time per request:	0.70ms
99th percentile time:		1.45ms
Slowest time for request:	101.00ms

=============================DATA=============================
Total response body sizes:		0
Avg response body per request:		0.00 Byte
Transfer rate per second:		0.00 Byte/s (0.00 MByte/s)
==========================RESPONSES==========================
20X Responses:		1000	(100.00%)
30X Responses:		0	(0.00%)
40X Responses:		0	(0.00%)
50X Responses:		0	(0.00%)
Errors:			0	(0.00%)

$ go-wrk -c 1 -n 1000 -m GET "http://localhost:3002/store?k=hi"
==========================BENCHMARK==========================
URL:				http://localhost:3002/store?k=hi

Used Connections:		1
Used Threads:			1
Total number of calls:		1000

===========================TIMINGS===========================
Total time passed:		1.16s
Avg time per request:		1.15ms
Requests per second:		862.38
Median time per request:	0.85ms
99th percentile time:		2.57ms
Slowest time for request:	106.00ms

=============================DATA=============================
Total response body sizes:		3000
Avg response body per request:		3.00 Byte
Transfer rate per second:		2587.15 Byte/s (0.00 MByte/s)
==========================RESPONSES==========================
20X Responses:		1000	(100.00%)
30X Responses:		0	(0.00%)
40X Responses:		0	(0.00%)
50X Responses:		0	(0.00%)
Errors:			0	(0.00%)

$ stop3cluster
[4]  + 26633 done       ./server -addr=:3003 -id=3 ${common_flags[@]}
[3]  + 26632 done       ./server -addr=:3002 -id=2 ${common_flags[@]}
[2]  + 26631 done       ./server -addr=:3001 -id=1 ${common_flags[@]}

go test -bench benchmarks

$ go test -v -timeout=5m -run=^$ -bench=. -benchmem -args -test.disableLogging
goos: darwin
goarch: amd64
pkg: github.com/ulysseses/raft
Benchmark_gRPCTransport_RTT-8                        	   66874	     16175 ns/op	     597 B/op	      16 allocs/op
Benchmark_1Node_ConsistencyLease_RoundRobin-8        	  179720	      6337 ns/op	     864 B/op	      13 allocs/op
Benchmark_1Node_ConsistencyStrict_RoundRobin-8       	  185638	      6495 ns/op	     856 B/op	      13 allocs/op
Benchmark_1Node_ConsistencyStale_RoundRobin-8        	  236056	      4656 ns/op	     757 B/op	      12 allocs/op
Benchmark_3Node_ConsistencyLease_RoundRobin-8        	       7	 143167377 ns/op	    1442 B/op	      21 allocs/op
Benchmark_3Node_ConsistencyStrict_RoundRobin-8       	       7	 157589649 ns/op	    1449 B/op	      21 allocs/op
Benchmark_3Node_ConsistencyStale_RoundRobin-8        	      10	 169992499 ns/op	    1217 B/op	      21 allocs/op
Benchmark_3Node_ProposalAsLeader-8                   	   98838	     12310 ns/op	    1204 B/op	      16 allocs/op
Benchmark_3Node_ProposalAsFollower-8                 	       6	 199972993 ns/op	     976 B/op	      18 allocs/op
Benchmark_3Node_ConsistencyLease_ReadAsLeader-8   	  418468	      2945 ns/op	     304 B/op	       5 allocs/op
Benchmark_3Node_ConsistencyLease_ReadAsFollower-8    	  201946	      5605 ns/op	     304 B/op	       5 allocs/op
Benchmark_3Node_ConsistencyStrict_ReadAsLeader-8     	     100	 450769282 ns/op	    1560 B/op	      31 allocs/op
--- BENCH: Benchmark_3Node_ConsistencyStrict_ReadAsLeader-8
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
    functional_test.go:798: failed attempt #1 of getting value 42
	... [output truncated]
Benchmark_3Node_ConsistencyStrict_ReadAsFollower-8   	   94327	     12136 ns/op	     304 B/op	       5 allocs/op
Benchmark_3Node_ConsistencyStale_ReadAsLeader-8      	 2327655	       512 ns/op	     208 B/op	       4 allocs/op
Benchmark_3Node_ConsistencyStale_ReadAsFollower-8    	 2327632	       509 ns/op	     208 B/op	       4 allocs/op

License

This project is licensed under the MIT license.

Documentation

Overview

Package raft implements a proof-of-concept Raft consensus algorithm (See https://raft.github.io/).

End-users of the raft package should configure the Raft cluster and then run the Raft cluster as follows: 

func main() {
  // Start a 1-node cluster.
  id := uint64(1)
  addresses := map[uint64]string{1: "tcp://localhost:8001"}
  tr, err := raft.NewTransportConfig(id, addresses).Build()
  if err != nil {
    log.Fatal(err)
  }
  psm, err := raft.NewProtocolConfig(id).Build(tr)
  if err != nil {
    log.Fatal(err)
  }
  app := newApplication()
  node, err := raft.NewNodeConfig(id).Build(psm, tr, app)
  if err != nil {
    log.Fatal(err)
  }

  node.Start()
  defer node.Stop()

  // Here, app can interact with node by calling node.Propose() and node.Read()
  // ...
}

See examples/kvstore/server for an example demo of a key-value store backed by a 3-node Raft cluster. Run start3cluster in the provided utils.sh

Index

Examples

Constants

This section is empty.

Variables

View Source 
var ProtocolConfigTemplate = ProtocolConfig{
	TickPeriod: 100 * time.Millisecond,

	HeartbeatTicks: 1,

	MinElectionTicks: 10,
	MaxElectionTicks: 20,

	Consistency: ConsistencyLease,

	Lease: 500 * time.Millisecond,
}

ProtocolConfigTemplate is a partially filled ProtocolConfig that contains default values. Do not use ProtocolConfigTemplate directly; use NewProtocolConfig() instead.

View Source 
var TransportConfigTemplate = TransportConfig{

	MsgBufferSize: 30,

	Medium: &GRPCMedium{

		DialTimeout:    3 * time.Second,
		ReconnectDelay: 3 * time.Second,

		ServerOptions: []grpc.ServerOption{

			grpc.KeepaliveParams(keepalive.ServerParameters{
				Time:    5 * time.Second,
				Timeout: 5 * time.Second,
			}),
		},
	},
}

TransportConfigTemplate is a partially filled TransportConfig that contains default values. Do not use TransportConfigTemplate directly; use NewTransportConfig() instead.

Functions

This section is empty.

Types

type Application 

type Application interface {
	// Apply applies the newly committed entries to the application.
	Apply(entries []pb.Entry) error
}

Application applies the committed raft entries. Applications interfacing with the Raft node must implement this interface.

Example (Register) 
package main

import (
	"context"
	"fmt"
	"io/ioutil"
	"log"
	"os"
	"path/filepath"
	"strconv"
	"sync"
	"time"

	"github.com/ulysseses/raft"
	"github.com/ulysseses/raft/pb"
)

// register is a single-valued store
type register struct {
	sync.RWMutex
	node *raft.Node
	x    int
}

// Apply implements Application for register
func (r *register) Apply(entries []pb.Entry) error {
	r.Lock()
	defer r.Unlock()
	for _, entry := range entries {
		if len(entry.Data) == 0 {
			continue
		}
		x, err := strconv.Atoi(string(entry.Data))
		if err != nil {
			return err
		}
		r.x = x
	}
	return nil
}

func (r *register) set(ctx context.Context, x int) error {
	s := strconv.Itoa(x)
	_, _, err := r.node.Propose(ctx, []byte(s))
	return err
}

func (r *register) get(ctx context.Context) (int, error) {
	if err := r.node.Read(ctx); err != nil {
		return 0, err
	}
	r.RLock()
	defer r.RUnlock()
	return r.x, nil
}

func main() {
	tmpDir, err := ioutil.TempDir("", "")
	if err != nil {
		log.Fatal(err)
	}
	defer os.RemoveAll(tmpDir)
	id := uint64(1)
	addresses := map[uint64]string{
		1: fmt.Sprintf("unix://%s", filepath.Join(tmpDir, "ExampleApplication_register.sock")),
	}

	// Start up the Raft node.
	r := &register{}
	tr, err := raft.NewTransportConfig(id, raft.WithAddresses(addresses)).Build()
	if err != nil {
		log.Fatal(err)
	}
	psm, err := raft.NewProtocolConfig(id).Build(tr)
	if err != nil {
		log.Fatal(err)
	}
	node, err := raft.NewNodeConfig(id).Build(psm, tr, r)
	if err != nil {
		log.Fatal(err)
	}
	r.node = node
	node.Start()
	defer node.Stop()

	// Wait for leader election
	time.Sleep(2 * time.Second)

	// Commit an entry via the register IDed 1
	if err := r.set(context.Background(), 42); err != nil {
		log.Fatalf("set failed: %v", err)
	}

	// Get that committed entry via register IDed 2
	if x, err := r.get(context.Background()); err == nil {
		fmt.Println(x)

	} else {
		log.Fatalf("get failed: %v", err)
	}
}

Output:

42

type ChannelMedium 

type ChannelMedium struct {
	// MemberIDs is a slice of the IDs of all the Raft cluster members, including self.
	MemberIDs []uint64
}

ChannelMedium sends messages over Go channels.

type Consistency 

type Consistency uint8

Consistency is the consistency mode that Raft operations should support. 

const (
	// ConsistencyLease follows the serializable consistency model. If there is a leadership
	// change, read requests are potentially stale for a maximum of the lease duration amount.
	ConsistencyLease Consistency = iota
	// ConsistencyStrict follows the linearizable consistency model. Every read request will
	// require a quorum's worth of heartbeat acks.
	ConsistencyStrict
	// ConsistencyStale follows the serializable consistency model.
	// Reads can be stale for an arbitrary amount of time. Writes never diverge.
	ConsistencyStale
)

func (Consistency) MarshalJSON 

func (c Consistency) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for Consistency

func (Consistency) String 

func (c Consistency) String() string

func (*Consistency) UnmarshalJSON 

func (c *Consistency) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler for Consistency

type GRPCMedium 

type GRPCMedium struct {
	// Addresses mapping Raft node ID to address to connect to.
	Addresses map[uint64]string

	// DialTimeout is the timeout for dialing to peers.
	// ReconnectDelay is the duration to wait before retrying to dial a connection.
	DialTimeout, ReconnectDelay time.Duration

	// ServerOptions is an optional list of grpc.ServerOptions to configure the gRPC server.
	ServerOptions []grpc.ServerOption

	// DialOptions is an optional list of grpc.DialOptions to configure dialing to the peer
	// gRPC servers.
	DialOptions []grpc.DialOption

	// CallOptions is an optional list of grpc.CallOptions to configure calling the Communicate RPC.
	CallOptions []grpc.CallOption
}

GRPCMedium sends messages over the gRPC RaftProtocol service (via client stream).

type Lease 

type Lease struct {
	// Timeout is the point in time that a lease should timeout.
	Timeout int64

	// Start marks the beginning of the attempt to extend the lease, i.e. sending out new heartbeats
	Start int64

	// If a majority of heartbeats were acked, the Timeout should be extended to Start + Extension
	Extension int64

	// Acks is the number of acks for the specified Start.
	Acks int
}

Lease contains fields related to lease for read requests. This is used only by leaders in ConsistencyLease mode.

type Medium 

type Medium interface {
	// contains filtered or unexported methods
}

Medium represents which the type of communication medium that pb.Messages should be sent. ChannelMedium sends messages over Go channels. GRPCMedium sends messages over the gRPC RaftProtocol service (via client stream).

type MemberState 

type MemberState struct {
	// peer node's ID
	ID uint64

	// last known largest index that this peer matches this node's log
	Match uint64

	// index of the prefix log of entries to send in the heartbeat to the peer
	Next uint64

	// whether or not the peer responded to the heartbeat within the election timeout
	Ack bool

	// vote was granted to elect us by this peer
	VoteGranted bool

	// Read of this member. This is only used by ConsistencyStrict.
	Read Read
}

MemberState contains all info about a member node from the perspective of this node.

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node is a Raft node that interacts with an Application state machine and network.

func (*Node) Members 

func (n *Node) Members() map[uint64]MemberState

Members returns the latest member states.

func (*Node) Propose 

func (n *Node) Propose(ctx context.Context, data []byte) (index uint64, term uint64, err error)

Propose should be called by the client/application. This method proposes data to the raft log. If error is non-nil, index and term should be used to check if the proposed entry was committed.

func (*Node) Read 

func (n *Node) Read(ctx context.Context) error

Read should be called by the client/application.

func (*Node) Start 

func (n *Node) Start()

Start starts the Raft node.

func (*Node) State 

func (n *Node) State() State

State returns the latest known state of the Raft node.

func (*Node) Stop 

func (n *Node) Stop() error

Stop stops the Raft node.

type NodeConfig 

type NodeConfig struct {
	// ID of the Raft node.
	ID uint64

	// Logger, if provided, will be used to log events.
	Logger *zap.Logger

	// Debug, if true, will log events at the DEBUG verbosity/granularity.
	Debug bool
}

NodeConfig configures Node-specific configuration.

func NewNodeConfig 

func NewNodeConfig(id uint64, opts ...NodeConfigOption) *NodeConfig

NewNodeConfig builds a NodeConfig for a Raft node.

func (*NodeConfig) Build 

func (c *NodeConfig) Build(
	psm *ProtocolStateMachine,
	tr Transport,
	a Application,
) (*Node, error)

Build builds a Raft node.

func (*NodeConfig) Verify 

func (c *NodeConfig) Verify() error

Verify verifies that the configuration is correct.

type NodeConfigOption 

type NodeConfigOption interface{ Transform(*NodeConfig) }

NodeConfigOption provides options to configure Node specifically.

func AddNodeLogger 

func AddNodeLogger() NodeConfigOption

AddNodeLogger adds a default production zap.Logger to the configuration.

func WithNodeDebug 

func WithNodeDebug(debug bool) NodeConfigOption

WithNodeDebug sets the debug field for the NodeConfig.

func WithNodeLogger 

func WithNodeLogger(logger *zap.Logger) NodeConfigOption

WithNodeLogger configures to use a specified logger for the protocol state machine.

type Proposal 

type Proposal struct {
	TID int64

	// Index is the proposed index.
	// Term is the proposed term.
	Index, Term uint64
	// contains filtered or unexported fields
}

Proposal is the context associated with a proposal.

type ProtocolConfig 

type ProtocolConfig struct {
	// ID of the Raft node.
	ID uint64

	// TickPeriod is the period of tiem at which the ticker should fire.
	TickPeriod time.Duration

	// HeartbeatTicks is the number of tick periods before a heartbeat
	// should fire.
	// MinElectionTicks is the minimum number of tick periods before an
	// election timeout should fire.
	// MaxElectionTicks is the maximum number of tick periods before an
	// election timeout should fire.
	HeartbeatTicks, MinElectionTicks, MaxElectionTicks uint

	// Consistency is the consistency level to use for the Raft cluster.
	Consistency Consistency

	// Lease is the duration of the read request lease. This is used only if ConsistencyLease.
	Lease time.Duration

	// HeartbeatTicker is the ticker to use for signaling when to send out heartbeats. If nil,
	// a default one based on TickPeriod and HeartbeatTicks is created and used.
	// ElectionTicker is the ticker to use for signaling when to timeout an election. If nil,
	// a default one based on TickPeriod, MinElectionTicks, and MaxElectionTicks is created and used.
	HeartbeatTicker, ElectionTicker Ticker

	// Logger, if provided, will be used to log events.
	Logger *zap.Logger

	// Debug, if true, will log events at the DEBUG verbosity/granularity.
	Debug bool
}

ProtocolConfig configures the Raft protocol of the Raft cluster.

func NewProtocolConfig 

func NewProtocolConfig(id uint64, opts ...ProtocolConfigOption) *ProtocolConfig

NewProtocolConfig builds a ProtocolConfig for a Raft node.

func (*ProtocolConfig) Build 

func (c *ProtocolConfig) Build(tr Transport) (*ProtocolStateMachine, error)

Build builds a ProtocolStateMachine from configuration

func (*ProtocolConfig) Verify 

func (c *ProtocolConfig) Verify() error

Verify verifies that the configuration is correct.

type ProtocolConfigOption 

type ProtocolConfigOption interface{ Transform(*ProtocolConfig) }

ProtocolConfigOption provides options to configure ProtocolConfig further.

func AddProtocolLogger 

func AddProtocolLogger() ProtocolConfigOption

AddProtocolLogger adds a default production zap.Logger to the configuration.

func WithConsistency 

func WithConsistency(consistency Consistency) ProtocolConfigOption

WithConsistency sets the consistency mode.

func WithElectionTicker 

func WithElectionTicker(ticker Ticker) ProtocolConfigOption

WithElectionTicker configures to use a specified election timeout ticker.

func WithHeartbeatTicker 

func WithHeartbeatTicker(ticker Ticker) ProtocolConfigOption

WithHeartbeatTicker configures to use a specified heartbeat ticker.

func WithHeartbeatTicks 

func WithHeartbeatTicks(heartbeatTicks uint) ProtocolConfigOption

WithHeartbeatTicks sets the specified heartbeat ticks.

func WithLease 

func WithLease(lease time.Duration) ProtocolConfigOption

WithLease sets the lease duration. This should be used if using ConsistencyLease.

func WithMaxElectionTicks 

func WithMaxElectionTicks(maxElectionTicks uint) ProtocolConfigOption

WithMaxElectionTicks sets the specified minimum election timeout ticks.

func WithMinElectionTicks 

func WithMinElectionTicks(minElectionTicks uint) ProtocolConfigOption

WithMinElectionTicks sets the specified minimum election timeout ticks.

func WithProtocolDebug 

func WithProtocolDebug(debug bool) ProtocolConfigOption

WithProtocolDebug sets the debug field for the ProtocolConfig.

func WithProtocolLogger 

func WithProtocolLogger(logger *zap.Logger) ProtocolConfigOption

WithProtocolLogger configures to use a specified logger for the protocol state machine.

func WithTickPeriod 

func WithTickPeriod(tickPeriod time.Duration) ProtocolConfigOption

WithTickPeriod sets a specified tick period.

type ProtocolStateMachine 

type ProtocolStateMachine struct {
	// contains filtered or unexported fields
}

ProtocolStateMachine represents the Raft Protocol state machine of a Raft node. It has a central event loop that interacts with a heartbeat ticker, election ticker, and Raft protocol messages sent/received over the transport network.

type Read 

type Read struct {
	// TID is the "transaction ID". It increases monotonically.
	TID int64

	// Index is the read index of the read request.
	Index uint64

	// Acks is the number of acks for the latest read request.
	Acks int
}

Read contains all fields relevant to read requests.

type Role 

type Role uint8

Role can be follower, candidate, or leader. 

const (
	// RoleFollower is the follower role.
	RoleFollower Role = iota
	// RoleCandidate is the candidate role.
	RoleCandidate
	// RoleLeader is the leader role.
	RoleLeader
)

func (Role) MarshalJSON 

func (r Role) MarshalJSON() ([]byte, error)

MarshalJSON implements json.Marshaler for Role

func (Role) String 

func (r Role) String() string

func (*Role) UnmarshalJSON 

func (r *Role) UnmarshalJSON(b []byte) error

UnmarshalJSON implements json.Unmarshaler for Role

type State 

type State struct {
	// id
	ID uint64

	// Consistency mode
	Consistency Consistency

	// quorum size
	QuorumSize int

	// cluster size
	ClusterSize int

	// role
	Role Role

	// current term
	Term uint64

	// who this node thinks currently is the leader.
	Leader uint64

	// committed index
	Commit uint64

	// who this node last voted for
	VotedFor uint64

	// last index of this node's log
	LastIndex uint64

	// largest term of this node's log
	LogTerm uint64

	// context of a proposal, if any.
	Proposal Proposal

	// context of a read request originating from this node, if any.
	// This is used only by ConsistencyStrict.
	Read Read

	// lease, if using ConsistencyLease mode.
	Lease Lease
}

State contains all state of a Node.

type Ticker 

type Ticker interface {
	// Start starts the ticker.
	Start()
	// Stop stops the ticker.
	Stop()
	// C returns the channel to read ticks from.
	C() <-chan struct{}

	// Reset resets the ticker.
	Reset()
}

Ticker sends ticks.

type Transport 

type Transport interface {
	// contains filtered or unexported methods
}

Transport sends and receives Raft protocol messages between Raft nodes of the cluster.

type TransportConfig 

type TransportConfig struct {
	// ID of the Raft node to configure.
	ID uint64

	// MsgBufferSize is the max number of Raft protocol messages per peer node allowed to be buffered
	// before the Raft node can process/send them out.
	MsgBufferSize int

	// Logger, if provided, will be used to log events.
	Logger *zap.Logger

	// Debug, if true, will log events at the DEBUG verbosity/granularity.
	Debug bool

	// Medium represents which the type of communication medium that pb.Messages should be sent.
	// The default medium is GRPCMedium.
	Medium Medium
}

TransportConfig configures transport for the Raft cluster.

func NewTransportConfig 

func NewTransportConfig(
	id uint64,
	opts ...TransportConfigOption,
) *TransportConfig

NewTransportConfig builds a TransportConfig for a Raft node.

func (*TransportConfig) Build 

func (c *TransportConfig) Build() (Transport, error)

Build builds a Transport from configuration.

func (*TransportConfig) Verify 

func (c *TransportConfig) Verify() error

Verify verifies that the configuration is correct.

type TransportConfigOption 

type TransportConfigOption interface{ Transform(*TransportConfig) }

TransportConfigOption provides options to configure TransportConfig further.

func AddTransportLogger 

func AddTransportLogger() TransportConfigOption

AddTransportLogger adds a default production zap.Logger to the configuration.

func WithAddresses 

func WithAddresses(addresses map[uint64]string) TransportConfigOption

WithAddresses sets the addresses of all Raft cluster nodes.

func WithChannelMedium 

func WithChannelMedium(memberIDs ...uint64) TransportConfigOption

WithChannelMedium configures to use Go channels to transport messages instead of default gRPC. This option should only be used in testing!

func WithGRPCCallOption 

func WithGRPCCallOption(opt grpc.CallOption) TransportConfigOption

WithGRPCCallOption adds a grpc.CallOption to grpc.NewServer

func WithGRPCDialOption 

func WithGRPCDialOption(opt grpc.DialOption) TransportConfigOption

WithGRPCDialOption adds a grpc.DialOption to grpc.NewServer

func WithGRPCServerOption 

func WithGRPCServerOption(opt grpc.ServerOption) TransportConfigOption

WithGRPCServerOption adds a grpc.ServerOption to grpc.NewServer

func WithSecurity 

func WithSecurity(opt grpc.DialOption) TransportConfigOption

WithSecurity configures gRPC to use security instead of the default grpc.WithInsecure option.

func WithTransportDebug 

func WithTransportDebug(debug bool) TransportConfigOption

WithTransportDebug sets the debug field for the TransportConfig.

func WithTransportLogger 

func WithTransportLogger(logger *zap.Logger) TransportConfigOption

WithTransportLogger configures to use a specified logger for the protocol state machine.

Source Files

View all Source files

Directories

Path Synopsis
examples
kvstore/kvpb
kvstore/server

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.