vault: github.com/hashicorp/vault/physical/raft Index | Files | Directories

package raft

import "github.com/hashicorp/vault/physical/raft"

Index

Package Files

fsm.go raft.go raft_util.go snapshot.go streamlayer.go types.pb.go

func EnsurePath Uses

func EnsurePath(path string, dir bool) error

EnsurePath is used to make sure a path exists

func NewRaftBackend Uses

func NewRaftBackend(conf map[string]string, logger log.Logger) (physical.Backend, error)

NewRaftBackend constructs a RaftBackend using the given directory

func NewRaftLayer Uses

func NewRaftLayer(logger log.Logger, raftTLSKeyring *TLSKeyring, clusterAddr net.Addr, baseTLSConfig *tls.Config) (*raftLayer, error)

NewRaftLayer creates a new raftLayer object. It parses the TLS information from the network config.

type BoltSnapshotSink Uses

type BoltSnapshotSink struct {
    // contains filtered or unexported fields
}

BoltSnapshotSink implements SnapshotSink optionally choosing to write to a file.

func (*BoltSnapshotSink) Cancel Uses

func (s *BoltSnapshotSink) Cancel() error

Cancel is used to indicate an unsuccessful end.

func (*BoltSnapshotSink) Close Uses

func (s *BoltSnapshotSink) Close() error

Close is used to indicate a successful end.

func (*BoltSnapshotSink) ID Uses

func (s *BoltSnapshotSink) ID() string

ID returns the ID of the snapshot, can be used with Open() after the snapshot is finalized.

func (*BoltSnapshotSink) Write Uses

func (s *BoltSnapshotSink) Write(b []byte) (int, error)

Write is used to append to the state file. We write to the buffered IO object to reduce the amount of context switches.

type BoltSnapshotStore Uses

type BoltSnapshotStore struct {
    // contains filtered or unexported fields
}

BoltSnapshotStore implements the SnapshotStore interface and allows snapshots to be made on the local disk. The main difference between this store and the file store is we make the distinction between snapshots that have been written by the FSM and by internal Raft operations. The former are treated as noop snapshots on Persist and are read in full from the FSM on Open. The latter are treated like normal file snapshots and are able to be opened and applied as usual.

func NewBoltSnapshotStore Uses

func NewBoltSnapshotStore(base string, retain int, logger log.Logger, fsm *FSM) (*BoltSnapshotStore, error)

NewBoltSnapshotStore creates a new BoltSnapshotStore based on a base directory. The `retain` parameter controls how many snapshots are retained. Must be at least 1.

func (*BoltSnapshotStore) Create Uses

func (f *BoltSnapshotStore) Create(version raft.SnapshotVersion, index, term uint64,
    configuration raft.Configuration, configurationIndex uint64, trans raft.Transport) (raft.SnapshotSink, error)

Create is used to start a new snapshot

func (*BoltSnapshotStore) List Uses

func (f *BoltSnapshotStore) List() ([]*raft.SnapshotMeta, error)

List returns available snapshots in the store. It only returns bolt snapshots. No snapshot will be returned if there are no indexes in the FSM.

func (*BoltSnapshotStore) Open Uses

func (f *BoltSnapshotStore) Open(id string) (*raft.SnapshotMeta, io.ReadCloser, error)

Open takes a snapshot ID and returns a ReadCloser for that snapshot.

func (*BoltSnapshotStore) ReapSnapshots Uses

func (f *BoltSnapshotStore) ReapSnapshots() error

ReapSnapshots reaps any snapshots beyond the retain count.

type ConfigurationValue Uses

type ConfigurationValue struct {
    Index                uint64    `protobuf:"varint,1,opt,name=index,proto3" json:"index,omitempty"`
    Servers              []*Server `protobuf:"bytes,2,rep,name=servers,proto3" json:"servers,omitempty"`
    XXX_NoUnkeyedLiteral struct{}  `json:"-"`
    XXX_unrecognized     []byte    `json:"-"`
    XXX_sizecache        int32     `json:"-"`
}

func (*ConfigurationValue) Descriptor Uses

func (*ConfigurationValue) Descriptor() ([]byte, []int)

func (*ConfigurationValue) GetIndex Uses

func (m *ConfigurationValue) GetIndex() uint64

func (*ConfigurationValue) GetServers Uses

func (m *ConfigurationValue) GetServers() []*Server

func (*ConfigurationValue) ProtoMessage Uses

func (*ConfigurationValue) ProtoMessage()

func (*ConfigurationValue) Reset Uses

func (m *ConfigurationValue) Reset()

func (*ConfigurationValue) String Uses

func (m *ConfigurationValue) String() string

func (*ConfigurationValue) XXX_DiscardUnknown Uses

func (m *ConfigurationValue) XXX_DiscardUnknown()

func (*ConfigurationValue) XXX_Marshal Uses

func (m *ConfigurationValue) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*ConfigurationValue) XXX_Merge Uses

func (m *ConfigurationValue) XXX_Merge(src proto.Message)

func (*ConfigurationValue) XXX_Size Uses

func (m *ConfigurationValue) XXX_Size() int

func (*ConfigurationValue) XXX_Unmarshal Uses

func (m *ConfigurationValue) XXX_Unmarshal(b []byte) error

type FSM Uses

type FSM struct {
    // contains filtered or unexported fields
}

FSM is Vault's primary state storage. It writes updates to an bolt db file that lives on local disk. FSM implements raft.FSM and physical.Backend interfaces.

func NewFSM Uses

func NewFSM(conf map[string]string, logger log.Logger) (*FSM, error)

NewFSM constructs a FSM using the given directory

func (*FSM) Apply Uses

func (f *FSM) Apply(log *raft.Log) interface{}

Apply will apply a log value to the FSM. This is called from the raft library.

func (*FSM) Delete Uses

func (f *FSM) Delete(ctx context.Context, path string) error

Delete deletes the given key from the bolt file.

func (*FSM) DeletePrefix Uses

func (f *FSM) DeletePrefix(ctx context.Context, prefix string) error

Delete deletes the given key from the bolt file.

func (*FSM) Get Uses

func (f *FSM) Get(ctx context.Context, path string) (*physical.Entry, error)

Get retrieves the value at the given path from the bolt file.

func (*FSM) LatestState Uses

func (f *FSM) LatestState() (*IndexValue, *ConfigurationValue)

LatestState returns the latest index and configuration values we have seen on this FSM.

func (*FSM) List Uses

func (f *FSM) List(ctx context.Context, prefix string) ([]string, error)

List retrieves the set of keys with the given prefix from the bolt file.

func (*FSM) Put Uses

func (f *FSM) Put(ctx context.Context, entry *physical.Entry) error

Put writes the given entry to the bolt file.

func (*FSM) Restore Uses

func (f *FSM) Restore(r io.ReadCloser) error

Restore reads data from the provided reader and writes it into the FSM. It first deletes the existing bucket to clear all existing data, then recreates it so we can copy in the snapshot.

func (*FSM) SetNoopRestore Uses

func (f *FSM) SetNoopRestore(enabled bool)

SetNoopRestore is used to disable restore operations on raft startup. Because we are using persistent storage in our FSM we do not need to issue a restore on startup.

func (*FSM) Snapshot Uses

func (f *FSM) Snapshot() (raft.FSMSnapshot, error)

Snapshot implements the FSM interface. It returns a noop snapshot object.

func (*FSM) StoreConfiguration Uses

func (f *FSM) StoreConfiguration(index uint64, configuration raft.Configuration)

StoreConfig satisfies the raft.ConfigurationStore interface and persists the latest raft server configuration to the bolt file.

func (*FSM) Transaction Uses

func (f *FSM) Transaction(ctx context.Context, txns []*physical.TxnEntry) error

Transaction writes all the operations in the provided transaction to the bolt file.

type FSMApplyResponse Uses

type FSMApplyResponse struct {
    Success bool
}

FSMApplyResponse is returned from an FSM apply. It indicates if the apply was successful or not.

type FSMChunkStorage Uses

type FSMChunkStorage struct {
    // contains filtered or unexported fields
}

func (*FSMChunkStorage) FinalizeOp Uses

func (f *FSMChunkStorage) FinalizeOp(opNum uint64) ([]*raftchunking.ChunkInfo, error)

func (*FSMChunkStorage) GetChunks Uses

func (f *FSMChunkStorage) GetChunks() (raftchunking.ChunkMap, error)

func (*FSMChunkStorage) RestoreChunks Uses

func (f *FSMChunkStorage) RestoreChunks(chunks raftchunking.ChunkMap) error

func (*FSMChunkStorage) StoreChunk Uses

func (f *FSMChunkStorage) StoreChunk(chunk *raftchunking.ChunkInfo) (bool, error)

type IndexValue Uses

type IndexValue struct {
    Term                 uint64   `protobuf:"varint,1,opt,name=term,proto3" json:"term,omitempty"`
    Index                uint64   `protobuf:"varint,2,opt,name=index,proto3" json:"index,omitempty"`
    XXX_NoUnkeyedLiteral struct{} `json:"-"`
    XXX_unrecognized     []byte   `json:"-"`
    XXX_sizecache        int32    `json:"-"`
}

func (*IndexValue) Descriptor Uses

func (*IndexValue) Descriptor() ([]byte, []int)

func (*IndexValue) GetIndex Uses

func (m *IndexValue) GetIndex() uint64

func (*IndexValue) GetTerm Uses

func (m *IndexValue) GetTerm() uint64

func (*IndexValue) ProtoMessage Uses

func (*IndexValue) ProtoMessage()

func (*IndexValue) Reset Uses

func (m *IndexValue) Reset()

func (*IndexValue) String Uses

func (m *IndexValue) String() string

func (*IndexValue) XXX_DiscardUnknown Uses

func (m *IndexValue) XXX_DiscardUnknown()

func (*IndexValue) XXX_Marshal Uses

func (m *IndexValue) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*IndexValue) XXX_Merge Uses

func (m *IndexValue) XXX_Merge(src proto.Message)

func (*IndexValue) XXX_Size Uses

func (m *IndexValue) XXX_Size() int

func (*IndexValue) XXX_Unmarshal Uses

func (m *IndexValue) XXX_Unmarshal(b []byte) error

type LogData Uses

type LogData struct {
    Operations           []*LogOperation `protobuf:"bytes,1,rep,name=operations,proto3" json:"operations,omitempty"`
    XXX_NoUnkeyedLiteral struct{}        `json:"-"`
    XXX_unrecognized     []byte          `json:"-"`
    XXX_sizecache        int32           `json:"-"`
}

func (*LogData) Descriptor Uses

func (*LogData) Descriptor() ([]byte, []int)

func (*LogData) GetOperations Uses

func (m *LogData) GetOperations() []*LogOperation

func (*LogData) ProtoMessage Uses

func (*LogData) ProtoMessage()

func (*LogData) Reset Uses

func (m *LogData) Reset()

func (*LogData) String Uses

func (m *LogData) String() string

func (*LogData) XXX_DiscardUnknown Uses

func (m *LogData) XXX_DiscardUnknown()

func (*LogData) XXX_Marshal Uses

func (m *LogData) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*LogData) XXX_Merge Uses

func (m *LogData) XXX_Merge(src proto.Message)

func (*LogData) XXX_Size Uses

func (m *LogData) XXX_Size() int

func (*LogData) XXX_Unmarshal Uses

func (m *LogData) XXX_Unmarshal(b []byte) error

type LogOperation Uses

type LogOperation struct {
    // OpType is the Operation type
    OpType uint32 `protobuf:"varint,1,opt,name=op_type,json=opType,proto3" json:"op_type,omitempty"`
    // Flags is an opaque value, currently unused. Reserved.
    Flags uint64 `protobuf:"varint,2,opt,name=flags,proto3" json:"flags,omitempty"`
    // Key that is being affected
    Key string `protobuf:"bytes,3,opt,name=key,proto3" json:"key,omitempty"`
    // Value is optional, corresponds to the key
    Value                []byte   `protobuf:"bytes,4,opt,name=value,proto3" json:"value,omitempty"`
    XXX_NoUnkeyedLiteral struct{} `json:"-"`
    XXX_unrecognized     []byte   `json:"-"`
    XXX_sizecache        int32    `json:"-"`
}

func (*LogOperation) Descriptor Uses

func (*LogOperation) Descriptor() ([]byte, []int)

func (*LogOperation) GetFlags Uses

func (m *LogOperation) GetFlags() uint64

func (*LogOperation) GetKey Uses

func (m *LogOperation) GetKey() string

func (*LogOperation) GetOpType Uses

func (m *LogOperation) GetOpType() uint32

func (*LogOperation) GetValue Uses

func (m *LogOperation) GetValue() []byte

func (*LogOperation) ProtoMessage Uses

func (*LogOperation) ProtoMessage()

func (*LogOperation) Reset Uses

func (m *LogOperation) Reset()

func (*LogOperation) String Uses

func (m *LogOperation) String() string

func (*LogOperation) XXX_DiscardUnknown Uses

func (m *LogOperation) XXX_DiscardUnknown()

func (*LogOperation) XXX_Marshal Uses

func (m *LogOperation) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*LogOperation) XXX_Merge Uses

func (m *LogOperation) XXX_Merge(src proto.Message)

func (*LogOperation) XXX_Size Uses

func (m *LogOperation) XXX_Size() int

func (*LogOperation) XXX_Unmarshal Uses

func (m *LogOperation) XXX_Unmarshal(b []byte) error

type Peer Uses

type Peer struct {
    ID      string `json:"id"`
    Address string `json:"address"`
}

Peer defines the ID and Address for a given member of the raft cluster.

type RaftBackend Uses

type RaftBackend struct {
    // contains filtered or unexported fields
}

RaftBackend implements the backend interfaces and uses the raft protocol to persist writes to the FSM.

func (*RaftBackend) AddNonVotingPeer Uses

func (b *RaftBackend) AddNonVotingPeer(ctx context.Context, peerID, clusterAddr string) error

AddPeer adds a new server to the raft cluster

func (*RaftBackend) AddPeer Uses

func (b *RaftBackend) AddPeer(ctx context.Context, peerID, clusterAddr string) error

AddPeer adds a new server to the raft cluster

func (*RaftBackend) AppliedIndex Uses

func (b *RaftBackend) AppliedIndex() uint64

AppliedIndex returns the latest index applied to the FSM

func (*RaftBackend) Bootstrap Uses

func (b *RaftBackend) Bootstrap(ctx context.Context, peers []Peer) error

Bootstrap prepares the given peers to be part of the raft cluster

func (*RaftBackend) Delete Uses

func (b *RaftBackend) Delete(ctx context.Context, path string) error

Delete inserts an entry in the log to delete the given path

func (*RaftBackend) Get Uses

func (b *RaftBackend) Get(ctx context.Context, path string) (*physical.Entry, error)

Get returns the value corresponding to the given path from the fsm

func (*RaftBackend) GetConfiguration Uses

func (b *RaftBackend) GetConfiguration(ctx context.Context) (*RaftConfigurationResponse, error)

func (*RaftBackend) HAEnabled Uses

func (b *RaftBackend) HAEnabled() bool

HAEnabled is the implemention of the HABackend interface

func (*RaftBackend) Initialized Uses

func (b *RaftBackend) Initialized() bool

Initialized tells if raft is running or not

func (*RaftBackend) List Uses

func (b *RaftBackend) List(ctx context.Context, prefix string) ([]string, error)

List enumerates all the items under the prefix from the fsm

func (*RaftBackend) LockWith Uses

func (b *RaftBackend) LockWith(key, value string) (physical.Lock, error)

HAEnabled is the implemention of the HABackend interface

func (*RaftBackend) NodeID Uses

func (b *RaftBackend) NodeID() string

NodeID returns the identifier of the node

func (*RaftBackend) Peers Uses

func (b *RaftBackend) Peers(ctx context.Context) ([]Peer, error)

Peers returns all the servers present in the raft cluster

func (*RaftBackend) Put Uses

func (b *RaftBackend) Put(ctx context.Context, entry *physical.Entry) error

Put inserts an entry in the log for the put operation

func (*RaftBackend) RemovePeer Uses

func (b *RaftBackend) RemovePeer(ctx context.Context, peerID string) error

RemovePeer removes the given peer ID from the raft cluster. If the node is ourselves we will give up leadership.

func (*RaftBackend) RestoreSnapshot Uses

func (b *RaftBackend) RestoreSnapshot(ctx context.Context, metadata raft.SnapshotMeta, snap io.Reader) error

RestoreSnapshot applies the provided snapshot metadata and snapshot data to raft.

func (*RaftBackend) SetRestoreCallback Uses

func (b *RaftBackend) SetRestoreCallback(restoreCb restoreCallback)

SetRestoreCallback sets the callback to be used when a restoreCallbackOp is processed through the FSM.

func (*RaftBackend) SetServerAddressProvider Uses

func (b *RaftBackend) SetServerAddressProvider(provider raft.ServerAddressProvider)

SetServerAddressProvider sets a the address provider for determining the raft node addresses. This is currently only used in tests.

func (*RaftBackend) SetTLSKeyring Uses

func (b *RaftBackend) SetTLSKeyring(keyring *TLSKeyring) error

SetTLSKeyring is used to install a new keyring. If the active key has changed it will also close any network connections or streams forcing a reconnect with the new key.

func (*RaftBackend) SetupCluster Uses

func (b *RaftBackend) SetupCluster(ctx context.Context, opts SetupOpts) error

SetupCluster starts the raft cluster and enables the networking needed for the raft nodes to communicate.

func (*RaftBackend) Snapshot Uses

func (b *RaftBackend) Snapshot(out *logical.HTTPResponseWriter, access seal.Access) error

Snapshot takes a raft snapshot, packages it into a archive file and writes it to the provided writer. Seal access is used to encrypt the SHASUM file so we can validate the snapshot was taken using the same master keys or not.

func (*RaftBackend) TeardownCluster Uses

func (b *RaftBackend) TeardownCluster(clusterListener cluster.ClusterHook) error

TeardownCluster shuts down the raft cluster

func (*RaftBackend) Transaction Uses

func (b *RaftBackend) Transaction(ctx context.Context, txns []*physical.TxnEntry) error

Transaction applies all the given operations into a single log and applies it.

func (*RaftBackend) WriteSnapshotToTemp Uses

func (b *RaftBackend) WriteSnapshotToTemp(in io.ReadCloser, access seal.Access) (*os.File, func(), raft.SnapshotMeta, error)

WriteSnapshotToTemp reads a snapshot archive off the provided reader, extracts the data and writes the snapshot to a temporary file. The seal access is used to decrypt the SHASUM file in the archive to ensure this snapshot has the same master key as the running instance. If the provided access is nil then it will skip that validation.

type RaftConfigurationResponse Uses

type RaftConfigurationResponse struct {
    // Servers has the list of servers in the Raft configuration.
    Servers []*RaftServer `json:"servers"`

    // Index has the Raft index of this configuration.
    Index uint64 `json:"index"`
}

RaftConfigurationResponse is returned when querying for the current Raft configuration.

type RaftLock Uses

type RaftLock struct {
    // contains filtered or unexported fields
}

RaftLock implements the physical Lock interface and enables HA for this backend. The Lock uses the raftNotifyCh for receiving leadership edge triggers. Vault's active duty matches raft's leadership.

func (*RaftLock) Lock Uses

func (l *RaftLock) Lock(stopCh <-chan struct{}) (<-chan struct{}, error)

Lock blocks until we become leader or are shutdown. It returns a channel that is closed when we detect a loss of leadership.

func (*RaftLock) Unlock Uses

func (l *RaftLock) Unlock() error

Unlock gives up leadership.

func (*RaftLock) Value Uses

func (l *RaftLock) Value() (bool, string, error)

Value reads the value of the lock. This informs us who is currently leader.

type RaftServer Uses

type RaftServer struct {
    // NodeID is the name of the server
    NodeID string `json:"node_id"`

    // Address is the IP:port of the server, used for Raft communications
    Address string `json:"address"`

    // Leader is true if this server is the current cluster leader
    Leader bool `json:"leader"`

    // Protocol version is the raft protocol version used by the server
    ProtocolVersion string `json:"protocol_version"`

    // Voter is true if this server has a vote in the cluster. This might
    // be false if the server is staging and still coming online.
    Voter bool `json:"voter"`
}

RaftServer has information about a server in the Raft configuration

type Server Uses

type Server struct {
    Suffrage             int32    `protobuf:"varint,1,opt,name=suffrage,proto3" json:"suffrage,omitempty"`
    Id                   string   `protobuf:"bytes,2,opt,name=id,proto3" json:"id,omitempty"`
    Address              string   `protobuf:"bytes,3,opt,name=address,proto3" json:"address,omitempty"`
    XXX_NoUnkeyedLiteral struct{} `json:"-"`
    XXX_unrecognized     []byte   `json:"-"`
    XXX_sizecache        int32    `json:"-"`
}

func (*Server) Descriptor Uses

func (*Server) Descriptor() ([]byte, []int)

func (*Server) GetAddress Uses

func (m *Server) GetAddress() string

func (*Server) GetId Uses

func (m *Server) GetId() string

func (*Server) GetSuffrage Uses

func (m *Server) GetSuffrage() int32

func (*Server) ProtoMessage Uses

func (*Server) ProtoMessage()

func (*Server) Reset Uses

func (m *Server) Reset()

func (*Server) String Uses

func (m *Server) String() string

func (*Server) XXX_DiscardUnknown Uses

func (m *Server) XXX_DiscardUnknown()

func (*Server) XXX_Marshal Uses

func (m *Server) XXX_Marshal(b []byte, deterministic bool) ([]byte, error)

func (*Server) XXX_Merge Uses

func (m *Server) XXX_Merge(src proto.Message)

func (*Server) XXX_Size Uses

func (m *Server) XXX_Size() int

func (*Server) XXX_Unmarshal Uses

func (m *Server) XXX_Unmarshal(b []byte) error

type SetupOpts Uses

type SetupOpts struct {
    // TLSKeyring is the keyring to use for the cluster traffic.
    TLSKeyring *TLSKeyring

    // ClusterListener is the cluster hook used to register the raft handler and
    // client with core's cluster listeners.
    ClusterListener cluster.ClusterHook

    // StartAsLeader is used to specify this node should start as leader and
    // bypass the leader election. This should be used with caution.
    StartAsLeader bool
}

SetupOpts are used to pass options to the raft setup function.

type TLSKey Uses

type TLSKey struct {
    // ID is a unique identifier for this Key
    ID  string `json:"id"`

    // KeyType defines the algorighm used to generate the private keys
    KeyType string `json:"key_type"`

    // AppliedIndex is the earliest known raft index that safely contains this
    // key.
    AppliedIndex uint64 `json:"applied_index"`

    // CertBytes is the marshaled certificate.
    CertBytes []byte `json:"cluster_cert"`

    // KeyParams is the marshaled private key.
    KeyParams *certutil.ClusterKeyParams `json:"cluster_key_params"`

    // CreatedTime is the time this key was generated. This value is useful in
    // determining when the next rotation should be.
    CreatedTime time.Time `json:"created_time"`
    // contains filtered or unexported fields
}

TLSKey is a single TLS keypair in the Keyring

func GenerateTLSKey Uses

func GenerateTLSKey() (*TLSKey, error)

type TLSKeyring Uses

type TLSKeyring struct {
    // Keys is the set of available key pairs
    Keys []*TLSKey `json:"keys"`

    // AppliedIndex is the earliest known raft index that safely contains the
    // latest key in the keyring.
    AppliedIndex uint64 `json:"applied_index"`

    // Term is an incrementing identifier value used to quickly determine if two
    // states of the keyring are different.
    Term uint64 `json:"term"`

    // ActiveKeyID is the key ID to track the active key in the keyring. Only
    // the active key is used for dialing.
    ActiveKeyID string `json:"active_key_id"`
}

TLSKeyring is the set of keys that raft uses for network communication. Only one key is used to dial at a time but both keys will be used to accept connections.

func (*TLSKeyring) GetActive Uses

func (k *TLSKeyring) GetActive() *TLSKey

GetActive returns the active key.

Directories

PathSynopsis
logstore

Package raft imports 43 packages (graph) and is imported by 5 packages. Updated 2019-10-13. Refresh now. Tools for package owners.