storage

package
v0.17.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2023 License: Apache-2.0 Imports: 16 Imported by: 2

Documentation

Overview

Package storage defines the interfaces for the storage provider.

Index

Constants

View Source
const (
	// DefaultMeshDomain is the default domain for the mesh network.
	DefaultMeshDomain = "webmesh.internal"
	// DefaultIPv4Network is the default IPv4 network for the mesh.
	DefaultIPv4Network = "172.16.0.0/12"
	// DefaultNetworkPolicy is the default network policy for the mesh.
	DefaultNetworkPolicy = "accept"
	// DefaultBootstrapListenAddress is the default listen address for the bootstrap transport.
	DefaultBootstrapListenAddress = "[::]:9001"
	// DefaultBootstrapAdvertiseAddress is the default advertise address for the bootstrap transport.
	DefaultBootstrapAdvertiseAddress = "127.0.0.1:9001"
	// DefaultBootstrapPort is the default port for the bootstrap transport.
	DefaultBootstrapPort = 9001
	// DefaultMeshAdmin is the default mesh admin node ID.
	DefaultMeshAdmin = "admin"
)

Variables

View Source
var (
	// BootstrapNodesNetworkACLName is the name of the bootstrap nodes NetworkACL.
	BootstrapNodesNetworkACLName = []byte("bootstrap-nodes")
	// NetworkACLsPrefix is where NetworkACLs are stored in the database.
	NetworkACLsPrefix = types.RegistryPrefix.For([]byte("network-acls"))
	// RoutesPrefix is where Routes are stored in the database.
	RoutesPrefix = types.RegistryPrefix.For([]byte("routes"))
)
View Source
var (
	// MeshAdminRole is the name of the mesh admin role.
	MeshAdminRole = []byte("mesh-admin")
	// MeshAdminRoleBinding is the name of the mesh admin rolebinding.
	MeshAdminRoleBinding = []byte("mesh-admin")
	// VotersRole is the name of the voters role.
	VotersRole = []byte("voters")
	// VotersGroup is the name of the voters group.
	VotersGroup = []byte("voters")
	// BootstrapVotersRoleBinding is the name of the bootstrap voters rolebinding.
	BootstrapVotersRoleBinding = []byte("bootstrap-voters")
)
View Source
var EdgesPrefix = types.RegistryPrefix.ForString("edges")

EdgesPrefix is where edges are stored in the database. edges are indexed by their source and target node IDs in the format /registry/edges/<source>/<target>.

View Source
var ErrStopIteration = fmt.Errorf("stop iteration")

ErrStopIteration is a special error that can be returned by PrefixIterator to stop iteration.

View Source
var NodesPrefix = types.RegistryPrefix.ForString("nodes")

NodesPrefix is where nodes are stored in the database. nodes are indexed by their ID in the format /registry/nodes/<id>.

Functions

func ExpandACL added in v0.9.0

func ExpandACL(ctx context.Context, rbac RBAC, acl types.NetworkACL) error

ExpandACL will use the given RBAC interface to expand any group references in the ACL.

func ExpandACLs added in v0.9.0

func ExpandACLs(ctx context.Context, rbac RBAC, acls types.NetworkACLs) error

ExpandACLs will use the given RBAC interface to expand any group references in the ACLs.

func IsSystemGroup added in v0.9.0

func IsSystemGroup(name string) bool

IsSystemGroup returns true if the group is a system group.

func IsSystemRole added in v0.9.0

func IsSystemRole(name string) bool

IsSystemRole returns true if the role is a system role.

func IsSystemRoleBinding added in v0.9.0

func IsSystemRoleBinding(name string) bool

IsSystemRoleBinding returns true if the rolebinding is a system rolebinding.

func NewGraphWithStore added in v0.11.0

func NewGraphWithStore(store GraphStore) types.PeerGraph

NewGraphWithStore creates a new Graph instance with the given graph storage implementation.

Types

type BootstrapOptions added in v0.10.1

type BootstrapOptions struct {
	// MeshDomain is the mesh domain.
	MeshDomain string
	// IPv4Network is the IPv4 prefix.
	IPv4Network string
	// IPv6Network is the IPv6 prefix. If left unset,
	// one will be generated.
	IPv6Network string
	// Admin is the admin node ID.
	Admin string
	// DefaultNetworkPolicy is the default network policy.
	DefaultNetworkPolicy string
	// BootstrapNodes are the bootstrap nodes to use.
	BootstrapNodes []string
	// Voters are additional voting nodes to add to the voters group.
	Voters []string
	// DisableRBAC disables RBAC.
	DisableRBAC bool
}

BootstrapOptions are options for bootstrapping the database.

func (*BootstrapOptions) Default added in v0.14.14

func (b *BootstrapOptions) Default()

type BootstrapResults added in v0.10.1

type BootstrapResults struct {
	// NetworkV4 is the IPv4 network.
	NetworkV4 netip.Prefix
	// NetworkV6 is the IPv6 network.
	NetworkV6 netip.Prefix
	// MeshDomain is the mesh domain.
	MeshDomain string
}

BoostrapResults are the results of bootstrapping the database.

func Bootstrap added in v0.10.1

func Bootstrap(ctx context.Context, db MeshDB, opts *BootstrapOptions) (results BootstrapResults, err error)

Bootstrap attempts to bootstrap the given database. If data already exists, ErrAlreadyBootstrapped will be returned, but with results populated with the existing data.

type Consensus added in v0.7.0

type Consensus interface {
	// IsLeader returns true if the node is the leader of the storage group.
	IsLeader() bool
	// IsMember returns true if the node is a member of the storage group.
	IsMember() bool
	// StepDown should be called to relinquish leadership of the storage group.
	StepDown(context.Context) error
	// GetPeer returns the peer with the given ID.
	GetPeer(context.Context, string) (types.StoragePeer, error)
	// GetPeers returns the peers of the storage group.
	GetPeers(context.Context) ([]types.StoragePeer, error)
	// GetLeader returns the leader of the storage group.
	GetLeader(context.Context) (types.StoragePeer, error)
	// AddVoter adds a voter to the consensus group.
	AddVoter(context.Context, types.StoragePeer) error
	// AddObserver adds an observer to the consensus group.
	AddObserver(context.Context, types.StoragePeer) error
	// DemoteVoter demotes a voter to an observer.
	DemoteVoter(context.Context, types.StoragePeer) error
	// RemovePeer removes a peer from the consensus group. If wait
	// is true, the function will wait for the peer to be removed.
	RemovePeer(ctx context.Context, peer types.StoragePeer, wait bool) error
}

Consensus is the interface for managing storage consensus.

type ConsensusStorage added in v0.7.2

type ConsensusStorage interface {
	io.Closer
	raft.LogStore
	raft.StableStore

	// Snapshot returns a snapshot of the storage.
	Snapshot(ctx context.Context) (io.Reader, error)
	// Restore restores a snapshot of the storage.
	Restore(ctx context.Context, r io.Reader) error
}

ConsensusStorage is the interface for storing and retrieving data about the state of consensus. This is currently only used by the built-in raftstorage implementation.

type DualStorage added in v0.3.0

type DualStorage interface {
	MeshStorage
	ConsensusStorage
}

DualStorage represents a storage interface that can serve as both mesh and consensus storage.

type GraphStore added in v0.11.0

type GraphStore interface {
	// PeerGraphStore is a storage interface for peer graph data.
	types.PeerGraphStore

	// Subscribe subscribes to changes to nodes and edges.
	Subscribe(ctx context.Context, fn PeerSubscribeFunc) (context.CancelFunc, error)
}

GraphStore is a storage interface for graph data.

type KVSubscribeFunc added in v0.9.0

type KVSubscribeFunc func(key, value []byte)

KVSubscribeFunc is the function signature for subscribing to changes to a key.

type MeshDB added in v0.9.0

type MeshDB interface {
	// MeshDataStore is the underlying MeshDataStore instance.
	MeshDataStore
	// Peers returns a simplified interface for managing nodes in the mesh
	// via the underlying MeshDataStore.
	Peers() Peers
}

MeshDB is the interface for the mesh database. It provides access to all storage interfaces.

type MeshDataStore added in v0.11.0

type MeshDataStore interface {
	// GraphStore returns the interface for managing network topology and data
	// about peers.
	GraphStore() GraphStore
	// RBAC returns the interface for managing RBAC policies in the mesh.
	RBAC() RBAC
	// MeshState returns the interface for querying mesh state.
	MeshState() MeshState
	// Networking returns the interface for managing networking in the mesh.
	Networking() Networking
}

MeshDataStore is an interface for storing and retrieving data about the state of the mesh. It can be implemented by external providers to be wrapped into a MeshDB for use throughout the library.

type MeshState added in v0.9.0

type MeshState interface {
	// SetMeshState sets the full mesh state.
	SetMeshState(ctx context.Context, state types.NetworkState) error
	// GetMeshState returns the full mesh state.
	GetMeshState(ctx context.Context) (types.NetworkState, error)
}

MeshState is the interface for querying mesh state.

type MeshStorage added in v0.3.0

type MeshStorage interface {
	// Close should close the underlying storage as well as any other resources
	// that the provider may have allocated. This should be called automatically
	// by the provider.
	io.Closer

	// GetValue returns the value of a key.
	GetValue(ctx context.Context, key []byte) ([]byte, error)
	// PutValue sets the value of a key. TTL is optional and can be set to 0.
	PutValue(ctx context.Context, key, value []byte, ttl time.Duration) error
	// Delete removes a key.
	Delete(ctx context.Context, key []byte) error
	// ListKeys returns all keys with a given prefix.
	ListKeys(ctx context.Context, prefix []byte) ([][]byte, error)
	// IterPrefix iterates over all keys with a given prefix. It is important
	// that the iterator not attempt any write operations as this will cause
	// a deadlock. The iteration will stop if the iterator returns an error.
	IterPrefix(ctx context.Context, prefix []byte, fn PrefixIterator) error
	// Subscribe will call the given function whenever a key with the given prefix is changed.
	// The returned function can be called to unsubscribe.
	Subscribe(ctx context.Context, prefix []byte, fn KVSubscribeFunc) (context.CancelFunc, error)
}

MeshStorage is the interface for storing and retrieving data about the state of the mesh.

type Networking added in v0.9.0

type Networking interface {
	// PutNetworkACL creates or updates a NetworkACL.
	PutNetworkACL(ctx context.Context, acl types.NetworkACL) error
	// GetNetworkACL returns a NetworkACL by name.
	GetNetworkACL(ctx context.Context, name string) (types.NetworkACL, error)
	// DeleteNetworkACL deletes a NetworkACL by name.
	DeleteNetworkACL(ctx context.Context, name string) error
	// ListNetworkACLs returns a list of NetworkACLs.
	ListNetworkACLs(ctx context.Context) (types.NetworkACLs, error)
	// PutRoute creates or updates a Route.
	PutRoute(ctx context.Context, route types.Route) error
	// GetRoute returns a Route by name.
	GetRoute(ctx context.Context, name string) (types.Route, error)
	// GetRoutesByNode returns a list of Routes for a given Node.
	GetRoutesByNode(ctx context.Context, nodeID types.NodeID) (types.Routes, error)
	// GetRoutesByCIDR returns a list of Routes for a given CIDR.
	GetRoutesByCIDR(ctx context.Context, cidr netip.Prefix) (types.Routes, error)
	// DeleteRoute deletes a Route by name.
	DeleteRoute(ctx context.Context, name string) error
	// ListRoutes returns a list of Routes.
	ListRoutes(ctx context.Context) (types.Routes, error)
}

Networking is the interface to the database models for network resources.

type PeerFilter added in v0.9.0

type PeerFilter func(types.MeshNode) bool

PeerFilter is a filter for nodes.

func FilterAgainstNode added in v0.10.1

func FilterAgainstNode(nodeID types.NodeID) PeerFilter

FilterAgainstNode returns a new filter that matches nodes that are not a given node ID.

func FilterByFeature added in v0.10.1

func FilterByFeature(feature v1.Feature) PeerFilter

FilterByFeature returns a new filter that matches nodes with a given feature.

func FilterByIPv4Prefix added in v0.13.7

func FilterByIPv4Prefix(prefix netip.Prefix) PeerFilter

FilterByIPv4Prefix returns a new filter that matches nodes whose private IPv4 address is in a given prefix.

func FilterByIPv6Prefix added in v0.13.7

func FilterByIPv6Prefix(prefix netip.Prefix) PeerFilter

FilterByIPv6Prefix returns a new filter that matches nodes whose private IPv6 address is in a given prefix.

func FilterByIsPublic added in v0.10.1

func FilterByIsPublic() PeerFilter

FilterByIsPublic returns a new filter that matches public nodes.

func FilterByZoneID added in v0.10.1

func FilterByZoneID(zoneID string) PeerFilter

FilterByZoneID returns a new filter that matches nodes in a given zone.

type PeerFilters added in v0.9.0

type PeerFilters []PeerFilter

PeerFilters is a list of filters.

func (PeerFilters) Filter added in v0.9.0

func (f PeerFilters) Filter(nodes []types.MeshNode) []types.MeshNode

Filter filters a list of nodes.

func (PeerFilters) Match added in v0.9.0

func (f PeerFilters) Match(node types.MeshNode) bool

Match returns true if the node matches all filters.

type PeerSubscribeFunc added in v0.9.0

type PeerSubscribeFunc func([]types.MeshNode)

PeerSubscribeFunc is a function that can be used to subscribe to peer changes. The function is called with multiple peers when the change reflects a new edge being added or removed. The function is called with a single peer when the change reflects a node being added or removed.

type Peers added in v0.9.0

type Peers interface {
	// Graph should return the underlying graph instance.
	Graph() types.PeerGraph
	// Put creates or updates a node.
	Put(ctx context.Context, n types.MeshNode) error
	// Get gets a node by ID.
	Get(ctx context.Context, id types.NodeID) (types.MeshNode, error)
	// GetByPubKey gets a node by their public key.
	GetByPubKey(ctx context.Context, key crypto.PublicKey) (types.MeshNode, error)
	// Delete deletes a node.
	Delete(ctx context.Context, id types.NodeID) error
	// List lists all nodes.
	List(ctx context.Context, filters ...PeerFilter) ([]types.MeshNode, error)
	// ListIDs lists all node IDs.
	ListIDs(ctx context.Context) ([]types.NodeID, error)
	// Subscribe subscribes to node changes.
	Subscribe(ctx context.Context, fn PeerSubscribeFunc) (context.CancelFunc, error)
	// AddEdge adds an edge between two nodes.
	PutEdge(ctx context.Context, edge types.MeshEdge) error
	// GetEdge gets an edge between two nodes.
	GetEdge(ctx context.Context, from, to types.NodeID) (types.MeshEdge, error)
	// RemoveEdge removes an edge between two nodes.
	RemoveEdge(ctx context.Context, from, to types.NodeID) error
}

Peers is the peers interface.

type PrefixIterator

type PrefixIterator func(key, value []byte) error

PrefixIterator is the function signature for iterating over all keys with a given prefix.

type Provider added in v0.7.0

type Provider interface {
	// Close should close the underlying storage as well as any other resources
	// that the provider may have allocated.
	io.Closer
	// Start should start the provider and any resources that it may need.
	Start(context.Context) error
	// Bootstrap should bootstrap the provider for first-time usage.
	Bootstrap(context.Context) error
	// Status returns the status of the storage provider. It should never error.
	// If inaccurate status is available, the node should return itself as a peer
	// with a message describing the inaccuracy.
	Status() *v1.StorageStatus
	// ListenPort should return the TCP port that the storage provider is listening on.
	ListenPort() uint16
	// MeshDB returns the underlying MeshDB instance. The provider does not
	// need to guarantee consistency on read operations.
	MeshDB() MeshDB
	// Consensus returns the underlying Consensus instance for managing voting/observing
	// nodes and leader election.
	Consensus() Consensus
	// MeshStorage returns the underlying raw MeshStorage instance. The provider does
	// not need to guarantee consistency on read operations. This should only be used
	// for arbitrary key/value storage that has not been abstracted behind the MeshDB.
	MeshStorage() MeshStorage
}

Provider is a provider of MeshStorage.

type RBAC added in v0.9.0

type RBAC interface {
	// SetEnabled sets the RBAC enabled state.
	SetEnabled(ctx context.Context, enabled bool) error
	// GetEnabled returns the RBAC enabled state.
	GetEnabled(ctx context.Context) (bool, error)

	// PutRole creates or updates a role.
	PutRole(ctx context.Context, role types.Role) error
	// GetRole returns a role by name.
	GetRole(ctx context.Context, name string) (types.Role, error)
	// DeleteRole deletes a role by name.
	DeleteRole(ctx context.Context, name string) error
	// ListRoles returns a list of all roles.
	ListRoles(ctx context.Context) (types.RolesList, error)

	// PutRoleBinding creates or updates a rolebinding.
	PutRoleBinding(ctx context.Context, rolebinding types.RoleBinding) error
	// GetRoleBinding returns a rolebinding by name.
	GetRoleBinding(ctx context.Context, name string) (types.RoleBinding, error)
	// DeleteRoleBinding deletes a rolebinding by name.
	DeleteRoleBinding(ctx context.Context, name string) error
	// ListRoleBindings returns a list of all rolebindings.
	ListRoleBindings(ctx context.Context) ([]types.RoleBinding, error)

	// PutGroup creates or updates a group.
	PutGroup(ctx context.Context, group types.Group) error
	// GetGroup returns a group by name.
	GetGroup(ctx context.Context, name string) (types.Group, error)
	// DeleteGroup deletes a group by name.
	DeleteGroup(ctx context.Context, name string) error
	// ListGroups returns a list of all groups.
	ListGroups(ctx context.Context) ([]types.Group, error)

	// ListNodeRoles returns a list of all roles for a node.
	ListNodeRoles(ctx context.Context, nodeID types.NodeID) (types.RolesList, error)
	// ListUserRoles returns a list of all roles for a user.
	ListUserRoles(ctx context.Context, user types.NodeID) (types.RolesList, error)
}

RBAC is the interface to the database models for RBAC.

Directories

Path Synopsis
Package errors contains error definitions for storage providers.
Package errors contains error definitions for storage providers.
Package meshdb implements a storage.Database using any storage.MeshStorage instance.
Package meshdb implements a storage.Database using any storage.MeshStorage instance.
graphstore
Package graph implements a graph data structure for the mesh network.
Package graph implements a graph data structure for the mesh network.
networking
Package networking contains interfaces to the database models for Network ACLs and Routes.
Package networking contains interfaces to the database models for Network ACLs and Routes.
rbac
Package rbac contains interfaces to the database models for RBAC.
Package rbac contains interfaces to the database models for RBAC.
state
Package state provides an interface for querying mesh state.
Package state provides an interface for querying mesh state.
Package providers contains the built-in storage providers.
Package providers contains the built-in storage providers.
backends/badgerdb
Package badgerdb implements the storage backends using BadgerDB.
Package badgerdb implements the storage backends using BadgerDB.
external
Package external provides a storage provider that uses a storage plugin to manage mesh storage and consensus.
Package external provides a storage provider that uses a storage plugin to manage mesh storage and consensus.
passthrough
Package passthrough provides a passthrough storage provider.
Package passthrough provides a passthrough storage provider.
raftstorage
Package raftstorage implements a Raft-backed storage provider.
Package raftstorage implements a Raft-backed storage provider.
raftstorage/fsm
Package fsm implements the Raft FSM.
Package fsm implements the Raft FSM.
raftstorage/raftlogs
Package raftlogs provides facilities for applying raft logs to a database.
Package raftlogs provides facilities for applying raft logs to a database.
raftstorage/snapshots
Package snapshots provides an interface for managing raft snapshots.
Package snapshots provides an interface for managing raft snapshots.
Package rpcdb provides a meshdb that operates over RPC.
Package rpcdb provides a meshdb that operates over RPC.
Package rpcsrv contains utilities for serving mesh databases over RPC.
Package rpcsrv contains utilities for serving mesh databases over RPC.
Package testutil contains testing utilities for storage providers and backends.
Package testutil contains testing utilities for storage providers and backends.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL