cluster

package
v0.0.0-...-8d377ce Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 29, 2024 License: Apache-2.0 Imports: 42 Imported by: 0

Documentation

Overview

cluster builds on the launch package and implements launching Metropolis nodes and clusters in a virtualized environment using qemu. It's kept in a separate package as it depends on a Metropolis node image, which might not be required for some use of the launch library.

Index

Constants

View Source
const SOCKSPort uint16 = 1080

Variables

View Source
var (
	// InsecurePrivateKey is a ED25519 key that can be used during development as
	// a fixed Owner Key when bootstrapping new clusters.
	InsecurePrivateKey = ed25519.NewKeyFromSeed([]byte(
		"\xb5\xcf\x35\x0f\xbf\xfb\xea\xfa\xa0\xf0\x29\x9d\xfa\xf7\xca\x6f" +
			"\xa2\xc2\xc7\x87\xd7\x03\x3e\xb2\x11\x4f\x36\xe0\x22\x73\x4f\x87"))
	// InsecurePublicKey is the ED25519 public key corresponding to
	// InsecurePrivateKey.
	InsecurePublicKey = InsecurePrivateKey.Public().(ed25519.PublicKey)
	// InsecureClusterBootstrap is a ClusterBootstrap message to be used within
	// NodeParameters when bootstrapping a development cluster that should be owned
	// by the InsecurePrivateKey.
	InsecureClusterBootstrap = &apb.NodeParameters_ClusterBootstrap{
		OwnerPublicKey: InsecurePublicKey,
	}
)

ClusterPorts contains all ports handled by Nanoswitch.

NodePorts is the list of ports a fully operational Metropolis node listens on

Functions

func LaunchNode

func LaunchNode(ctx context.Context, ld, sd string, options *NodeOptions) error

LaunchNode launches a single Metropolis node instance with the given options. The instance runs mostly paravirtualized but with some emulated hardware similar to how a cloud provider might set up its VMs. The disk is fully writable, and the changes are kept across reboots and shutdowns. ld and sd point to the launch directory and the socket directory, holding the nodes' state files (storage, tpm state, firmware state), and UNIX socket files (swtpm <-> QEMU interplay) respectively. The directories must exist before LaunchNode is called. LaunchNode will update options.Runtime and options.Mac if either are not initialized.

func MetroctlRunfilePath

func MetroctlRunfilePath() (string, error)

MetroctlRunfilePath returns the absolute path to the metroctl binary available if the built target depends on //metropolis/cli/metroctl. Otherwise, an error is returned.

func NewSerialFileLogger

func NewSerialFileLogger(p string) (io.ReadWriter, error)

Types

type Cluster

type Cluster struct {
	// Owner is the TLS Certificate of the owner of the test cluster. This can be
	// used to authenticate further clients to the running cluster.
	Owner tls.Certificate
	// Ports is the PortMap used to access the first nodes' services (defined in
	// ClusterPorts) and the SOCKS proxy (at SOCKSPort).
	Ports launch.PortMap

	// Nodes is a map from Node ID to its runtime information.
	Nodes map[string]*NodeInCluster
	// NodeIDs is a list of node IDs that are backing this cluster, in order of
	// creation.
	NodeIDs []string

	// CACertificate is the cluster's CA certificate.
	CACertificate *x509.Certificate

	// SOCKSDialer is used by DialNode to establish connections to nodes via the
	// SOCKS server ran by nanoswitch.
	SOCKSDialer proxy.Dialer
	// contains filtered or unexported fields
}

Cluster is the running Metropolis cluster launched using the LaunchCluster function.

func LaunchCluster

func LaunchCluster(ctx context.Context, opts ClusterOptions) (*Cluster, error)

LaunchCluster launches a cluster of Metropolis node VMs together with a Nanoswitch instance to network them all together.

The given context will be used to run all qemu instances in the cluster, and canceling the context or calling Close() will terminate them.

func (*Cluster) AllNodesHealthy

func (c *Cluster) AllNodesHealthy(ctx context.Context) error

AllNodesHealthy returns nil if all the nodes in the cluster are seemingly healthy.

func (*Cluster) ApproveNode

func (c *Cluster) ApproveNode(ctx context.Context, id string) error

ApproveNode approves a node by ID, waiting for it to become UP.

func (*Cluster) Close

func (c *Cluster) Close() error

Close cancels the running clusters' context and waits for all virtualized nodes to stop. It returns an error if stopping the nodes failed, or one of the nodes failed to fully start in the first place.

func (*Cluster) ConnectOptions

func (c *Cluster) ConnectOptions() *metroctl.ConnectOptions

ConnectOptions returns metroctl.ConnectOptions that describe connectivity to the launched cluster.

func (*Cluster) CuratorClient

func (c *Cluster) CuratorClient() (*grpc.ClientConn, error)

CuratorClient returns an authenticated owner connection to a Curator instance within Cluster c, or nil together with an error.

func (*Cluster) DialNode

func (c *Cluster) DialNode(_ context.Context, addr string) (net.Conn, error)

DialNode is a grpc.WithContextDialer compatible dialer which dials nodes by their ID. This is performed by connecting to the cluster nanoswitch via its SOCKS proxy, and using the cluster node list for name resolution.

For example:

grpc.Dial("metropolis-deadbeef:1234", grpc.WithContextDialer(c.DialNode))

func (*Cluster) GetKubeClientSet

func (c *Cluster) GetKubeClientSet() (kubernetes.Interface, error)

GetKubeClientSet gets a Kubernetes client set accessing the Metropolis Kubernetes authenticating proxy using the cluster owner identity. It currently has access to everything (i.e. the cluster-admin role) via the owner-admin binding.

func (*Cluster) KubernetesControllerNodeAddresses

func (c *Cluster) KubernetesControllerNodeAddresses(ctx context.Context) ([]string, error)

KubernetesControllerNodeAddresses returns the list of IP addresses of nodes which are currently Kubernetes controllers, ie. run an apiserver. This list might be empty if no node is currently configured with the 'KubernetesController' node.

func (*Cluster) MakeKubernetesWorker

func (c *Cluster) MakeKubernetesWorker(ctx context.Context, id string) error

MakeKubernetesWorker adds the KubernetesWorker role to a node by ID.

func (*Cluster) MakeMetroctlWrapper

func (c *Cluster) MakeMetroctlWrapper() (string, error)

MakeMetroctlWrapper builds and returns the path to a shell script which calls metroctl (from //metropolis/cli/metroctl, which must be included as a data dependency of the built target) with all the required flags to connect to the launched cluster.

func (*Cluster) MetroctlFlags

func (c *Cluster) MetroctlFlags() string

MetroctlFlags return stringified flags to pass to a metroctl binary to connect to the launched cluster.

func (*Cluster) RebootNode

func (c *Cluster) RebootNode(ctx context.Context, idx int) error

RebootNode reboots the cluster member node matching the given index, and waits for it to rejoin the cluster. It will use the given context ctx to run cluster API requests, whereas the resulting QEMU process will be created using the cluster's context c.ctxT. The nodes are indexed starting at 0.

type ClusterOptions

type ClusterOptions struct {
	// The number of nodes this cluster should be started with.
	NumNodes int

	// If true, node logs will be saved to individual files instead of being printed
	// out to stderr. The path of these files will be still printed to stdout.
	//
	// The files will be located within the launch directory inside TEST_TMPDIR (or
	// the default tempdir location, if not set).
	NodeLogsToFiles bool

	// LeaveNodesNew, if set, will leave all non-bootstrap nodes in NEW, without
	// bootstrapping them. The nodes' address information in Cluster.Nodes will be
	// incomplete.
	LeaveNodesNew bool

	// Optional local registry which will be made available to the cluster to
	// pull images from. This is a more efficient alternative to preseeding all
	// images used for testing.
	LocalRegistry *localregistry.Server
}

ClusterOptions contains all options for launching a Metropolis cluster.

type NodeInCluster

type NodeInCluster struct {
	// ID of the node, which can be used to dial this node's services via DialNode.
	ID     string
	Pubkey []byte
	// Address of the node on the network ran by nanoswitch. Not reachable from the
	// host unless dialed via DialNode or via the nanoswitch SOCKS proxy (reachable
	// on Cluster.Ports[SOCKSPort]).
	ManagementAddress string
}

NodeInCluster represents information about a node that's part of a Cluster.

type NodeOptions

type NodeOptions struct {
	// Name is a human-readable identifier to be used in debug output.
	Name string

	// Ports contains the port mapping where to expose the internal ports of the VM to
	// the host. See IdentityPortMap() and ConflictFreePortMap(). Ignored when
	// ConnectToSocket is set.
	Ports launch.PortMap

	// If set to true, reboots are honored. Otherwise, all reboots exit the Launch()
	// command. Metropolis nodes generally restart on almost all errors, so unless you
	// want to test reboot behavior this should be false.
	AllowReboot bool

	// By default, the VM is connected to the Host via SLIRP. If ConnectToSocket is
	// set, it is instead connected to the given file descriptor/socket. If this is
	// set, all port maps from the Ports option are ignored. Intended for networking
	// this instance together with others for running more complex network
	// configurations.
	ConnectToSocket *os.File

	// When PcapDump is set, all traffic is dumped to a pcap file in the
	// runtime directory (e.g. "net0.pcap" for the first interface).
	PcapDump bool

	// SerialPort is an io.ReadWriter over which you can communicate with the serial
	// port of the machine. It can be set to an existing file descriptor (like
	// os.Stdout/os.Stderr) or any Go structure implementing this interface.
	SerialPort io.ReadWriter

	// NodeParameters is passed into the VM and subsequently used for bootstrapping or
	// registering into a cluster.
	NodeParameters *apb.NodeParameters

	// Mac is the node's MAC address.
	Mac *net.HardwareAddr

	// Runtime keeps the node's QEMU runtime state.
	Runtime *NodeRuntime
}

NodeOptions contains all options that can be passed to Launch()

type NodeRuntime

type NodeRuntime struct {

	// CtxC is the QEMU context's cancellation function.
	CtxC context.CancelFunc
	// contains filtered or unexported fields
}

NodeRuntime keeps the node's QEMU runtime options.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL