Documentation ¶
Overview ¶
Package cluster represents the cluster components of Sif
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Node ¶
type Node interface { IsCoordinator() bool Start(sif.DataFrame) error GracefulStop() error Stop() error Run(ctx context.Context) (*Result, error) }
Node is a member of a Sif cluster, either coordinating or performing work. Nodes present several methods to control their lifecycle.
func CreateNode ¶
func CreateNode(opts *NodeOptions) (Node, error)
CreateNode creates a Sif node, deriving role from environment variables
func CreateNodeInRole ¶
func CreateNodeInRole(role NodeRole, opts *NodeOptions) (Node, error)
CreateNodeInRole creates a Sif node in a specific role (Coordinator or Worker)
type NodeOptions ¶
type NodeOptions struct { Port int // port for this Node to bind to Host string // hostname for this Node to bind to CoordinatorPort int // port for the Coordinator Node (potentially identical to Port if this is the Coordinator) CoordinatorHost string // [REQUIRED] hostname of the Coordinator Node (potentially identical to Host if this is the Coordinator) NumWorkers int // [REQUIRED] the number of Workers to wait for before running the job WorkerJoinTimeout time.Duration // how long the Coordinator should wait for Workers to join WorkerJoinRetries int // how many times a Worker should retry connecting to the Coordinator (at one second intervals) RPCTimeout time.Duration // timeout for all RPC calls TempDir string // location for storing temporary files (primarily persisted partitions) CacheMemoryHighWatermark uint64 // the number of partitions to retain in memory before swapping to disk IgnoreRowErrors bool // iff true, log row transformation errors instead of crashing immediately }
NodeOptions are options for a Node, configuring elements of a Sif cluster
func CloneNodeOptions ¶
func CloneNodeOptions(opts *NodeOptions) *NodeOptions
CloneNodeOptions makes a copy of a NodeOptions
type Result ¶
type Result struct { Accumulated sif.Accumulator Collected map[string]sif.CollectedPartition }
Result represents the result of a job, which may either be nil, an Accumulator or a set of CollectedPartitions
Click to show internal directories.
Click to hide internal directories.