distributed

package
v0.0.0-...-fecdc3c Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 5, 2021 License: Apache-2.0 Imports: 7 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type DistributedOption

type DistributedOption struct {
	RequiredFiles []resource.FileResource
	Master        string
	DataCenter    string
	Rack          string
	TaskMemoryMB  int
	FlowBid       float64
	Module        string
	IsProfiling   bool
}

func Option

func Option() *DistributedOption

func (*DistributedOption) GetFlowRunner

func (o *DistributedOption) GetFlowRunner() flow.FlowRunner

func (*DistributedOption) SetDataCenter

func (o *DistributedOption) SetDataCenter(dataCenter string) *DistributedOption

func (*DistributedOption) SetMaster

func (o *DistributedOption) SetMaster(master string) *DistributedOption

func (*DistributedOption) SetProfiling

func (o *DistributedOption) SetProfiling(isProfiling bool) *DistributedOption

SetProfiling profiling will generate cpu and memory profile files when the executors are completed.

func (*DistributedOption) WithFile

func (o *DistributedOption) WithFile(relatedFile, toFolder string) *DistributedOption

WithFile sends any related file over to gleam agents so the task can still access these files on gleam agents. The files are placed on the executed task's current working directory.

type DistributedPlanner

type DistributedPlanner struct {
}

func Planner

func Planner() *DistributedPlanner

func (*DistributedPlanner) GetFlowRunner

func (o *DistributedPlanner) GetFlowRunner() flow.FlowRunner

func (*DistributedPlanner) RunFlowContext

func (fcd *DistributedPlanner) RunFlowContext(ctx context.Context, fc *flow.Flow)

driver runs on local, controlling all tasks

Directories

Path Synopsis
Package agent runs on servers with computing resources, and executes tasks sent by driver.
Package agent runs on servers with computing resources, and executes tasks sent by driver.
Package driver coordinates distributed execution.
Package driver coordinates distributed execution.
scheduler
Schedule tasks to run on available resources assigned by master.
Schedule tasks to run on available resources assigned by master.
Package master collects data from agents, and manage named network channels.
Package master collects data from agents, and manage named network channels.
ui
Package netchan creates network channels.
Package netchan creates network channels.
Disk-backed queue
Disk-backed queue

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL