Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var JobNotFoundError = errors.New("job not found")
Functions ¶
func IsShimEntry ¶
func IsShimEntry() bool
IsShimEntry returns true if the process is executing as a runner shim entrypoint.
Types ¶
type CgroupRunner ¶
type CgroupRunner struct {
// contains filtered or unexported fields
}
CgroupRunner is a runner that uses cgroups to organize the jobs.
func NewCgroupRunner ¶
func NewCgroupRunner(config *RunnerConfig) (*CgroupRunner, error)
NewCgroupRunner returns a new runner, or an error. Note that creating a runner means that the current process is moved to a new shim cgroup called 'runner'. The IO, CPU and memory controllers get enabled for subtree hierarchy. The config is optional.
func (*CgroupRunner) Output ¶
func (r *CgroupRunner) Output(name string) (output <-chan []byte, cancel func(), err error)
Output streams the job's output on channel output, which can be canceled by calling the provided cancel function. The output channel will automatically close when the job completes and becomes inactive. If the job is already inactive, all of its output will be received in chunks over the output channel, after which the channel will be closed too.
TODO use a smarter output channel to convey a struct with either output bytes or an error
func (*CgroupRunner) Start ¶
func (r *CgroupRunner) Start(config Config) (string, error)
Start a new job and return its name or an error.
This will spawn a new process by calling /proc/self/exe with a number of command line arguments. The child is run with a new PID, memory and network namespace. The child's stderr and stdout will be redirected to a file, while stdin is closed. The actual job's command is launched through a shim which moves the child to a new cgroup under the current hierarchy and execs the command. The caller must provide means for reaching the shim's entrypoint, preferrably testing with IsShimEntry() and calling ShimEntry() if the test is positive.
func (*CgroupRunner) Status ¶
func (r *CgroupRunner) Status(name string) (*Status, error)
Status of a job. If the job has already completed, its status will indicate that the active status is false.
func (*CgroupRunner) Stop ¶
func (r *CgroupRunner) Stop(name string) (*Status, error)
Stop stops a job with a given name, returns the job's status or an error. All resources associated with a job are released, its output file is removed, thus if the output is of value, it should be collected earlier. If the job's process has not completed yet, it will be forcefully killed.
type Config ¶
type Config struct {
// Command to run
Command []string
// CPUMax in the format of cgroupv2 cpu.max
CPUMax string
// CPUMax in the format of cgroupv2 io.max
IOMax string
// CPUMax in the format of cgroupv2 memory.max
MemoryMax string
}
Config of a job
type RunnerConfig ¶
type RunnerConfig struct {
// StorageRoot is where the job output will be stored
StorageRoot string
// JobNamePrefix is a prefix to use for job names, eg. "pile."
JobNamePrefix string
}
RunnerConfig is a configuration for the runner
Source Files