pool

package
v4.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 11, 2024 License: MIT Imports: 9 Imported by: 15

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AllocateParallel

func AllocateParallel(numWorkers uint64, allocator func() (*worker.Process, error)) ([]*worker.Process, error)

AllocateParallel allocate required number of stack

func NewPoolAllocator

func NewPoolAllocator(ctx context.Context, timeout time.Duration, maxExecs uint64, factory Factory, cmd Command, command []string, log *zap.Logger) func() (*worker.Process, error)

NewPoolAllocator initializes allocator of the workers

Types

type Command

type Command func(cmd []string) *exec.Cmd

Command is a function that returns a new exec.Cmd instance for the given command string.

type Config

type Config struct {
	// Debug flag creates new fresh worker before every request.
	Debug bool
	// Command used to override the server command with the custom one
	Command []string `mapstructure:"command"`
	// MaxQueueSize is maximum allowed queue size with the pending requests to the workers poll
	MaxQueueSize uint64 `mapstructure:"max_queue_size"`
	// NumWorkers defines how many sub-processes can be run at once. This value
	// might be doubled by Swapper while hot-swap. Defaults to number of CPU cores.
	NumWorkers uint64 `mapstructure:"num_workers"`
	// MaxJobs defines how many executions is allowed for the worker until
	// its destruction. set 1 to create new process for each new task, 0 to let
	// worker handle as many tasks as it can.
	MaxJobs uint64 `mapstructure:"max_jobs"`
	// AllocateTimeout defines for how long pool will be waiting for a worker to
	// be freed to handle the task. Defaults to 60s.
	AllocateTimeout time.Duration `mapstructure:"allocate_timeout"`
	// DestroyTimeout defines for how long pool should be waiting for worker to
	// properly destroy, if timeout reached worker will be killed. Defaults to 60s.
	DestroyTimeout time.Duration `mapstructure:"destroy_timeout"`
	// ResetTimeout defines how long pool should wait before start killing workers
	ResetTimeout time.Duration `mapstructure:"reset_timeout"`
	// Stream read operation timeout
	StreamTimeout time.Duration `mapstructure:"stream_timeout"`
	// Supervision config to limit worker and pool memory usage.
	Supervisor *SupervisorConfig `mapstructure:"supervisor"`
}

Config .. Pool config Configures the pool behavior.

func (*Config) InitDefaults

func (cfg *Config) InitDefaults()

InitDefaults enables default config values.

type Factory

type Factory interface {
	// SpawnWorkerWithContext creates a new WorkerProcess process based on given command with context.
	// Process must not be started.
	SpawnWorkerWithContext(context.Context, *exec.Cmd, ...worker.Options) (*worker.Process, error)
	// Close the factory and underlying connections.
	Close() error
}

Factory is responsible for wrapping given command into tasks WorkerProcess.

type SupervisorConfig

type SupervisorConfig struct {
	// WatchTick defines how often to check the state of worker.
	WatchTick time.Duration `mapstructure:"watch_tick"`
	// TTL defines the maximum time for the worker is allowed to live.
	TTL time.Duration `mapstructure:"ttl"`
	// IdleTTL defines the maximum duration worker can spend in idle mode. Disabled when 0.
	IdleTTL time.Duration `mapstructure:"idle_ttl"`
	// ExecTTL defines maximum lifetime per job.
	ExecTTL time.Duration `mapstructure:"exec_ttl"`
	// MaxWorkerMemory limits memory per worker.
	MaxWorkerMemory uint64 `mapstructure:"max_worker_memory"`
}

func (*SupervisorConfig) InitDefaults

func (cfg *SupervisorConfig) InitDefaults()

InitDefaults enables default config values.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL