Documentation
¶
Overview ¶
Package pool provides specific implementations of the amboy.Runner interface that serve as the worker pools for tasks in work queues.
Intentionally, most of the important logic about job execution and dispatching happens in the Queue implementation, and the Runner implementations are simplistic.
Local Workers Pool ¶
The LocalWorkers is a simple worker pool implementation that spawns a collection of (n) workers and dispatches jobs to worker threads, that consume work items from the Queue's Next() method.
Rate Limiting Pools ¶
Amboy includes two rate limiting pools, to control the flow of tasks processed by the queue. The "simple" implementation sleeps for a configurable interval in-between each task, while the averaged tool, uses an exponential weighted average and a targeted number of tasks to complete over an interval to achieve a reasonable flow of tasks through the runner.
Index ¶
- func NewAbortablePool(size int, q amboy.Queue) amboy.AbortableRunner
- func NewLocalWorkers(numWorkers int, q amboy.Queue) amboy.Runner
- func NewMovingAverageRateLimitedWorkers(size, targetNum int, period time.Duration, q amboy.Queue) (amboy.AbortableRunner, error)
- func NewSimpleRateLimitedWorkers(size int, sleepInterval time.Duration, q amboy.Queue) (amboy.Runner, error)
- func NewSingle() amboy.Runner
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewAbortablePool ¶
func NewAbortablePool(size int, q amboy.Queue) amboy.AbortableRunner
NewAbortablePool produces a simple implementation of a worker pool that provides access to cancel running jobs. The cancellation functions work by creating context cancelation function and then canceling the contexts passed to the jobs specifically.
func NewLocalWorkers ¶
NewLocalWorkers is a constructor for pool of worker processes that execute jobs from a queue locally, and takes arguments for the number of worker processes and a amboy.Queue object.
func NewMovingAverageRateLimitedWorkers ¶
func NewMovingAverageRateLimitedWorkers(size, targetNum int, period time.Duration, q amboy.Queue) (amboy.AbortableRunner, error)
NewMovingAverageRateLimitedWorkers returns a worker pool implementation that attempts to run a target number of tasks over a specified period to provide a more stable dispatching rate. It uses an exponentially weighted average of task time when determining the rate, which favors recent tasks over previous tasks.
Returns an error if the size or target numbers are less than one and if the period is less than a millisecond.
func NewSimpleRateLimitedWorkers ¶
func NewSimpleRateLimitedWorkers(size int, sleepInterval time.Duration, q amboy.Queue) (amboy.Runner, error)
NewSimpleRateLimitedWorkers returns a worker pool that sleeps for the specified interval after completing each task. After that interval, the runner will run the next available task as soon as its ready.
The constructor returns an error if the size (number of workers) is less than 1 or the interval is less than a millisecond.
Types ¶
This section is empty.
Source Files