amboy: github.com/mongodb/amboy/pool Index | Files

package pool

import "github.com/mongodb/amboy/pool"

Package pool provides specific implementations of the amboy.Runner interface that serve as the worker pools for tasks in work queues.

Intentionally, most of the important logic about job execution and dispatching happens in the Queue implementation, and the Runner implementations are simplistic.

Grouped Pool

The MultiPool implementation is substantially similar to the Simple Pool; however, one MultiPool instance can service jobs from multiple Queues. In this case, the Runner *must* be attached to all of the queues before workers are spawned and the queues begin dispatching tasks.

Local Workers Pool

The LocalWorkers is a simple worker pool implementation that spawns a collection of (n) workers and dispatches jobs to worker threads, that consume work items from the Queue's Next() method.

Index

Package Files

doc.go group.go local.go single.go

type Group Uses

type Group struct {
    // contains filtered or unexported fields
}

Group is a Runner implementation that can, potentially, run tasks from multiple queues at the same time.

func NewGroup Uses

func NewGroup(numWorkers int) *Group

NewGroup start creates a new Runner instance with the specified number of worker processes capable of running jobs from multiple queues.

func (*Group) Close Uses

func (r *Group) Close()

Close cancels all pending workers and waits for the running processes to return.

func (*Group) SetQueue Uses

func (r *Group) SetQueue(q amboy.Queue) error

SetQueue adds a new queue object to the Runner instance. There is no way to remove a amboy.Queue from a runner object, and no way to add a a amboy.Queue after starting to dispatch jobs.

func (*Group) Start Uses

func (r *Group) Start(ctx context.Context) error

Start initializes all worker process, and returns an error if the Runner has already started.

func (*Group) Started Uses

func (r *Group) Started() bool

Started returns true if the runner's worker threads have started, and false otherwise.

type LocalWorkers Uses

type LocalWorkers struct {
    // contains filtered or unexported fields
}

LocalWorkers is a very minimal implementation of a worker pool, and supports a configurable number of workers to process Job tasks.

func NewLocalWorkers Uses

func NewLocalWorkers(numWorkers int, q amboy.Queue) *LocalWorkers

NewLocalWorkers is a constructor for LocalWorkers objects, and takes arguments for the number of worker processes and a amboy.Queue object.

func (*LocalWorkers) Close Uses

func (r *LocalWorkers) Close()

Close terminates all worker processes as soon as possible.

func (*LocalWorkers) SetQueue Uses

func (r *LocalWorkers) SetQueue(q amboy.Queue) error

SetQueue allows callers to inject alternate amboy.Queue objects into constructed Runner objects. Returns an error if the Runner has started.

func (*LocalWorkers) Start Uses

func (r *LocalWorkers) Start(ctx context.Context) error

Start initializes all worker process, and returns an error if the Runner has already started.

func (*LocalWorkers) Started Uses

func (r *LocalWorkers) Started() bool

Started returns true when the Runner has begun executing tasks. For LocalWorkers this means that workers are running.

type SingleRunner Uses

type SingleRunner struct {
    // contains filtered or unexported fields
}

SingleRunner is an implementation of of the amboy.Runner interface that hosts runs all tasks on one, and only one worker. Useful for testing the system with a different task executor.

func NewSingleRunner Uses

func NewSingleRunner() *SingleRunner

NewSingleRunner returns a new single-worker pool.

func (*SingleRunner) Close Uses

func (r *SingleRunner) Close()

Close terminates the work on the Runner. If a job is executing, the job will complete and the process will terminate before beginning a new job. If the queue has not started, Close is a no-op.

func (*SingleRunner) SetQueue Uses

func (r *SingleRunner) SetQueue(q amboy.Queue) error

SetQueue allows callers to inject alternate amboy.Queue objects into constructed Runner objects. Returns an error if the Runner has started.

func (*SingleRunner) Start Uses

func (r *SingleRunner) Start(ctx context.Context) error

Start takes a context and starts the internal worker and job processing thread. You can terminate the work of the Runner by canceling the context, or with the close method. Returns an error if the queue is not set. If the Runner is already running, Start is a no-op.

func (*SingleRunner) Started Uses

func (r *SingleRunner) Started() bool

Started returns true when the Runner has begun executing tasks. For LocalWorkers this means that workers are running.

Package pool imports 7 packages (graph) and is imported by 2 packages. Updated 2017-03-25. Refresh now. Tools for package owners.