gocloud.dev: gocloud.dev/pubsub/batcher Index | Files

package batcher

import "gocloud.dev/pubsub/batcher"

Package batcher supports batching of items. Create a Batcher with a handler and add items to it. Items are accumulated while handler calls are in progress; when the handler returns, it will be called again with items accumulated since the last call. Multiple concurrent calls to the handler are supported.


Package Files


func Split Uses

func Split(n int, opts *Options) []int

Split determines how to split n (representing n items) into batches based on opts. It returns a slice of batch sizes.

For example, Split(10) might return [10], [5, 5], or [2, 2, 2, 2, 2] depending on opts. opts may be nil to accept defaults.

Split will return nil if n is less than o.MinBatchSize.

The sum of returned batches may be less than n (e.g., if n is 10x larger than o.MaxBatchSize, but o.MaxHandlers is less than 10).

type Batcher Uses

type Batcher struct {
    // contains filtered or unexported fields

A Batcher batches items.

func New Uses

func New(itemType reflect.Type, opts *Options, handler func(interface{}) error) *Batcher

New creates a new Batcher.

itemType is type that will be batched. For example, if you want to create batches of *Entry, pass reflect.TypeOf(&Entry{}) for itemType.

opts can be nil to accept defaults.

handler is a function that will be called on each bundle. If itemExample is of type T, the argument to handler is of type []T.

func (*Batcher) Add Uses

func (b *Batcher) Add(ctx context.Context, item interface{}) error

Add adds an item to the batcher. It blocks until the handler has processed the item and reports the error that the handler returned. If Shutdown has been called, Add immediately returns an error.

func (*Batcher) AddNoWait Uses

func (b *Batcher) AddNoWait(item interface{}) <-chan error

AddNoWait adds an item to the batcher and returns immediately. When the handler is called on the item, the handler's error return value will be sent to the channel returned from AddNoWait.

func (*Batcher) Shutdown Uses

func (b *Batcher) Shutdown()

Shutdown waits for all active calls to Add to finish, then returns. After Shutdown is called, all subsequent calls to Add fail. Shutdown should be called only once.

type Options Uses

type Options struct {
    // Maximum number of concurrent handlers. Defaults to 1.
    MaxHandlers int
    // Minimum size of a batch. Defaults to 1.
    MinBatchSize int
    // Maximum size of a batch. 0 means no limit.
    MaxBatchSize int

Options sets options for Batcher.

Package batcher imports 4 packages (graph) and is imported by 7 packages. Updated 2020-06-14. Refresh now. Tools for package owners.