cockroach: Index | Files

package requestbatcher

import ""

Package requestbatcher is a library to enable easy batching of roachpb requests.

Batching in general represents a tradeoff between throughput and latency. The underlying assumption being that batched operations are cheaper than an individual operation. If this is not the case for your workload, don't use this library.

Batching assumes that data with the same key can be sent in a single batch. The initial implementation uses rangeID as the key explicitly to avoid creating an overly general solution without motivation but interested readers should recognize that it would be easy to extend this package to accept an arbitrary comparable key.


Package Files



const (
    // DefaultInFlightBackpressureLimit is the InFlightBackpressureLimit used if
    // a zero value for that setting is passed in a Config to New.
    // TODO(ajwerner): Justify this number.
    DefaultInFlightBackpressureLimit = 1000

type Config Uses

type Config struct {

    // Name of the batcher, used for logging, timeout errors, and the stopper.
    Name string

    // Sender can round-trip a batch. Sender must not be nil.
    Sender kv.Sender

    // Stopper controls the lifecycle of the Batcher. Stopper must not be nil.
    Stopper *stop.Stopper

    // MaxSizePerBatch is the maximum number of bytes in individual requests in a
    // batch. If MaxSizePerBatch <= 0 then no limit is enforced.
    MaxSizePerBatch int

    // MaxMsgsPerBatch is the maximum number of messages.
    // If MaxMsgsPerBatch <= 0 then no limit is enforced.
    MaxMsgsPerBatch int

    // MaxKeysPerBatchReq is the maximum number of keys that each batch is
    // allowed to touch during one of its requests. If the limit is exceeded,
    // the batch is paginated over a series of individual requests. This limit
    // corresponds to the MaxSpanRequestKeys assigned to the Header of each
    // request. If MaxKeysPerBatchReq <= 0 then no limit is enforced.
    MaxKeysPerBatchReq int

    // MaxWait is the maximum amount of time a message should wait in a batch
    // before being sent. If MaxWait is <= 0 then no wait timeout is enforced.
    // It is inadvisable to disable both MaxIdle and MaxWait.
    MaxWait time.Duration

    // MaxIdle is the amount of time a batch should wait between message additions
    // before being sent. The idle timer allows clients to observe low latencies
    // when throughput is low. If MaxWait is <= 0 then no wait timeout is
    // enforced. It is inadvisable to disable both MaxIdle and MaxWait.
    MaxIdle time.Duration

    // InFlightBackpressureLimit is the number of batches in flight above which
    // sending clients should experience backpressure. If the batcher has more
    // requests than this in flight it will not accept new requests until the
    // number of in flight batches is again below this threshold. This value does
    // not limit the number of batches which may ultimately be in flight as
    // batches which are queued to send but not yet in flight will still send.
    // Note that values	less than or equal to zero will result in the use of
    // DefaultInFlightBackpressureLimit.
    InFlightBackpressureLimit int

    // NowFunc is used to determine the current time. It defaults to timeutil.Now.
    NowFunc func() time.Time

Config contains the dependencies and configuration for a Batcher.

type RequestBatcher Uses

type RequestBatcher struct {
    // contains filtered or unexported fields

RequestBatcher batches requests destined for a single range based on a configured batching policy.

func New Uses

func New(cfg Config) *RequestBatcher

New creates a new RequestBatcher.

func (*RequestBatcher) Send Uses

func (b *RequestBatcher) Send(
    ctx context.Context, rangeID roachpb.RangeID, req roachpb.Request,
) (roachpb.Response, error)

Send sends req as a part of a batch. An error is returned if the context is canceled before the sending of the request completes. The context with the latest deadline for a batch is used to send the underlying batch request.

func (*RequestBatcher) SendWithChan Uses

func (b *RequestBatcher) SendWithChan(
    ctx context.Context, respChan chan<- Response, rangeID roachpb.RangeID, req roachpb.Request,
) error

SendWithChan sends a request with a client provided response channel. The client is responsible for ensuring that the passed respChan has a buffer at least as large as the number of responses it expects to receive. Using an insufficiently buffered channel can lead to deadlocks and unintended delays processing requests inside the RequestBatcher.

type Response Uses

type Response struct {
    Resp roachpb.Response
    Err  error

Response is exported for use with the channel-oriented SendWithChan method. At least one of Resp or Err will be populated for every sent Response.

Package requestbatcher imports 10 packages (graph) and is imported by 5 packages. Updated 2021-01-21. Refresh now. Tools for package owners.