grpc: Index | Files

package adaptive

import ""

Package adaptive provides functionality for adaptive client-side throttling.


Package Files

adaptive.go lookback.go

type Throttler Uses

type Throttler struct {
    // contains filtered or unexported fields

Throttler implements a client-side throttling recommendation system. All methods are safe for concurrent use by multiple goroutines.

The throttler has the following knobs for which we will use defaults for now. If there is a need to make them configurable at a later point in time, support for the same will be added. * Duration: amount of recent history that will be taken into account for

making client-side throttling decisions. A default of 30 seconds is used.

* Bins: number of bins to be used for bucketing historical data. A default

of 100 is used.

* RatioForAccepts: ratio by which accepts are multiplied, typically a value

slightly larger than 1.0. This is used to make the throttler behave as if
the backend had accepted more requests than it actually has, which lets us
err on the side of sending to the backend more requests than we think it
will accept for the sake of speeding up the propagation of state. A
default of 2.0 is used.

* RequestsPadding: is used to decrease the (client-side) throttling

probability in the low QPS regime (to speed up propagation of state), as
well as to safeguard against hitting a client-side throttling probability
of 100%. The weight of this value decreases as the number of requests in
recent history grows. A default of 8 is used.

The adaptive throttler attempts to estimate the probability that a request will be throttled using recent history. Server requests (both throttled and accepted) are registered with the throttler (via the RegisterBackendResponse method), which then recommends client-side throttling (via the ShouldThrottle method) with probability given by: (requests - RatioForAccepts * accepts) / (requests + RequestsPadding)

func New Uses

func New() *Throttler

New initializes a new adaptive throttler with the default values.

func (*Throttler) RegisterBackendResponse Uses

func (t *Throttler) RegisterBackendResponse(throttled bool)

RegisterBackendResponse registers a response received from the backend for a request allowed by ShouldThrottle. This should be called for every response received from the backend (i.e., once for each request for which ShouldThrottle returned false).

func (*Throttler) ShouldThrottle Uses

func (t *Throttler) ShouldThrottle() bool

ShouldThrottle returns a probabilistic estimate of whether the server would throttle the next request. This should be called for every request before allowing it to hit the network. If the returned value is true, the request should be aborted immediately (as if it had been throttled by the server).

Package adaptive imports 3 packages (graph). Updated 2020-08-15. Refresh now. Tools for package owners.