pachyderm: github.com/pachyderm/pachyderm/src/server/pkg/backoff Index | Examples | Files

package backoff

import "github.com/pachyderm/pachyderm/src/server/pkg/backoff"

Package backoff implements backoff algorithms for retrying operations.

Use Retry function for retrying operations that may fail. If Retry does not meet your needs, copy/paste the function into your project and modify as you wish.

There is also Ticker type similar to time.Ticker. You can use it if you need to work with channels.

See Examples section below for usage examples.

Index

Examples

Package Files

backoff.go exponential.go retry.go ticker.go

Constants

const (
    DefaultInitialInterval     = 500 * time.Millisecond
    DefaultRandomizationFactor = 0.5
    DefaultMultiplier          = 1.5
    DefaultMaxInterval         = 60 * time.Second
    DefaultMaxElapsedTime      = 15 * time.Minute
)

Default values for ExponentialBackOff.

const Stop time.Duration = -1

Stop indicates that no more retries should be made for use in NextBackOff().

Variables

var SystemClock = systemClock{}

SystemClock implements Clock interface that uses time.Now().

func Retry Uses

func Retry(o Operation, b BackOff) error

Retry the operation o until it does not return error or BackOff stops. o is guaranteed to be run at least once. It is the caller's responsibility to reset b after Retry returns.

Retry sleeps the goroutine for the duration returned by BackOff after a failed operation returns.

Code:

// An operation that may fail.
operation := func() error {
    return nil // or an error
}

err := Retry(operation, NewExponentialBackOff())
if err != nil {
    // Handle error.
    return
}

// Operation is successful.

func RetryNotify Uses

func RetryNotify(operation Operation, b BackOff, notify Notify) error

RetryNotify calls notify function with the error and wait duration for each failed attempt before sleep.

type BackOff Uses

type BackOff interface {
    // NextBackOff returns the duration to wait before retrying the operation,
    // or backoff.Stop to indicate that no more retries should be made.
    //
    // Example usage:
    //
    // 	duration := backoff.NextBackOff();
    // 	if (duration == backoff.Stop) {
    // 		// Do not retry operation.
    // 	} else {
    // 		// Sleep for duration and retry operation.
    // 	}
    //
    NextBackOff() time.Duration

    // Reset to initial state.
    Reset()
}

BackOff is a backoff policy for retrying an operation.

type Clock Uses

type Clock interface {
    Now() time.Time
}

Clock is an interface that returns current time for BackOff.

type ConstantBackOff Uses

type ConstantBackOff struct {
    Interval time.Duration

    // After MaxElapsedTime the ConstantBackOff stops.
    // It never stops if MaxElapsedTime == 0.
    MaxElapsedTime time.Duration
    // contains filtered or unexported fields
}

ConstantBackOff is a backoff policy that always returns the same backoff delay. This is in contrast to an exponential backoff policy, which returns a delay that grows longer as you call NextBackOff() over and over again.

Note: Implementation is not thread-safe

func NewConstantBackOff Uses

func NewConstantBackOff(d time.Duration) *ConstantBackOff

NewConstantBackOff ...

func RetryEvery Uses

func RetryEvery(d time.Duration) *ConstantBackOff

RetryEvery is an alias for NewConstantBackoff, with a nicer name for inline calls

func (*ConstantBackOff) For Uses

func (b *ConstantBackOff) For(maxElapsed time.Duration) *ConstantBackOff

For sets b.MaxElapsedTime to 'maxElapsed' and returns b

func (*ConstantBackOff) GetElapsedTime Uses

func (b *ConstantBackOff) GetElapsedTime() time.Duration

GetElapsedTime returns the elapsed time since an ExponentialBackOff instance is created and is reset when Reset() is called.

The elapsed time is computed using time.Now().UnixNano().

func (*ConstantBackOff) NextBackOff Uses

func (b *ConstantBackOff) NextBackOff() time.Duration

NextBackOff ...

func (*ConstantBackOff) Reset Uses

func (b *ConstantBackOff) Reset()

Reset ...

type ExponentialBackOff Uses

type ExponentialBackOff struct {
    InitialInterval     time.Duration
    RandomizationFactor float64
    Multiplier          float64
    MaxInterval         time.Duration
    // After MaxElapsedTime the ExponentialBackOff stops.
    // It never stops if MaxElapsedTime == 0.
    MaxElapsedTime time.Duration
    Clock          Clock
    // contains filtered or unexported fields
}

ExponentialBackOff is a backoff implementation that increases the backoff period for each retry attempt using a randomization function that grows exponentially.

NextBackOff() is calculated using the following formula:

randomized interval =
    RetryInterval * (random value in range [1 - RandomizationFactor, 1 + RandomizationFactor])

In other words NextBackOff() will range between the randomization factor percentage below and above the retry interval.

For example, given the following parameters:

RetryInterval = 2
RandomizationFactor = 0.5
Multiplier = 2

the actual backoff period used in the next retry attempt will range between 1 and 3 seconds, multiplied by the exponential, that is, between 2 and 6 seconds.

Note: MaxInterval caps the RetryInterval and not the randomized interval.

If the time elapsed since an ExponentialBackOff instance is created goes past the MaxElapsedTime, then the method NextBackOff() starts returning backoff.Stop.

The elapsed time can be reset by calling Reset().

Example: Given the following default arguments, for 10 tries the sequence will be, and assuming we go over the MaxElapsedTime on the 10th try:

Request #  RetryInterval (seconds)  Randomized Interval (seconds)

 1          0.5                     [0.25,   0.75]
 2          0.75                    [0.375,  1.125]
 3          1.125                   [0.562,  1.687]
 4          1.687                   [0.8435, 2.53]
 5          2.53                    [1.265,  3.795]
 6          3.795                   [1.897,  5.692]
 7          5.692                   [2.846,  8.538]
 8          8.538                   [4.269, 12.807]
 9         12.807                   [6.403, 19.210]
10         19.210                   backoff.Stop

Note: Implementation is not thread-safe.

func New10sBackOff Uses

func New10sBackOff() *ExponentialBackOff

New10sBackOff returns a backoff that's slightly more aggressive than NewExponentialBackOff. The Max Elapsed time for this backoff is 10s, and the initial backoff is 100ms (instead of 500). Therefore this will retry at most 10 times and then fail (depending on RPC timeout), and may be more useful for interactive RPCs than the default timeout of 60s.

func NewExponentialBackOff Uses

func NewExponentialBackOff() *ExponentialBackOff

NewExponentialBackOff creates an instance of ExponentialBackOff using default values.

func NewInfiniteBackOff Uses

func NewInfiniteBackOff() *ExponentialBackOff

NewInfiniteBackOff creates an instance of ExponentialBackOff that never ends.

func NewTestingBackOff Uses

func NewTestingBackOff() *ExponentialBackOff

NewTestingBackOff returns a backoff tuned towards waiting for a Pachyderm state change in a test

func (*ExponentialBackOff) GetElapsedTime Uses

func (b *ExponentialBackOff) GetElapsedTime() time.Duration

GetElapsedTime returns the elapsed time since an ExponentialBackOff instance is created and is reset when Reset() is called.

The elapsed time is computed using time.Now().UnixNano().

func (*ExponentialBackOff) NextBackOff Uses

func (b *ExponentialBackOff) NextBackOff() time.Duration

NextBackOff calculates the next backoff interval using the formula:

Randomized interval = RetryInterval +/- (RandomizationFactor * RetryInterval)

func (*ExponentialBackOff) Reset Uses

func (b *ExponentialBackOff) Reset()

Reset the interval back to the initial retry interval and restarts the timer.

type Notify Uses

type Notify func(error, time.Duration) error

Notify is a notify-on-error function. It receives an operation error and backoff delay if the operation failed (with an error).

If the notify function returns an error itself, we stop retrying and return the error.

NOTE that if the backoff policy stated to stop retrying, the notify function isn't called.

type Operation Uses

type Operation func() error

An Operation is executing by Retry() or RetryNotify(). The operation will be retried using a backoff policy if it returns an error.

type StopBackOff Uses

type StopBackOff struct{}

StopBackOff is a fixed backoff policy that always returns backoff.Stop for NextBackOff(), meaning that the operation should never be retried.

func (*StopBackOff) NextBackOff Uses

func (b *StopBackOff) NextBackOff() time.Duration

NextBackOff ...

func (*StopBackOff) Reset Uses

func (b *StopBackOff) Reset()

Reset ...

type Ticker Uses

type Ticker struct {
    C <-chan time.Time
    // contains filtered or unexported fields
}

Ticker holds a channel that delivers `ticks' of a clock at times reported by a BackOff.

Ticks will continue to arrive when the previous operation is still running, so operations that take a while to fail could run in quick succession.

Code:

// An operation that may fail.
operation := func() error {
    return nil // or an error
}

ticker := NewTicker(NewExponentialBackOff())

var err error

// Ticks will continue to arrive when the previous operation is still running,
// so operations that take a while to fail could run in quick succession.
for range ticker.C {
    if err = operation(); err != nil {
        log.Println(err, "will retry...")
        continue
    }

    ticker.Stop()
    break
}

if err != nil {
    // Operation has failed.
    return
}

// Operation is successful.
return

func NewTicker Uses

func NewTicker(b BackOff) *Ticker

NewTicker returns a new Ticker containing a channel that will send the time at times specified by the BackOff argument. Ticker is guaranteed to tick at least once. The channel is closed when Stop method is called or BackOff stops.

func (*Ticker) Stop Uses

func (t *Ticker) Stop()

Stop turns off a ticker. After Stop, no more ticks will be sent.

type ZeroBackOff Uses

type ZeroBackOff struct{}

ZeroBackOff is a fixed backoff policy whose backoff time is always zero, meaning that the operation is retried immediately without waiting, indefinitely.

func (*ZeroBackOff) NextBackOff Uses

func (b *ZeroBackOff) NextBackOff() time.Duration

NextBackOff ...

func (*ZeroBackOff) Reset Uses

func (b *ZeroBackOff) Reset()

Reset ...

Package backoff imports 4 packages (graph) and is imported by 12 packages. Updated 2019-01-29. Refresh now. Tools for package owners.