rehttp: github.com/PuerkitoBio/rehttp Index | Examples | Files

package rehttp

import "github.com/PuerkitoBio/rehttp"

Package rehttp implements an HTTP transport that handles retries. An HTTP client can be created with a *rehttp.Transport as RoundTripper and it will apply the retry strategy to its requests.

The retry strategy is provided by the Transport, which determines whether or not the request should be retried, and if so, what delay to apply before retrying, based on the RetryFn and DelayFn functions passed to NewTransport.

The package offers common delay strategies as ready-made functions that return a DelayFn:

- ConstDelay(delay time.Duration) DelayFn
- ExpJitterDelay(base, max time.Duration) DelayFn

It also provides common retry helpers that return a RetryFn:

- RetryIsErr(func(error) bool) RetryFn
- RetryHTTPMethods(methods ...string) RetryFn
- RetryMaxRetries(max int) RetryFn
- RetryStatuses(statuses ...int) RetryFn
- RetryStatusInterval(fromStatus, toStatus int) RetryFn
- RetryTemporaryErr() RetryFn

Those can be combined with RetryAny or RetryAll as needed. RetryAny enables retries if any of the RetryFn return true, while RetryAll enables retries if all RetryFn return true. Typically, the RetryFn of the Transport should use at least RetryMaxRetries and some other retry condition(s), combined using RetryAll.

By default, the Transport will buffer the request's body in order to be able to retry the request, as a request attempt will consume and close the existing body. Sometimes this is not desirable, so it can be prevented by setting PreventRetryWithBody to true on the Transport. Doing so will disable retries when a request has a non-nil body.

This package requires Go version 1.6+, since it uses the new http.Request.Cancel field in order to cancel requests. It doesn't implement the deprecated http.Transport.CancelRequest method (https://golang.org/pkg/net/http/#Transport.CancelRequest).

On Go1.7+, it uses the context returned by http.Request.Context to check for cancelled requests.

It should work on Go1.5, but only if there is no timeout set on the *http.Client. Go's stdlib will return an error on the first request if that's the case, because it requires a RoundTripper that implements the CancelRequest method.

Code:

tr := rehttp.NewTransport(
    nil, // will use http.DefaultTransport
    rehttp.RetryAll(rehttp.RetryMaxRetries(3), rehttp.RetryTemporaryErr()), // max 3 retries for Temporary errors
    rehttp.ConstDelay(time.Second),                                         // wait 1s between retries
)
client := &http.Client{
    Transport: tr,
    Timeout:   10 * time.Second, // Client timeout applies to all retries as a whole
}
res, err := client.Get("http://example.com")
if err != nil {
    // handle err
}
// handle response
res.Body.Close()

Index

Examples

Package Files

context_post17.go rehttp.go

Variables

var PRNG = rand.New(rand.NewSource(time.Now().UnixNano()))

PRNG is the *math.Rand value to use to add jitter to the backoff algorithm used in ExpJitterDelay. By default it uses a *rand.Rand initialized with a source based on the current time in nanoseconds.

type Attempt Uses

type Attempt struct {
    // Index is the attempt index starting at 0.
    Index int

    // Request is the request for this attempt. If a non-nil Response
    // is present, this is the same as Response.Request, but since a
    // Response may not be available, it is guaranteed to be set on this
    // field.
    Request *http.Request

    // Response is the response for this attempt. It may be nil if an
    // error occurred an no response was received.
    Response *http.Response

    // Error is the error returned by the attempt, if any.
    Error error
}

Attempt holds the data describing a RoundTrip attempt.

type DelayFn Uses

type DelayFn func(attempt Attempt) time.Duration

DelayFn is the signature for functions that return the delay to apply before the next retry.

func ConstDelay Uses

func ConstDelay(delay time.Duration) DelayFn

ConstDelay returns a DelayFn that always returns the same delay.

func ExpJitterDelay Uses

func ExpJitterDelay(base, max time.Duration) DelayFn

ExpJitterDelay returns a DelayFn that returns a delay between 0 and base * 2^attempt capped at max (an exponential backoff delay with jitter).

See the full jitter algorithm in: http://www.awsarchitectureblog.com/2015/03/backoff.html

type RetryFn Uses

type RetryFn func(attempt Attempt) bool

RetryFn is the signature for functions that return whether a retry should be done for the request.

func RetryAll Uses

func RetryAll(retryFns ...RetryFn) RetryFn

RetryAll returns a RetryFn that allows a retry if all retryFns return true. If retryFns is empty, it always returns true.

func RetryAny Uses

func RetryAny(retryFns ...RetryFn) RetryFn

RetryAny returns a RetryFn that allows a retry as long as one of the retryFns returns true. If retryFns is empty, it always returns false.

func RetryHTTPMethods Uses

func RetryHTTPMethods(methods ...string) RetryFn

RetryHTTPMethods returns a RetryFn that retries if the request's HTTP method is one of the provided methods. It is meant to be used in conjunction with another RetryFn such as RetryTemporaryErr combined using RetryAll, otherwise this function will retry any successful request made with one of the provided methods.

func RetryIsErr Uses

func RetryIsErr(fn func(error) bool) RetryFn

RetryIsErr returns a RetryFn that retries if the provided error predicate - a function that receives an error and returns a boolean - returns true for the error associated with the Attempt. Note that fn may be called with a nil error.

func RetryMaxRetries Uses

func RetryMaxRetries(max int) RetryFn

RetryMaxRetries returns a RetryFn that retries if the number of retries is less than or equal to max.

func RetryStatusInterval Uses

func RetryStatusInterval(fromStatus, toStatus int) RetryFn

RetryStatusInterval returns a RetryFn that retries if the response's status code is in the provided half-closed interval [fromStatus, toStatus) (that is, it retries if fromStatus <= Response.StatusCode < toStatus, so RetryStatusInterval(400, 500) would retry for any 4xx code, but not for 500).

func RetryStatuses Uses

func RetryStatuses(statuses ...int) RetryFn

RetryStatuses returns a RetryFn that retries if the response's status code is one of the provided statuses.

func RetryTemporaryErr Uses

func RetryTemporaryErr() RetryFn

RetryTemporaryErr returns a RetryFn that retries if the Attempt's error is a temporary error. A temporary error is one that implements the Temporary() bool method. Most errors from the net package implement this.

type Transport Uses

type Transport struct {
    http.RoundTripper

    // PreventRetryWithBody prevents retrying if the request has a body. Since
    // the body is consumed on a request attempt, in order to retry a request
    // with a body, the body has to be buffered in memory. Setting this
    // to true avoids this buffering: the retry logic is bypassed if the body
    // is non-nil.
    PreventRetryWithBody bool
    // contains filtered or unexported fields
}

Transport wraps a RoundTripper such as *http.Transport and adds retry logic.

func NewTransport Uses

func NewTransport(rt http.RoundTripper, retry RetryFn, delay DelayFn) *Transport

NewTransport creates a Transport with a retry strategy based on retry and delay to control the retry logic. It uses the provided RoundTripper to execute the requests. If rt is nil, http.DefaultTransport is used.

func (*Transport) RoundTrip Uses

func (t *Transport) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements http.RoundTripper for the Transport type. It calls its underlying http.RoundTripper to execute the request, and adds retry logic as per its configuration.

Package rehttp imports 9 packages (graph) and is imported by 13 packages. Updated 2019-04-15. Refresh now. Tools for package owners.