httpretry

package module
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 22, 2023 License: MIT Imports: 10 Imported by: 11

README

Go Report Card Go build Codecov GoDoc GitHub license

httpRetry

Enriches the standard go http client with retry functionality using a wrapper around the Roundtripper interface.

The advantage of this library is that it makes use of the default http.Client. This means you can provide it to any library that accepts the go standard http.Client. This in turn gives you the possibility to add resilience to a lot of http based go libraries with just a single line of code. Of course it can also be used as standalone http client in your own projects.

Installation

go get -u github.com/ybbus/httpretry

Quickstart

To get a standard http client with retry functionality:

client := httpretry.NewDefaultClient()
// use this as usual when working with http.Client

This single line of code returns a default http.Client that uses an exponential backoff and sends up to 5 retries if the request was not successful. Requests will be retried if the error seems to be temporary or the requests returns a status code that may change over time (e.g. GetwayTimeout).

Modify / customize the Roundtripper (http.Transport)

Since httpretry wraps the actual Roundtripper of the http.Client, you should not try to replace / modify the client.Transport field after creation.

You either configure the http.Client upfront and then "make" it retryable like in this code:

customHttpClient := &http.Client{}
customHttpClient.Transport = &http.Transport{...}

retryClient := httpretry.NewCustomClient(cumstomHttpClient)

or you use one of the available helper functions to gain access to the underlying Roundtripper / http.Transport:

// replaces the original roundtripper
httpretry.ReplaceOriginalRoundtripper(retryClient, myRoundTripper)

// modifies the embedded http.Transport by providing a function that receives the client.Transport as parameter
httpretry.ModifyOriginalTransport(retryClient, func(t *http.Transport) { t.TLSHandshakeTimeout = 5 * time.Second })

// returns the embedded Roundtripper
httpretry.GetOriginalRoundtripper(retryClient)

// returns the embedded Roundtripper as http.Transport if it is of that type
httpretry.GetOriginalTransport(retryClient)
Customize retry settings

You may provide your own Backoff- and RetryPolicy.

client := httpretry.NewDefaultClient(
    // retry up to 5 times
    httpretry.WithMaxRetryCount(5),
    // retry on status >= 500, if err != nil, or if response was nil (status == 0)
    httpretry.WithRetryPolicy(func(statusCode int, err error) bool {
      return err != nil || statusCode >= 500 || statusCode == 0
    }),
    // every retry should wait one more second
    httpretry.WithBackoffPolicy(func(attemptNum int) time.Duration {
      return time.Duration(attemptNum+1) * 1 * time.Second
    }),
)

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (

	// ConstantBackoff waits for the exact same duration after a failed retry.
	//
	// constantWait: the constant backoff
	//
	// maxJitter: random interval [0, maxJitter) added to the exponential backoff
	//
	// Example:
	//   minWait = 2 * time.Seconds
	//   maxJitter = 0 * time.Seconds
	//
	//   Backoff will be: 2, 2, 2, ...
	ConstantBackoff = func(constantWait time.Duration, maxJitter time.Duration) BackoffPolicy {
		if constantWait < 0 {
			constantWait = 0
		}
		if maxJitter < 0 {
			maxJitter = 0
		}

		return func(attemptCount int) time.Duration {
			return constantWait + randJitter(maxJitter)
		}
	}

	// LinearBackoff increases the backoff time by multiplying the minWait duration by the number of attempts.
	//
	// minWait: the initial backoff
	//
	// maxWait: sets an upper bound on the maximum time to wait between two requests. set to 0 for no upper bound
	//
	// maxJitter: random interval [0, maxJitter) added to the linear backoff
	//
	// Example:
	//   minWait = 1 * time.Seconds
	//   maxWait = 5 * time.Seconds
	//   maxJitter = 0 * time.Seconds
	//
	//   Backoff will be: 1, 2, 3, 4, 5, 5, 5, ...
	LinearBackoff = func(minWait time.Duration, maxWait time.Duration, maxJitter time.Duration) BackoffPolicy {
		if minWait < 0 {
			minWait = 0
		}
		if maxJitter < 0 {
			maxJitter = 0
		}
		if maxWait < minWait {
			maxWait = 0
		}
		return func(attemptCount int) time.Duration {
			nextWait := time.Duration(attemptCount)*minWait + randJitter(maxJitter)
			if maxWait > 0 {
				return minDuration(nextWait, maxWait)
			}
			return nextWait
		}
	}

	// ExponentialBackoff increases the backoff exponentially by multiplying the minWait with 2^attemptCount
	//
	// minWait: the initial backoff
	//
	// maxWait: sets an upper bound on the maximum time to wait between two requests. set to 0 for no upper bound
	//
	// maxJitter: random interval [0, maxJitter) added to the exponential backoff
	//
	// Example:
	//   minWait = 1 * time.Seconds
	//   maxWait = 60 * time.Seconds
	//   maxJitter = 0 * time.Seconds
	//
	//   Backoff will be: 1, 2, 4, 8, 16, 32, 60, 60, ...
	ExponentialBackoff = func(minWait time.Duration, maxWait time.Duration, maxJitter time.Duration) BackoffPolicy {
		if minWait < 0 {
			minWait = 0
		}
		if maxJitter < 0 {
			maxJitter = 0
		}
		if maxWait < minWait {
			maxWait = 0
		}
		return func(attemptCount int) time.Duration {
			nextWait := time.Duration(math.Pow(2, float64(attemptCount-1)))*minWait + randJitter(maxJitter)
			if maxWait > 0 {
				return minDuration(nextWait, maxWait)
			}
			return nextWait
		}
	}
)

Functions

func GetOriginalRoundtripper added in v0.12.0

func GetOriginalRoundtripper(client *http.Client) http.RoundTripper

GetOriginalRoundtripper returns the original roundtripper that was embedded in the retry roundtripper.

func GetOriginalTransport added in v0.12.0

func GetOriginalTransport(client *http.Client) (*http.Transport, error)

GetOriginalTransport retrieves the original http.Transport that was mebedded in the retry roundtripper.

func ModifyOriginalTransport added in v0.12.0

func ModifyOriginalTransport(client *http.Client, f func(transport *http.Transport)) error

ModifyOriginalTransport allows to modify the original http.Transport that was embedded in the retry roundtipper.

func NewCustomClient added in v0.12.0

func NewCustomClient(client *http.Client, opts ...Option) *http.Client

NewCustomClient returns the provided http client with retry functionality wrapped around the Roundtripper (client.Transport).

You should not replace the client.Transport field after creating the retry client, otherwise you will lose the retry functionality.

If you need to change the original client.Transport field you may use the helper functions:

err := httpretry.ModifyTransport(retryClient, func(t *http.Transport){t.TLSHandshakeTimeout = 5 * time.Second})
if err != nil { ... } // will be nil if embedded Roundtripper was not of type http.Transport

func NewDefaultClient

func NewDefaultClient(opts ...Option) *http.Client

NewDefaultClient returns a default http client with retry functionality wrapped around the Roundtripper (client.Transport).

You should not replace the client.Transport field, otherwise you will lose the retry functionality.

If you need to set / change the original client.Transport field you have two options:

  1. create your own http client and use NewCustomClient() function to enrich the client with retry functionality. client := &http.Client{} client.Transport = &http.Transport{ ... } retryClient := httpretry.NewCustomClient(client)
  2. use one of the helper functions (e.g. httpretry.ModifyOriginalTransport(retryClient)) to retrieve and change the Transport. retryClient := httpretry.NewDefaultClient() err := httpretry.ModifyOriginalTransport(retryClient, func(t *http.Transport){t.TLSHandshakeTimeout = 5 * time.Second}) if err != nil { ... } // will be nil if embedded Roundtripper was not of type http.Transport

func ReplaceOriginalRoundtripper added in v0.12.0

func ReplaceOriginalRoundtripper(client *http.Client, roundtripper http.RoundTripper) error

ReplaceOriginalRoundtripper replaces the original roundtripper that was embedded in the retry roundtripper

Types

type BackoffPolicy

type BackoffPolicy func(attemptCount int) time.Duration

BackoffPolicy is used to calculate the time to wait, before executing another request.

The backoff can be calculated by taking the current number of retries into consideration.

type Option

type Option func(*RetryRoundtripper)

Option is a function type to modify the RetryRoundtripper configuration

func WithBackoffPolicy

func WithBackoffPolicy(backoffPolicy BackoffPolicy) Option

WithBackoffPolicy sets the user defined backoff policy.

Default: ExponentialBackoff(1*time.Second, 30*time.Second, 200*time.Millisecond)

func WithMaxRetryCount

func WithMaxRetryCount(maxRetryCount int) Option

WithMaxRetryCount sets the maximum number of retries if an http request was not successful.

Default: 5

func WithRetryPolicy

func WithRetryPolicy(retryPolicy RetryPolicy) Option

WithRetryPolicy sets the user defined retry policy.

Default: RetryPolicy checks for some common errors that are likely not retryable and for status codes that should be retried.

For example:

  • url parsing errors
  • too many redirects
  • certificate errors
  • BadGateway
  • ServiceUnavailable
  • etc.

type RetryPolicy added in v0.12.0

type RetryPolicy func(statusCode int, err error) bool

RetryPolicy decides if a request should be retried.

This is done by examining the response status code and the error message of the last response.

The statusCode may be 0 if there was no response available (e.g. in case of a request error).

type RetryRoundtripper

type RetryRoundtripper struct {
	Next             http.RoundTripper
	MaxRetryCount    int
	ShouldRetry      RetryPolicy
	CalculateBackoff BackoffPolicy
}

RetryRoundtripper is the roundtripper that will wrap around the actual http.Transport roundtripper to enrich the http client with retry functionality.

func (*RetryRoundtripper) RoundTrip

func (r *RetryRoundtripper) RoundTrip(req *http.Request) (*http.Response, error)

RoundTrip implements the actual roundtripper interface (http.RoundTripper).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL