README
¶
Middleware
Retry
Retries a request on http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout or configurable http error code
The http request object is wrapped to buffer and ensure the request data stream is still available on the retry. Error responses a matching
error code are discarded when there is a retry option left
Peek model
Optimistic approach to read the model name from the request payload when it is the first attribute of a json object. The request data is buffered so
that no data is lost for any following step. The model is stored in the X-Model so that upstream processes can read it from there.
When the model attribute is not the first or not read within the buffered data, the result is empty.
Example request payload:
{
"model": "my-model",
"messages": []
}
goos: darwin
goarch: arm64
pkg: github.com/alpe/netutils/common
BenchmarkDecoding/medium/PeekModel-12 1624220 726.0 ns/op
BenchmarkDecoding/medium/JsonDecoder-12 8212 144236 ns/op
BenchmarkDecoding/big/PeekModel-12 1661097 727.7 ns/op
BenchmarkDecoding/big/JsonDecoder-12 433 2735448 ns/op
BenchmarkDecoding/minimal/PeekModel-12 1779588 678.8 ns/op
BenchmarkDecoding/minimal/JsonDecoder-12 2108025 565.9 ns/op
BenchmarkDecoding/small/PeekModel-12 1651448 722.1 ns/op
BenchmarkDecoding/small/JsonDecoder-12 1299910 921.3 ns/op
Documentation
¶
Index ¶
Constants ¶
const XModelHeader = "X-Model"
Variables ¶
This section is empty.
Functions ¶
func MustRegisterRetryMetrics ¶
func MustRegisterRetryMetrics(r prometheus.Registerer)
MustRegisterRetryMetrics registers the totalRetries metric with the provided prometheus.Registerer. It is a convenience function that ensures the metric is registered successfully, and panics if registration fails.
func NewRetryMiddleware ¶
func NewRetryMiddleware(maxRetries int, other http.Handler, optRetryStatusCodes ...int) http.Handler
NewRetryMiddleware creates a new HTTP middleware that adds retry functionality. It takes the maximum number of retries, the next handler in the middleware chain, and an optional list of retryable status codes. If the maximum number of retries is 0, it returns the next handler without adding any retries. If the list of retryable status codes is empty, it uses a default set of status codes (http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout). The function creates a RetryMiddleware struct with the given parameters and returns it as a http.Handler.
Types ¶
type CapturedBodySource ¶
CapturedBodySource represents an interface for capturing and retrieving the body of an HTTP request.
type PeekModelMiddleware ¶
type PeekModelMiddleware struct {
// contains filtered or unexported fields
}
PeekModelMiddleware implements an optimistic approach to read the model name from the request payload when it is the first attribute of a json object payload. This may not always be the case so the downstream processes need to take care of the payload handling themselves.
If the X-Model header is already set, it will not modify the request.
func NewPeekModelMiddleware ¶
func NewPeekModelMiddleware(nextHandler http.Handler, bufferSize int) *PeekModelMiddleware
NewPeekModelMiddleware is the constructor for the PeekModelMiddleware type. It takes a http.Handler as a parameter, which is the next handler in the middleware chain. The bufferSize defines the number of bytes read to fetch the model attribute.
func (*PeekModelMiddleware) ServeHTTP ¶
func (p *PeekModelMiddleware) ServeHTTP(resp http.ResponseWriter, req *http.Request)
ServeHTTP handle the request.
type RetryMiddleware ¶
type RetryMiddleware struct {
// contains filtered or unexported fields
}
func (RetryMiddleware) ServeHTTP ¶
func (r RetryMiddleware) ServeHTTP(resp http.ResponseWriter, req *http.Request)
ServeHTTP handles the HTTP request by capturing the request body, calling the next handler in the chain, and retrying if necessary. It captures the request body using a LazyBodyCapturer, and sets a captured response writer using NewDiscardableResponseWriter. It retries the request if the response was discarded and the response status code is retryable. It uses exponential backoff for retries with a random jitter. The maximum number of retries is determined by the maxRetries field of RetryMiddleware.
type StatusCodeCapturer ¶
type StatusCodeCapturer interface {
CapturedStatusCode() int
}
type XBodyCapturer ¶
type XBodyCapturer interface {
io.ReadCloser
CapturedBodySource
Capture()
}
func NewLazyBodyCapturer ¶
func NewLazyBodyCapturer(body io.ReadCloser) XBodyCapturer
NewLazyBodyCapturer constructor.
type XResponseWriter ¶
type XResponseWriter interface {
http.ResponseWriter
StatusCodeCapturer
}
func NewDiscardableResponseWriter ¶
func NewDiscardableResponseWriter(writer http.ResponseWriter, isDiscardable func(status int) bool) XResponseWriter
NewDiscardableResponseWriter creates a new instance of the response writer delegator. It takes a http.ResponseWriter and a function to determine by the status code if content should be written or discarded (for retry).
Source Files