httplb

package module
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 19, 2024 License: Apache-2.0 Imports: 21 Imported by: 3

README

httplb

Build Report Card GoDoc

httplb provides client-side load balancing for net/http clients. By default, clients are designed for server-to-server and RPC workloads:

  • They support HTTP/1.1, HTTP/2, and H2C.
  • They periodically re-resolve names using DNS.
  • They use a round-robin load balancing policy.

Random, fewest-pending, and power-of-two load balancing policies are also available. Clients with more complex needs can customize the underlying transports, name resolution, and load balancing. They can also add subsetting and active health checking.

httplb takes care to build all this functionality underneath the standard library's *http.Client, so httplb is usable anywhere you're currently using net/http.

Example

Here's a quick example of how to get started with httplb:

package main

import (
	"log"
	"net"
	"time"

	"github.com/bufbuild/httplb"
	"github.com/bufbuild/httplb/picker"
)

func main() {
	client := httplb.NewClient(
		// Switch from the default round-robin policy to power-of-two.
		httplb.WithPicker(picker.NewPowerOfTwo),
	)
	defer client.Close()
	resp, err := client.Get("https://example.com")
	if err != nil {
		log.Fatalln(err)
	}
	defer resp.Body.Close()
	log.Println(resp.Status)
}

If you're using Connect, you can use httplb for your RPC clients:

func main() {
	client := httplb.NewClient()
	defer client.Close()
	pingClient := pingv1connect.NewPingServiceClient(
		client,
		"http://localhost:8080/",
	)
	req := connect.NewRequest(&pingv1.PingRequest{
		Number: 42,
	})
	res, err := pingClient.Ping(context.Background(), req)
	if err != nil {
		log.Fatalln(err)
	}
	log.Println(res.Msg)
}

If you know the server supports HTTP/2 without TLS (HTTP/2 over cleartext, or H2C for short), use the h2c scheme in your URLs instead of http:

	pingClient := pingv1connect.NewPingServiceClient(
		client,
		"h2c://localhost:8080/",
	)

For more information on how to use httplb, especially for advanced use cases, take a look at the full documentation.

Ecosystem

  • connect-go: RPC library, compatible with httplb.

Status: Alpha

This project is currently in alpha. The API should be considered unstable and likely to change.

Offered under the Apache 2 license.

Documentation

Overview

Package httplb provides http.Client instances that are suitable for use for server-to-server communications, like RPC. This adds features on top of the standard net/http library for name/address resolution, health checking, and load balancing. It also provides more suitable defaults and much simpler support for HTTP/2 over plaintext.

To create a new client use the NewClient function. This function accepts numerous options, many for configuring the behavior of the underlying transports. It also provides options for using a custom name resolver or a custom load balancing policy and for enabling active health checking.

The returned client also has a notion of "closing", via its Close method. This step will wait for outstanding requests to complete and then close all connections and also teardown any other goroutines that it may have started to perform name resolution and health checking. The client cannot be used after it has been closed.

Default Behavior

Without any options, the returned client behaves differently than http.DefaultClient in the following key ways:

  1. The client will re-resolve addresses in DNS every 5 minutes. The http.DefaultClient does not re-resolve predictably.

  2. The client will route requests in a round-robin fashion to all addresses returned by the DNS system (both A and AAAA records). even with HTTP/2.

    This differs from the http.DefaultClient, which will use only a single connection if it can, even if DNS resolves many addresses. With HTTP 1.1, an http.DefaultClient will create additional connections to handle multiple concurrent requests (since an HTTP 1.1 connection can only service one request at a time). But with HTTP/2, it likely will *never* use additional connections: it only creates another connection if the concurrency limit exceeds the server's "max concurrent streams" (which the server provides when the connection is initially established and is typically on the order of 100).

The above behavior alone should make the httplb.Client distribute load to backends in a much more appropriate way, especially when using HTTP/2. But the real power of a client returned by this package is that it can be customized with different name resolution and load balancing policies, via the WithResolver and WithPicker options. Active health checking can be enabled via the WithHealthChecks option.

Transport Architecture

The clients created by this function use a transport implementation that is actually a hierarchy of three layers:

  1. The "root" of the hierarchy (which is the actual Transport field value of an http.Client returned by NewClient) serves as a collection of transport pools, grouped by URL scheme and hostname:port. For a given request, it examines the URL's scheme and hostname:port to decide which transport pool should handle the request.
  2. The next level of the hierarchy is the transport pool. A transport pool manages multiple transports that all correspond to the same URL scheme and hostname:port. This is the layer in which most of the features are implemented, like name resolution and load balancing. Each transport pool manages a pool of "leaf" transports, each to a potentially different resolved address.
  3. The bottom of the hierarchy, the "leaf" transports, are just http.RoundTripper implementations that each represent a single, logical connection to a single resolved address. A leaf transport could actually consist of more than one *physical* connection, like when using HTTP 1.1 and multiple active requests are made to the same leaf transport. This level of transport is often an *http.Transport, that handles physical connections to the remote host and implements the actual HTTP protocol.

Custom Transports

One of the options in this package, WithTransport, allows users to implement custom transports and select them using custom URL schemes. This could be used, for example, to enable HTTP/3 with a URL such as "http3://...".

This package even uses this capability to provide simple use of HTTP/2 over plaintext, also called "h2c". In addition to supporting "http" and "https" URL schemes, this package also supports "h2c" as a URL scheme.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	*http.Client
}

Client is an HTTP client that supports configurable client-side load balancing, name resolution, and subsetting. It embeds the standard library's *http.Client and exposes some additional operations.

If working with a library that requires an *http.Client, use the embedded Client. If working with a library that requires an http.RoundTripper, use the Transport field.

func NewClient 

func NewClient(options ...ClientOption) *Client

NewClient constructs a new HTTP client optimized for server-to-server communication. By default, the client re-resolves addresses every 5 minutes and uses a round-robin load balancing policy.

func (*Client) Close 

func (c *Client) Close() error

Close the HTTP client, releasing any resources and stopping any associated background goroutines.

type ClientOption 

type ClientOption interface {
	// contains filtered or unexported methods
}

ClientOption is an option used to customize the behavior of an HTTP client that is created via NewClient.

func WithAllowBackendTarget 

func WithAllowBackendTarget(scheme, hostPort string) ClientOption

WithAllowBackendTarget configures the client to only allow requests to the given target (identified by URL scheme and host:port).

When configured this way, connections to the backend target will be kept warm, meaning that associated transports will not be closed due to inactivity, regardless of the idle transport timeout configuration.

The scheme and host:port given must match those of associated requests. So if requests omit the port from the URL (for example), then the hostPort given here should also omit the port.

func WithDefaultTimeout 

func WithDefaultTimeout(duration time.Duration) ClientOption

WithDefaultTimeout limits requests that otherwise have no timeout to the given timeout. Unlike WithRequestTimeout, if the request's context already has a deadline, then no timeout is applied. Otherwise, the given timeout is used and applies to the entire duration of the request, from sending the first request byte to receiving the last response byte.

func WithDialer 

func WithDialer(dialFunc func(ctx context.Context, network, addr string) (net.Conn, error)) ClientOption

WithDialer configures the HTTP client to use the given function to establish network connections. If no WithDialer option is provided, a default net.Dialer is used that uses a 30-second dial timeout and configures the connection to use TCP keep-alive every 30 seconds.

func WithHealthChecks 

func WithHealthChecks(checker health.Checker) ClientOption

WithHealthChecks configures the HTTP client to use the given health checker. This provides details about which resolved addresses are healthy or not.

If no such option is given, then no health checking is done and all connections will be considered healthy.

func WithIdleConnectionTimeout 

func WithIdleConnectionTimeout(duration time.Duration) ClientOption

WithIdleConnectionTimeout configures a timeout for how long an idle connection will remain open. If zero or no WithIdleConnectionTimeout option is used, idle connections will be left open indefinitely. If backend servers or intermediary proxies/load balancers place time limits on idle connections, this should be configured to be less than that time limit, to prevent the client from trying to use a connection could be concurrently closed by a server for being idle for too long.

func WithIdleTransportTimeout 

func WithIdleTransportTimeout(duration time.Duration) ClientOption

WithIdleTransportTimeout configures a timeout for how long an idle transport will remain available. Transports are created per target host. When a transport is closed, all underlying connections are also closed.

This differs from WithIdleConnectionTimeout in that it is for managing client resources, to prevent the underlying set of transports from growing too large if the HTTP client is used for dynamic outbound requests. Whereas WithIdleConnectionTimeout is to coordinate with servers that close idle connections.

If zero or no WithIdleTransportTimeout option is used, a default of 15 minutes will be used.

To prevent transports from being closed due to being idle, set an arbitrarily large timeout (i.e. math.MaxInt64) or use WithAllowBackendTarget.

func WithMaxResponseHeaderBytes 

func WithMaxResponseHeaderBytes(limit int) ClientOption

WithMaxResponseHeaderBytes configures the maximum size of response headers to consume. If zero or if no WithMaxResponseHeaderBytes option is used, the HTTP client will default to a 1 MB limit (2^20 bytes).

func WithNoProxy 

func WithNoProxy() ClientOption

WithNoProxy returns an option that disables use of HTTP proxies.

func WithPicker 

func WithPicker(newPicker func(prev picker.Picker, allConns conn.Conns) picker.Picker) ClientOption

WithPicker configures the HTTP client to use the given function to create picker instances. The factory is invoked to create a new picker every time the set of usable connections changes for a target.

func WithProxy 

func WithProxy(
	proxyFunc func(*http.Request) (*url.URL, error),
	proxyConnectHeadersFunc func(ctx context.Context, proxyURL *url.URL, target string) (http.Header, error),
) ClientOption

WithProxy configures how the HTTP client interacts with HTTP proxies for reaching remote hosts.

The given proxyFunc returns the URL of a proxy server to use for the given HTTP request. If no proxy should be used, it should return nil, nil. If an error is returned, the request fails immediately with that error. If this function is set to nil or no WithProxy option is provided, http.ProxyFromEnvironment will be used as the proxyFunc. (Also see WithNoProxy.)

The given onProxyConnectFunc, if non-nil, provides a way to examine the response from the proxy for a CONNECT request. If the onProxyConnectFunc returns an error, the request will fail immediately with that error.

The given proxyConnectHeadersFunc, if non-nil, provides a way to supply extra request headers to the proxy for a CONNECT request. The target provided to this function is the "host:port" to which to connect. If no extra headers should be added to the request, the function should return nil, nil. If the function returns an error, the request will fail immediately with that error.

func WithRedirects 

func WithRedirects(redirectFunc RedirectFunc) ClientOption

WithRedirects configures how the HTTP client handles redirect responses. If no such option is provided, the client will not follow any redirects.

func WithRequestTimeout 

func WithRequestTimeout(duration time.Duration) ClientOption

WithRequestTimeout limits all requests to the given timeout. This time is the entire duration of the request, including sending the request, writing the request body, waiting for a response, and consuming the response body.

func WithResolver 

func WithResolver(res resolver.Resolver) ClientOption

WithResolver configures the HTTP client to use the given resolver, which is how hostnames are resolved into individual addresses for the underlying connections.

If not provided, the default resolver will resolve A and AAAA records using net.DefaultResolver.

func WithRootContext 

func WithRootContext(ctx context.Context) ClientOption

WithRootContext configures the root context used for any background goroutines that an HTTP client may create. If not specified, context.Background is used.

If the given context is cancelled (or times out), many functions of the HTTP client may fail to operate correctly. It should only be cancelled after the HTTP client is no longer in use, and may be used to eagerly free any associated resources.

func WithRoundTripperMaxLifetime added in v0.2.0 

func WithRoundTripperMaxLifetime(duration time.Duration) ClientOption

WithRoundTripperMaxLifetime configures a limit for how long a single round tripper (or "leaf" transport) will be used. If no option is given, round trippers are retained indefinitely, until their parent transport is closed (which can happen if the transport is idle; see WithIdleTransportTimeout). When a round tripper reaches its maximum lifetime, a new one is first created to replace it. Any in-progress operations are allowed to finish, but no new operations will use it.

This function is mainly useful when the target host is a layer-4 proxy. In this situation, it is possible for multiple round trippers to all get connected to the same backend host, resulting in poor load distribution. With a lifetime limit, a single round tripper will get "recycled", and its replacement is likely to be connected to a different backend host. So when a transport gets into a scenario where it has poor backend diversity, the lifetime limit allows it to self-heal.

func WithTLSConfig 

func WithTLSConfig(config *tls.Config, handshakeTimeout time.Duration) ClientOption

WithTLSConfig adds custom TLS configuration to the HTTP client. The given config is used when using TLS to communicate with servers. The given timeout is applied to the TLS handshake step. If the given timeout is zero or no WithTLSConfig option is used, a default timeout of 10 seconds will be used.

func WithTransport 

func WithTransport(scheme string, transport Transport) ClientOption

WithTransport returns an option that uses a custom transport to create http.RoundTripper instances for the given URL scheme. This allows one to override the default implementations for "http", "https", and "h2c" schemes and also allows one to support custom URL schemes that map to these custom transports.

type RedirectFunc 

type RedirectFunc func(req *http.Request, via []*http.Request) error

RedirectFunc is a function that advises an HTTP client on whether to follow a redirect. The given req is the redirected request, based on the server's previous status code and "Location" header, and the given via is the set of requests already issued, each resulting in a redirect. The via slice is sorted oldest first, so the first element is the always the original request and the last element is the latest redirect.

See FollowRedirects.

func FollowRedirects 

func FollowRedirects(limit int) RedirectFunc

FollowRedirects is a helper to create a RedirectFunc that will follow up to the given number of redirects. If a request sequence results in more redirects than the given limit, the request will fail.

type RoundTripperResult 

type RoundTripperResult struct {
	// RoundTripper is the actual round-tripper that handles requests.
	RoundTripper http.RoundTripper
	// Scheme, if non-empty, is the scheme to use for requests to RoundTripper. This
	// replaces the request's original scheme. This is useful when a custom scheme
	// is used to trigger a custom transport, but the underlying RoundTripper still
	// expects a non-custom scheme, such as "http" or "https".
	Scheme string
	// Close is an optional function that will be called (if non-nil) when this
	// round-tripper is no longer needed.
	Close func()
	// contains filtered or unexported fields
}

RoundTripperResult is the result type created by a Transport. The contained RoundTripper represents a "leaf" transport used for sending requests.

type Transport 

type Transport interface {
	// NewRoundTripper creates a new [http.RoundTripper] for requests using the
	// given scheme to the given host, per the given config.
	NewRoundTripper(scheme, target string, config TransportConfig) RoundTripperResult
}

Transport is used to create round trippers that handle requests to a single resolved address.

type TransportConfig 

type TransportConfig struct {
	// DialFunc should be used by the round-tripper establish network connections.
	DialFunc func(ctx context.Context, network, addr string) (net.Conn, error)
	// ProxyFunc should be used to control HTTP proxying behavior. If the function
	// returns a non-nil URL for a given request, that URL represents the HTTP proxy
	// that should be used.
	ProxyFunc func(*http.Request) (*url.URL, error)
	// ProxyConnectHeadersFunc should be called, if non-nil, before sending a CONNECT
	// request, to query for headers to add to that request. If it returns an
	// error, the round-trip operation should fail immediately with that error.
	ProxyConnectHeadersFunc func(ctx context.Context, proxyURL *url.URL, target string) (http.Header, error)
	// MaxResponseHeaderBytes configures the maximum size of the response status
	// line and response headers.
	MaxResponseHeaderBytes int64
	// IdleConnTimeout, if non-zero, is used to expire idle network connections.
	IdleConnTimeout time.Duration
	// TLSClientConfig, is present, provides custom TLS configuration for use
	// with secure ("https") servers.
	TLSClientConfig *tls.Config
	// TLSHandshakeTimeout configures the maximum time allowed for a TLS handshake
	// to complete.
	TLSHandshakeTimeout time.Duration
	// KeepWarm indicates that the round-tripper should try to keep a ready
	// network connection open to reduce any delays in processing a request.
	KeepWarm bool
}

TransportConfig defines the options used to create a round-tripper.

Source Files

View all Source files

Directories

Path Synopsis
Package attribute provides a type-safe container of custom attributes named Values.
 Package attribute provides a type-safe container of custom attributes named Values.
Package conn provides the representation of a logical connection.
 Package conn provides the representation of a logical connection.
Package health provides pluggable health checking for an httplb.Client.
 Package health provides pluggable health checking for an httplb.Client.
balancertesting
Package balancertesting provides some helper functions and types that can be useful when testing custom load balancer implementations.
 Package balancertesting provides some helper functions and types that can be useful when testing custom load balancer implementations.
clocktest
Package clocktest exists to allow interoperability with our Clock interface and the Clockwork FakeClock.
 Package clocktest exists to allow interoperability with our Clock interface and the Clockwork FakeClock.
conns
Package conns contains internal helpers relating to conn.Conns values.
 Package conns contains internal helpers relating to conn.Conns values.
Package picker provides functionality for picking a connection.
 Package picker provides functionality for picking a connection.
Package resolver provides functionality for custom name resolution.
 Package resolver provides functionality for custom name resolution.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.