lbv2

type LoadBalancer ¶

type LoadBalancer struct {
	// contains filtered or unexported fields
}

The LoadBalancer object makes decisions about where traffic should go. It contains the list of active backend nodes (updated asynchronously by calling Update), and a list of filters and policies used to select a backend for every incoming request.

Filters look at each available backend node and assign them a score, possibly depending on the incoming request. The policy then picks a backend among the available nodes based on their score. By convention, it is possible to remove a node from the list of candidates by setting its score to zero.

Combinations of simple filters can implement relatively complex traffic control behaviors: from straightforward round-robin to geoip latency minimization to capacity-aware, utilization-based load balancing.

The LoadBalancer keeps track of how many requests were sent to each backend node, to compute an up-to-date estimation of its current utilization. This should help mitigate "thundering herd" and "laser death ray" scenarios caused by delays in the utilization feedback loop.

The computational model makes some generic assumptions about incoming traffic, the results will be better the more real traffic actually matches these assumptions. First, all requests are assumed to be identical, utilization-wise: there is a single query cost metric. While this implies that the accuracy of the cost estimation for each specific query may be low, the effect evens out with large numbers as long as the statistical distribution of request types varies little over time. Secondly, since the estimation logic is local to each frontend, the model assumes that each frontend receives an equal share of incoming traffic.

func New(name string) *LoadBalancer

func (l *LoadBalancer) AddFilter(f NodeFilter)

func (l *LoadBalancer) Choose(ctx RequestContext) (Node, error)

func (l *LoadBalancer) DisableUtilizationPredictors()

func (l *LoadBalancer) GetPredictor(dimension int) Predictor

func (l *LoadBalancer) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (l *LoadBalancer) SetPolicy(p Policy)

func (l *LoadBalancer) Update(nodes NodeList)

type Node ¶

type Node interface {
	// Name of the node (unique identifier).
	Name() string

	// Current utilization for the specified dimension.
	Utilization(int) NodeUtilization
}

type NodeFilter ¶

type NodeFilter interface {
	Filter(RequestContext, []NodeScore) []NodeScore
}

Node filters operate on the list of available nodes (and associated weights), possibly modifying it in-place. They can be composed out of simpler parts, like individual nodeScorers wrapped by an nodeScorerFilter (when no request-specific initialization step is required).

func NewActiveNodesFilter() NodeFilter

func NewCapacityAvailableFilter(pred Predictor) NodeFilter

func NewCapacityAvailableScorer(pred Predictor) NodeFilter

func NodeScorerFilter(s NodeScorer) NodeFilter

type NodeList ¶

type NodeList interface {
	Len() int
	Get(int) Node
}

NodeList is a wrapper for a container of Nodes.

type NodeScore ¶

type NodeScore struct {
	Score float64
	Node  Node
}

func (c NodeScore) SetScore(w float64) NodeScore

type NodeScorer ¶

type NodeScorer interface {
	Score(RequestContext, Node) float64
}

Score a single node.

type NodeScorerFunc ¶

type NodeScorerFunc func(RequestContext, Node) float64

func (f NodeScorerFunc) Score(ctx RequestContext, n Node) float64

type NodeUtilization ¶

type NodeUtilization struct {
	// Utilization is a number between 0 and 1.
	Utilization float64

	// Some request-related metric. It doesn't really matter
	// whether it is a counter or a gauge, as long as it is
	// roughly proportional to the utilization.
	Requests int
}

Node utilization along a specific dimension. Utilization is treated as a vector, the LoadBalancer is opaque to the actual meaning of the dimensions used (though it makes sense for Requests to be the same across dimensions).

type Policy ¶

type Policy interface {
	GetNode([]NodeScore) Node
}

type PolicyFunc ¶

type PolicyFunc func([]NodeScore) Node

func (f PolicyFunc) GetNode(wnodes []NodeScore) Node

type Predictor ¶

type Predictor interface {
	Utilization(Node) float64
}

A Predictor estimates utilization along a specific dimension using a (relatively simple) cost query model.

type RequestContext ¶

type RequestContext interface {
	RemoteAddr() net.IP
}

Provides contextual information on the incoming request.