README
¶
xDS
Background
xDS is the set of discovery services and APIs used by Envoy to discover its dynamic resources.
xDS Server
Gloo Edge is an xDS server. It maintains a snapshot-based, in-memory cache and responds to xDS requests with the resources that are requested.
Snapshot
A snapshot is a versioned group of resources. In Gloo Edge, we rely on an Envoy snapshot, a snapshot of the xDS resources that Gloo serves to Envoy.
Snapshot Cache
A snapshot cache maintains a single versioned snapshot per key. It also responds to open xDS requests.
xDS Callbacks
xDS callbacks are a set of callbacks that are invoked asynchronously during the lifecycle of an xDS request.
Gloo Edge open source does not define any xDS callbacks. However, these callbacks are a type of extension that can be injected at runtime. Gloo Edge Enterprise defines xDS callbacks, and injects them into the Control Plane at runtime.
Server
An xDS server defines a set of handlers for streaming discovery requests.
xDS Services
The xDS server is configured to expose the following discovery services in Gloo Edge:
ListenerDiscoveryService
The ListenerDiscoveryService allows Envoy to discover Listeners at runtime.
RouteDiscoveryService
The RouteDiscoveryService allows Envoy to discover routing configuration for an HttpConnectionManager filter at runtime.
ClusterDiscoveryService
The ClusterDiscoveryService allows Envoy to discover routable destinations at runtime.
EndpointDiscoveryService
The EndpointDiscoveryService allows Envoy to discover members in a cluster at runtime.
AggregatedDiscoveryService
The AggregatedDiscoveryService allows Envoy to discover all resource types over a single stream at runtime.
SoloDiscoveryService
The SoloDiscoveryService is a custom xDS service, used to serve resources of Any type, that is based on Envoy's Aggregated Discovery Service.
In addition to serving configuration for Envoy resources, the Gloo xDS server is also responsible for serving configuration to a number of enterprise extensions (ie ext-auth and rate-limit)
The SoloDiscoveryService is required to serve these extension resources. It is largely based on the Envoy v2 API, and since it is purely an internal API, we do not need to upgrade the API to match the Envoy xDS API. This issue contains additional context around the reason behind this custom discovery service.
xDS Requests
Gloo Edge supports managing configuration for multiple proxies through a single xDS server. To do so, it stores each snapshot in the cache at a key that is unique to that proxy.
To guarantee that proxies initiate requests for the snapshot they want, it is critical that we establish a naming pattern for cache keys. This pattern must be used both by the proxies requesting the resources from the cache, and the controllers that set the resources in the cache.
The naming convention that we follow is "NAMESPACE~NAME"
Proxies identify the cache key that they are interested in by specifying their node.metadata.role to the cache key using the above naming pattern. An example of this can be found in the bootstrap configuration for proxies
xDS Debugging
Debugging xDS behavior can be challenging, below are a few techniques to help:
Control Plane Logging
The Gloo translation loop is responsible for converting a Gloo API Snapshot into an xDS Snapshot. After completing a translation loop, there is a log line indicating what key in the snapshot cache was updated.
xDS Debug Endpoint
Gloo supports running in dev mode, and when that is enabled, xDS Snapshots are exposed via an endpoint.
Useful information
Documentation
¶
Overview ¶
Package server provides an implementation of a streaming xDS server.
Package server provides an implementation of a streaming xDS server.
Index ¶
- Constants
- func NewAdsSnapshotCache(ctx context.Context) cache.SnapshotCache
- func NewEndpointsSnapshotFromResources(endpoints cache.Resources, clusters cache.Resources) cache.Snapshot
- func NewNodeRoleHasher() *nodeRoleHasher
- func NewSnapshotFromResources(endpoints cache.Resources, clusters cache.Resources, routes cache.Resources, ...) cache.Snapshot
- func SetEdsOnCluster(out *envoy_config_cluster_v3.Cluster, settings *v1.Settings)
- func SetupEnvoyXds(grpcServer *grpc.Server, xdsServer envoyserver.Server, ...)
- func SnapshotCacheKey(proxy *v1.Proxy) string
- func SnapshotCacheKeys(proxies v1.ProxyList) []string
- type EnvoyServerV3
- type EnvoySnapshot
- type GlooXdsServer
Constants ¶
const FallbackNodeCacheKey = "misconfigured-node"
FallbackNodeCacheKey is used to let nodes know they have a bad config we assign a "fix me" snapshot for bad nodes
Variables ¶
This section is empty.
Functions ¶
func NewAdsSnapshotCache ¶ added in v1.9.25
func NewAdsSnapshotCache(ctx context.Context) cache.SnapshotCache
NewAdsSnapshotCache returns a snapshot-based cache, used to serve xDS requests
func NewEndpointsSnapshotFromResources ¶ added in v1.6.16
func NewNodeRoleHasher ¶ added in v1.9.25
func NewNodeRoleHasher() *nodeRoleHasher
func SetEdsOnCluster ¶ added in v0.13.16
func SetEdsOnCluster(out *envoy_config_cluster_v3.Cluster, settings *v1.Settings)
SetEdsOnCluster marks an Envoy Cluster to receive its Endpoints from the xDS Server https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/upstream/service_discovery#arch-overview-service-discovery-types-eds In Gloo, we support both streaming (gRPC) and polling (REST)
NOTE: REST EDS was introduced as a way of bypassing https://github.com/envoyproxy/envoy/issues/13070 a bug in Envoy that would cause clusters to warm, without endpoints. That bug has since been resolved and gRPC EDS is preferred as the polling solution is more resource intensive and could delay updates by as much as 5 seconds (or whatever the refresh delay is)
func SetupEnvoyXds ¶
func SetupEnvoyXds(grpcServer *grpc.Server, xdsServer envoyserver.Server, envoyCache envoycache.SnapshotCache)
register xDS methods with GRPC server
func SnapshotCacheKey ¶ added in v1.9.25
SnapshotCacheKey returns the key used to identify a Proxy resource in a SnapshotCache This key must match the node.metadata.role of the Envoy Node
func SnapshotCacheKeys ¶ added in v1.9.25
SnapshotCacheKeys returns a list with the SnapshotCacheKey for each Proxy
Types ¶
type EnvoyServerV3 ¶ added in v1.6.0
type EnvoyServerV3 interface {
envoy_service_endpoint_v3.EndpointDiscoveryServiceServer
envoy_service_cluster_v3.ClusterDiscoveryServiceServer
envoy_service_route_v3.RouteDiscoveryServiceServer
envoy_service_listener_v3.ListenerDiscoveryServiceServer
envoy_service_discovery_v3.AggregatedDiscoveryServiceServer
}
Server is a collection of handlers for streaming discovery requests.
func NewEnvoyServerV3 ¶ added in v1.6.0
func NewEnvoyServerV3(genericServer server.Server) EnvoyServerV3
type EnvoySnapshot ¶
type EnvoySnapshot struct {
// Endpoints are items in the EDS V3 response payload.
Endpoints cache.Resources
// Clusters are items in the CDS response payload.
Clusters cache.Resources
// Routes are items in the RDS response payload.
Routes cache.Resources
// Listeners are items in the LDS response payload.
Listeners cache.Resources
}
Snapshot is an internally consistent snapshot of xDS resources. Consistently is important for the convergence as different resource types from the snapshot may be delivered to the proxy in arbitrary order.
func NewSnapshot ¶
func NewSnapshot( version string, endpoints []cache.Resource, clusters []cache.Resource, routes []cache.Resource, listeners []cache.Resource, ) *EnvoySnapshot
NewSnapshot creates a snapshot from response types and a version.
func (*EnvoySnapshot) Clone ¶ added in v0.15.0
func (s *EnvoySnapshot) Clone() cache.Snapshot
func (*EnvoySnapshot) Consistent ¶
func (s *EnvoySnapshot) Consistent() error
Consistent check verifies that the dependent resources are exactly listed in the snapshot: - all EDS resources are listed by name in CDS resources - all RDS resources are listed by name in LDS resources
Note that clusters and listeners are requested without name references, so Envoy will accept the snapshot list of clusters as-is even if it does not match all references found in xDS.
func (*EnvoySnapshot) Equal ¶ added in v1.6.0
func (this *EnvoySnapshot) Equal(that *EnvoySnapshot) bool
Equal checks is 2 snapshots are equal, important since reflect.DeepEqual no longer works with proto4
func (*EnvoySnapshot) GetResources ¶
func (s *EnvoySnapshot) GetResources(typ string) cache.Resources
GetResources selects snapshot resources by type.
func (*EnvoySnapshot) MakeConsistent ¶ added in v1.9.25
func (s *EnvoySnapshot) MakeConsistent()
MakeConsistent removes any items that fail to link to parent resources in the snapshot. It will also add placeholder routes for listeners referencing non-existent routes.
type GlooXdsServer ¶ added in v1.8.0
type GlooXdsServer interface {
discovery_service.SoloDiscoveryServiceServer
}
Server includes handlers for streaming aggregate discovery requests via the SoloDiscoveryService.
func NewGlooXdsServer ¶ added in v1.8.0
func NewGlooXdsServer(genericServer server.Server) GlooXdsServer
Source Files