README
¶
Sandboxes for remote Salvo executions
This directory contains definition of Salvo sandboxes. A sandbox is the collection of resources needed for a single Salvo execution. These resources are deployed in AWS. At the minimum a sandbox will contain:
- a VM running Nighthawk (the load generator).
- a VM running Envoy (the system under test or SUT).
- a VM running the Nighthawk test server (the test backend).
Note that the subnets used between the components are part of the common infrastructure and are already deployed as part of the VPC configuration done in the ci-infra repository.
Various sandbox types differ in the amount of instances deployed, amount of support systems deployed and the topology.
Available Sandbox types
The default sandbox
The default sandbox contains the bare minimum of components needed to execute
Salvo tests. The diagram below outlines the topology and content of the default
sandbox. The default sandbox is currently available for the x64 architecture
only.
Sandbox creation
When Salvo executes, an execution of the AZP CI pipeline is started. Each CI pipeline execution is uniquely identified by a build ID. This execution will produce binaries and VM disk images for all the sandbox components.
Once the binaries and VM disk images are built, the AZP CI pipeline starts a
job in the salvo-control agent pool on AZP. This job is then picked up by a
Salvo control VM running a Salvo controller that instruments the sandbox
creation.
The Salvo controller uses the Terraform templates found in this directory, to deploy the sandbox components in AWS.
Sandbox instance life-cycle
The Terraform configuration for sandboxes uses shared state bucket deployed in S3 on AWS. The sandbox instances that will be deployed or destroyed are determined based on the variables passed to Terraform.
When a sandbox is starting up, it needs to locate the binaries and VM disk images that were produced by the AZP CI pipeline. This is achieved by providing the sandbox with the build ID that uniquely identifies the AZP CI pipeline execution that produced them.
Each of the following variables is a list of these build IDs. Each list represents one sandbox type. Each listed build ID represents an instance of the sandbox type that should be deployed.
Supported sandbox types:
default_sandbox_x64_build_ids.
For example the Terraform command to deploy a single instance of the default
sandbox with build ID 136112 is:
terraform apply --var="default_sandbox_build_ids=[\"136112\"]"
Any instances that were deployed, but are not named in the variables passed to
Terraform will be destroyed when Terraform is executed. To remove all deployed
sandboxes, simply execute terraform apply in this directory with no
arguments.
terraform apply
Documentation
¶
Overview ¶
instances.go defined data types that described deployed sandbox instances.
json.go contains functions that unmarshal JSON messages received from Terraform.
Package sandboxes manages the life cycle of a Salvo remote sandbox.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Instance ¶
type Instance struct {
// Type identifies the type of the sandbox instance.
Type Type
// LoadGenerators are the load generator VMs deployed in the sandbox.
LoadGenerators []*VM
// SUTs are the system under test VMs deployed in the sandbox.
SUTs []*VM
// Backends are the backend VMs deployed in the sandbox.
Backends []*VM
}
Instance describes a single sandbox instance.
type Instances ¶
type Instances []int64
Instances represent sandbox instances, each integer is an AZP Build ID corresponding to AZP pipeline execution that built the components for the sandbox.
type Manager ¶
type Manager struct {
// contains filtered or unexported fields
}
Manager manages sandbox instances.
func NewManager ¶
NewManager creates a new Sandbox manager.
type Option ¶
type Option interface {
// contains filtered or unexported methods
}
Option is used to provide options to NewManager.
type Type ¶
type Type int
Type enumerates the types of sandboxes supported.
const (
// TypeDefaultSandboxX64 is the default sandbox, see https://github.com/envoyproxy/envoy-perf/tree/main/salvo-remote/sandboxes/terraform/default_sandbox_x64.
TypeDefaultSandboxX64 Type
)
type VM ¶
type VM struct {
// IP is the primary private IP address of the VM.
// This is the IP address on the interface that either originates or receives load.
IP net.IP
// ControlIP is a private IP address that can be used by the control VM when accessing the VM.
// Note, the control VM is the VM that runs salvo-remote (this code).
ControlIP net.IP
}
VM describes a VM deployed in a sandbox.
Source Files