container-orchestration

Container Orchestration Benchmark

The purpose of the container orchestration benchmark (cnbm-co for short) is to provide a vendor-neutral, extendable benchmark for container orchestration systems. The current focus is on stateless workloads and we're implementing it for the following container orchestration systems (targets):

• DC/OS
• Kubernetes

If you want to contribute, simply fork this repo, add your implementation in pkg/ and send in a PR.

Contents:

• Using
  • Launching
  • Availability matrix
• Developing
• Design

Using

Launching

In general:

$ ./cnbm-co launch -h
Launches the CNBM container orchestration benchmark

Usage:
  cnbm-co launch [flags]

Flags:
  -h, --help             help for launch
  -p, --params string    Comma separated key-value pair list of target-specific configuration parameters. For example: k1=v1,k2=v2
  -r, --runtype string   The benchmark run type. Allowed values: [scaling distribution apicalls servicediscovery recovery]
  -t, --target string    The target container orchestration system to benchmark. Allowed values: [dcos kubernetes]

Global Flags:
      --config string   config file (default is $HOME/.cnbm.yaml)

DC/OS

$ ./cnbm-co launch --runtype scaling --target dcos -p dcosurl=http://example.com,dcosacstoken=123
INFO[0000] Setting up DC/OS scaling benchmark
INFO[0000] Executing DC/OS scaling benchmark
INFO[0000] Deploying a new application
INFO[0000] RESULT:
 Target: DC/OS
 Output: {}
 Elapsed time: 0s

Kubernetes

To benchmark a Kubernetes cluster, use --target kubernetes, for example, to launch the scaling run type, do:

$ ./cnbm-co launch --runtype scaling --target kubernetes --params kubeconfig=/Users/mhausenblas/.kube/config,ns=cnbm,numpods=3,cpu=50m,mem=300Mi
INFO[0000] Setting up Kubernetes scaling benchmark
INFO[0000] Executing Kubernetes scaling benchmark
INFO[0006] Tearing down Kubernetes scaling benchmark
INFO[0008] RESULT:
 Target: Kubernetes
 Output: {benchmark succeeded}
 Elapsed time: 6.752147169s

Note the following params that are required, depending on the --runtype:

• scaling
  • kubeconfig … the Kubernetes config to use, for example /Users/mhausenblas/.kube/config
  • ns … the namespace to use, for example cnbm
  • numpods … the number of pods to launch, for example, 10
  • cpu … the CPU seconds (at least 40m) per pod
  • mem … the memory (at least 300Mi) per pod
• servicediscovery
  • kubeconfig … the Kubernetes config to use, for example /Users/mhausenblas/.kube/config
  • ns … the namespace to use, for example cnbm

Availability matrix

The following matrix shows the availability of benchmark run types per target:

Developing

Building

$ make
Building the CNBM CO CLI
go build -ldflags "-X github.com/cnbm/container-orchestration/cli/cmd.releaseVersion=0.1.0" -o ./cnbm-co cli/main.go

Vendoring

We are using Go dep for dependency management. If you don't have dep installed yet, do go get -u github.com/golang/dep/cmd/dep now and then:

$ dep ensure

Testing

For unit tests we use the go test command, for example:

$ go test -v -short -run Test* .