Container Orchestration Benchmark
The purpose of the container orchestration benchmark (cnbm-co
for short) is to provide a vendor-neutral, extendable benchmark for container orchestration systems. The current focus is on stateless workloads and we're implementing it for the following container orchestration systems (targets):
If you want to contribute, simply fork this repo, add your implementation in pkg/
and send in a PR.
Contents:
Using
Launching
In general:
$ ./cnbm-co launch -h
Launches the CNBM container orchestration benchmark
Usage:
cnbm-co launch [flags]
Flags:
-h, --help help for launch
-p, --params string Comma separated key-value pair list of target-specific configuration parameters. For example: k1=v1,k2=v2
-r, --runtype string The benchmark run type. Allowed values: [scaling distribution apicalls servicediscovery recovery]
-t, --target string The target container orchestration system to benchmark. Allowed values: [dcos kubernetes]
Global Flags:
--config string config file (default is $HOME/.cnbm.yaml)
DC/OS
$ ./cnbm-co launch --runtype scaling --target dcos -p dcosurl=http://example.com,dcosacstoken=123
INFO[0000] Setting up DC/OS scaling benchmark
INFO[0000] Executing DC/OS scaling benchmark
INFO[0000] Deploying a new application
INFO[0000] RESULT:
Target: DC/OS
Output: {}
Elapsed time: 0s
Kubernetes
To benchmark a Kubernetes cluster, use --target kubernetes
, for example, to launch the scaling
run type, do:
$ ./cnbm-co launch --runtype scaling --target kubernetes --params kubeconfig=/Users/mhausenblas/.kube/config,ns=cnbm,numpods=3,cpu=50m,mem=300Mi
INFO[0000] Setting up Kubernetes scaling benchmark
INFO[0000] Executing Kubernetes scaling benchmark
INFO[0006] Tearing down Kubernetes scaling benchmark
INFO[0008] RESULT:
Target: Kubernetes
Output: {benchmark succeeded}
Elapsed time: 6.752147169s
Note the following params that are required, depending on the --runtype
:
scaling
kubeconfig
… the Kubernetes config to use, for example /Users/mhausenblas/.kube/config
ns
… the namespace to use, for example cnbm
numpods
… the number of pods to launch, for example, 10
cpu
… the CPU seconds (at least 40m
) per pod
mem
… the memory (at least 300Mi
) per pod
servicediscovery
kubeconfig
… the Kubernetes config to use, for example /Users/mhausenblas/.kube/config
ns
… the namespace to use, for example cnbm
Availability matrix
The following matrix shows the availability of benchmark run types per target:
benchmark run type |
DC/OS |
Kubernetes |
scaling |
Y |
Y |
distribution |
N |
N |
distribution |
N |
N |
apicalls |
N |
N |
servicediscovery |
N |
N |
Developing
Building
$ make
Building the CNBM CO CLI
go build -ldflags "-X github.com/cnbm/container-orchestration/cli/cmd.releaseVersion=0.1.0" -o ./cnbm-co cli/main.go
Vendoring
We are using Go dep for dependency management. If you don't have dep
installed yet, do go get -u github.com/golang/dep/cmd/dep
now and then:
$ dep ensure
Testing
For unit tests we use the go test
command, for example:
$ go test -v -short -run Test* .