testground

Here contains test plans to be run on Testground.

Setup

git clone https://github.com/testground/testground.git
cd testground
#  compile Testground and all related dependencies
make install
#  start the Testground daemon, listening by default on localhost:8042
testground daemon

Run below from another terminal window. This will add tests here to the testground home directory, which is $HOME/testground by default, unless you explicitly define $TESTGROUND_HOME.

testground plan import --from . --name go-threads

Run

Then you can run the prepared composition plans as follows:

#  This one sticks to an early version of go-threads
testground run composition --wait -f baseline-docker.toml
#  This one always test against the current head of go-threads
testground run composition --wait -f head-docker.toml
#  This one uses whatever version specified by go.mod in the current directory. Do check/update dependencies before running this.
testground run composition --wait -f current-docker.toml
#  This one is good for local test while developing.
testground run composition --wait -f current-exec.toml

You can also run individual test case and pass test parameters in command line. For example, below builds the test as native executables, runs 2 instances, with very verbose logs printed to the console. See manifest.toml for all test parameters.

testground run single --wait -p go-threads -t sync-threads -b exec:go -r local:exec -i 2 -tp verbose=2

Analysize

1. Check for errors running the tests
2. Check the recorded metrics and compare them with a baseline run. run_id is shown at both the begining and the end of the run.

cd $HOME/testground/data/outputs/local_docker/go-threads/<run_id>
find . -name "results.out" | xargs grep elapsed-seconds | sort -t ':' -k 4

Develop

To run these tests after making some changes to go-threads, do the following:

1. Commit you changes and push to a branch
2. GOPRIVATE=* go get github.com/textileio/go-threads@<your branch>
3. testground run composition --wait -f current-docker.toml and have a look at the results (See section Analysize)
4. Commit changes to the branch