README
¶
eth2-comply
eth2-comply is a no-code platform for Ethereum 2.0 API conformance testing. It provides blackbox conformance testing facilities and a simple syntax for specifying test cases.
Install
Check the release page for the latest eth2-comply binaries and test files.
Usage
eth2-comply --target http://localhost:5051
CLI arguments:
--testsRootPath to a directory tree on the filesystem containing JSON test cases.--testsRemoteURL to a zip file containing a valid tests directory tree.--testsRemotetakes precedence over--testsRootif both are specidied.--outDirLocation on the filesystem to download and unpack a zip file specified in--testsRemote. Has no meaning if--testsRemoteis not specified.--targetURL of any appliance serving the Ethereum 2.0 API.--timeoutTime after which to abandon waiting tests. Defaults to 10 minutes. Uses Go duration syntax.--subsetThe subset of paths to run tests for. For example, set this to "/v1/node" to only run tests for routes in that path. Defaults to "/" (all paths).--failSilentWhen true, return a 0 code even when tests fail. Defaults to false.
Syntax of test cases
Tests are specified as normal JSON objects. Only one test should be specified per file. Test files should be placed in the appropriate directory according to the API route they activate. Files should be named <expected-status-code>_<seq_num>.json where seq_num is just a unique number to prevent the file name from colliding with any other test case in that directory testing for the same expected status code in that route. The names and filepaths of test files have no bearing on how eth2-comply processes them, this topology is just an organizational methodology.
The following JSON fields have meaning in eth2-comply test case syntax. Any fields not listed have no meaning to eth2-comply and are ignored.
| field | required | value type | example |
|---|---|---|---|
| method | yes | string | "GET" |
| route | yes | string | "/beacon/committees" |
| reqBody | no | object | {"epoch": "0", "pubkeys": ["0xdeadbeef"]} |
| queryParams | no | object | {"epoch": "0"} |
| awaitSlot | no | int | 2666 |
| expectedRespStatus | no | int | 200 |
| expectedRespBody | no | object | [{"slot": "0", "index": "0", "committee": []}] |
Most of the fields meaning should be self-explanatory. awaitSlot can be used to make eth2-comply wait until the target node has synced the specified slot before executing the test.
When specifying expected response bodies, know that received and expected responses are canonicalized before being compared. This means that whitespace and key order do not matter in general. Remember that list order does matter; the way a list is specified literally is its canonical form, though nested objects are themselves canonicalized.
Build and run while developing
Build:
make build
Run:
bazel run -- cmd/eth2-comply:eth2-comply --target http://localhost:5051 --testsRoot $(PWD)/tests
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
pkg
|
|
|
oapi
package oapi primarily provides executors for Ethereum 2.0 API operations.
|
package oapi primarily provides executors for Ethereum 2.0 API operations. |
|
target
package target includes utilities for ascertaining the state of the target node, namely whether it is healthy and is ready to serve requests, and what slot it has synced to.
|
package target includes utilities for ascertaining the state of the target node, namely whether it is healthy and is ready to serve requests, and what slot it has synced to. |
|
testcases
package testcases defines the Case data structure, and a public function for instantiating slices of Cases from JSON test specifications in directory trees on the filesystem.
|
package testcases defines the Case data structure, and a public function for instantiating slices of Cases from JSON test specifications in directory trees on the filesystem. |