protospec

protospec

protospec is an extensible conformance testing tool for internet protocols. The tests are called 'spec' and can be written declaratively in YAML format and shared as OCI Artifacts.

This project is currently in the 'pathfinder' phase. Please be aware that incompatible changes may be made without notice.

Install

Please download the latest binary from the release page and place it in the path of your choice.

The container image can be downloaded as follows.

$ docker pull ghcr.io/summerwind/protospec:latest

Usage

To run protospec, you first need to download test files called 'spec bundle'. You can then run the tests with the spec bundle.

Here is an example of how to download and run the tests of RFC7540.

Downloading the spec bundle

The spec bundle can be downloaded using the protospec pull command as follows. This command will download it to the spec directory under the current directory.

$ protospec pull ghcr.io/summerwind/protospec-bundle-http2-rfc7540

Running tests using the spec bundle

You can now run the tests using the spec bundle you downloaded. In this example, you will run the test against an HTTP/2 server running on port 8080 of localhost.

$ protospec run -p 8080 ./spec

When you run the test, you will get the following results.

3.5: HTTP/2 Connection Preface
#1: Sends client connection preface
#2: Sends invalid connection preface

4.1: Frame Format
#1: Sends a frame with unknown type
#2: Sends a frame with undefined flag
#3: Sends a frame with reserved field bit

...

8.2: Server Push
#1: Sends a PUSH_PROMISE frame

94 tests, 93 passed, 0 skipped, 1 failed

You can use the -t option to run only specific tests.

$ protospec run -p 8080 -t 3.5/1 ./spec

License

protospec is made available under MIT license.