gopayloader

Gopayloader

Gopayloader is an HTTP/S benchmarking tool. Inspired by bombardier it also uses fasthttp which allows for fast creation and sending of requests due to low allocations and lots of other improvements. It uses this client by default, a different client can be used with --client flag.

Supports all HTTP versions, using quic-go for HTTP/3 client with --client nethttp3. For HTTP/2 can use with --client nethttp2. By default uses fasthttp HTTP/1.1 client.

Supports ability to generate custom JWTs to send in headers with payload (only limited by HDD size). This can be useful if the service being tested is JWT authenticated. Each JWT generated will be unique as contains a unique jti in claims i.e.

{
  "jti": "8f2d1472-084c-4662-ae74-04e0f1de4993"
}

A private key is supplied as a flag with optional flags to set other standard claims i.e. sub aud, iss and can also provide custom claims. It will then check if the required number of jwts has already been generated in a previous test by checking a file on disk. All JWTs are saved on disk in cache, this allows huge number of jwts to be generated without affecting in-memory use of gopayloader. Once all jwts have been generated the tests begin, and jwts are streamed from disk to requests. This keeps the memory footprint low. The other major benefit to pre-generating is all of CPU cycles can be dedicated to sending the requests thus achieving higher RPS.

Contributing

Contributions are welcome, please read the guidelines here.

Installation

Can install with (supported go versions >= 1.20)

go install github.com/domsolutions/gopayloader@latest 

Or download pre-compiled binaries from releases

Benchmark comparisons

All tests are running against below HTTP/1.1 server;

./gopayloader http-server -p 8081 -s 1 --fasthttp-1

Tested running for 30 seconds reqs over 125 connections

Gopayloader tested with

./gopayloader run http://localhost:8081 -c 125 --time 30s 

achieved mean RPS of 53,098

Usage

To list all available flags run;

Load test HTTP/S server - supports HTTP/1.1 HTTP/2 HTTP/3

Usage:
  gopayloader run <host>(host format - protocol://host:port/path i.e. https://localhost:443/some-path) [flags]

Flags:
  -b, --body string              request body
      --body-file string         read request body from file
      --client string            fasthttp for fast http/1.1 requests
                                 nethttp for standard net/http requests using http/1.1
                                 nethttp2 for standard net/http requests using http/2
                                 nethttp3 for standard net/http requests supporting http/3 using quic-go (default "fasthttp")
  -c, --connections uint         Number of simultaneous connections (default 1)
  -k, --disable-keep-alive       Disable keep-alive connections
  -H, --headers strings          headers to send in request, can have multiple i.e -H 'content-type:application/json' -H' connection:close'
  -h, --help                     help for run
      --jwt-aud string           JWT audience (aud) claim
      --jwt-claims string        JWT custom claims
      --jwt-header string        JWT header field name
      --jwt-iss string           JWT issuer (iss) claim
      --jwt-key string           JWT signing private key path
      --jwt-kid string           JWT KID
      --jwt-sub string           JWT subject (sub) claim
  -f, --jwts-filename string     File path for pre-generated JWTs, separated by new lines
  -m, --method string            request method (default "GET")
      --mtls-cert string         mTLS cert path
      --mtls-key string          mTLS cert private key path
      --parallel                 Sends reqs in parallel per connection with HTTP/2
      --read-timeout duration    Read timeout (default 5s)
  -r, --requests int             Number of requests
      --skip-verify              Skip verify SSL cert signer
      --ticker duration          How often to print results while running in verbose mode (default 1s)
  -t, --time duration            Execution time window, if used with -r will uniformly distribute reqs within time window, without -r reqs are unlimited
  -v, --verbose                  verbose - slows down RPS slightly for long running tests
      --write-timeout duration   Write timeout (default 5s)

By default, it runs in quiet mode to dedicate all CPU cycles to sending requests to achieve max RPS. Verbose mode can be enabled with -v flag.

To run 1000000 requests across 150 connections;

./gopayloader run http://localhost:8081 -c 150 -r 1000000 

Gopayloader v0.1.0 HTTP/JWT authentication benchmark tool 
https://github.com/domsolutions/gopayloader

 INFO  Running 1,000,000 request/s with 150 connection/s against http://localhost:8081
 SUCCESS  Payload complete, calculating results
 SUCCESS  Gopayloader results 

+-----------------------+-------------------------------+
| METRIC                | RESULT                        |
+-----------------------+-------------------------------+
| Total time            | 18.983561852s                 |
| Start time            | Thu, 27 Apr 2023 12:04:47 BST |
| End time              | Thu, 27 Apr 2023 12:05:06 BST |
| Completed requests    | 1000000                       |
| Failed requests       | 0                             |
+-----------------------+-------------------------------+
| Average RPS           | 52677.153                     |
| Max RPS               | 54411                         |
| Min RPS               | 47096                         |
+-----------------------+-------------------------------+
| Req size (bytes)      | 42                            |
| Req size/second (MB)  | 2.225                         |
| Req total size (MB)   | 40.054                        |
+-----------------------+-------------------------------+
| Resp size (bytes)     | 135                           |
| Resp size/second (MB) | 7.153                         |
| Resp total size (MB)  | 128.746                       |
+-----------------------+-------------------------------+
| Average latency       | 2.816219ms                    |
| Max latency           | 62.938092ms                   |
| Min latency           | 74.879µs                      |
+-----------------------+-------------------------------+
| Response code; 200    | 1000000                       |
+-----------------------+-------------------------------+

To run 1000000 requests across 150 connections with jwts (jwts will only be generated when number of requests are specified);

Example header jwt generated;

{
  "alg": "ES256",
  "kid": "3434645743124",
  "typ": "JWT"
}

example body;

{
  "aud": "some-audience",
  "exp": 1714130039,
  "iss": "some-issuer",
  "jti": "05181473-bbd6-4d21-8942-d86c2e972b2b",
  "sub": "my-subject",
  "iat": 1719410063,
  "browser": "chrome"
}

Note jti will be different for each jwt.

Will set jwt value = header field my-jwt and sign with key ./private-key.pem and KID 3434645743124

./gopayloader run http://localhost:8081 -c 150 -r 1000000 --jwt-header "my-jwt" --jwt-key ./private-key.pem --jwt-kid 3434645743124 --jwt-sub "my-subject" --jwt-aud "some-audience" --jwt-iss "some-issuer" --jwt-claims "{\"iat\": 1719410063, \"browser\": \"chrome\"}"

Gopayloader v0.1.0 HTTP/JWT authentication benchmark tool 
https://github.com/domsolutions/gopayloader

 INFO  Sending jwts with requests, checking for jwts in cache
 INFO  Generating batch of 1000000 JWTs and saving to disk
 INFO  Running 1,000,000 request/s with 150 connection/s against http://localhost:8081
 SUCCESS  Payload complete, calculating results
 SUCCESS  Gopayloader results 

+-----------------------+-------------------------------+
| METRIC                | RESULT                        |
+-----------------------+-------------------------------+
| Total time            | 19.679817482s                 |
| Start time            | Thu, 27 Apr 2023 12:14:48 BST |
| End time              | Thu, 27 Apr 2023 12:15:08 BST |
| Completed requests    | 1000000                       |
| Failed requests       | 0                             |
+-----------------------+-------------------------------+
| Average RPS           | 50813.479                     |
| Max RPS               | 52708                         |
| Min RPS               | 48098                         |
+-----------------------+-------------------------------+
| Req size (bytes)      | 370                           |
| Req size/second (MB)  | 18.572                        |
| Req total size (MB)   | 352.859                       |
+-----------------------+-------------------------------+
| Resp size (bytes)     | 135                           |
| Resp size/second (MB) | 6.776                         |
| Resp total size (MB)  | 128.746                       |
+-----------------------+-------------------------------+
| Average latency       | 2.92205ms                     |
| Max latency           | 78.047387ms                   |
| Min latency           | 80.768µs                      |
+-----------------------+-------------------------------+
| Response code; 200    | 1000000                       |
+-----------------------+-------------------------------+

If you have your own JWTs you want to test, you can supply a file to send the JWTs i.e. ./my-jwts.txt where each jwt is separated by a new line.

./gopayloader run http://localhost:8081 -c 1 -r 1000000 --jwt-header "my-jwt" -f ./my-jwts.txt

To remove all generated jwts;

./gopayloader clear-cache