go-bb

go-bb

go-bb, short for Go Benchmark Binary, is a tool to extract code from traditional Go benchmark functions into independent executable with minimal overhead. The resulting executable can then be ran with traditional tools such as valgrind or perf, without being hit by the overhead of the test harness of Go test binaries.

This is a POC. It just works on my computer.

Install

go install github.com/pelletier/go-bb

Usage

Usage of go-bb:
  -n string
    	Regexp that matches the name of the Benchmark* function. Needs to match exactly one function.
  -no-src-cleanup
    	If true, do not clean up the temporary source directory.
  -o string
    	Path of the resulting binary.
  -p string
    	Path to a folder that contains the benchmark code (can be in any sub folder).

Example

$ go-bb -p ./example -n Me
Found matching function: BenchmarkMe (example_test.go)
Temporary source directory: /tmp/go-bb-2656927681
Copying from /home/thomas/src/github.com/pelletier/go-bb/example -> /tmp/go-bb-2656927681/bborig
Copied /home/thomas/src/github.com/pelletier/go-bb/example/example_test.go -> /tmp/go-bb-2656927681/bborig/example_test.go
Rewriting benchmark function
Renaming test files
Initializing module example.com/go-bb-2656927681
Running tidy
Compiling
Benchmark binary ready at /home/thomas/src/github.com/pelletier/go-bb/benchmark.binary
$ perf stat -- ./benchmark.binary

Comparison with go test

For the binary above, here's perf running on the binary generated by go-bb:

$ perf stat -- ./benchmark.binary

 Performance counter stats for './benchmark.binary':

              1.08 msec task-clock:u              #    0.816 CPUs utilized
                 0      context-switches:u        #    0.000 K/sec
                 0      cpu-migrations:u          #    0.000 K/sec
               299      page-faults:u             #    0.276 M/sec
           805,764      cycles:u                  #    0.743 GHz                      (75.16%)
             3,186      stalled-cycles-frontend:u #    0.40% frontend cycles idle
             2,022      stalled-cycles-backend:u  #    0.25% backend cycles idle
           860,990      instructions:u            #    1.07  insn per cycle
                                                  #    0.00  stalled cycles per insn
           147,157      branches:u                #  135.702 M/sec
             5,533      branch-misses:u           #    3.76% of all branches          (25.37%)

       0.001329578 seconds time elapsed

       0.000000000 seconds user
       0.001613000 seconds sys

And here is the equivalent binary generated by go test:

$ go test -c ./example && perf stat ./example.test -test.run=None -test.bench=Me -test.benchtime=1x -test.count=1
goos: linux
goarch: amd64
pkg: github.com/pelletier/go-bb/example
cpu: AMD Ryzen 9 5950X 16-Core Processor
BenchmarkMe-32    	       1	       230.0 ns/op	 347.83 MB/s
PASS

 Performance counter stats for './example.test -test.run=None -test.bench=Me -test.benchtime=1x -test.count=1':

              4.04 msec task-clock:u              #    1.499 CPUs utilized
                 0      context-switches:u        #    0.000 K/sec
                 0      cpu-migrations:u          #    0.000 K/sec
               625      page-faults:u             #    0.155 M/sec
         5,639,931      cycles:u                  #    1.396 GHz                      (67.27%)
            33,278      stalled-cycles-frontend:u #    0.59% frontend cycles idle
           126,317      stalled-cycles-backend:u  #    2.24% backend cycles idle
         8,628,210      instructions:u            #    1.53  insn per cycle
                                                  #    0.01  stalled cycles per insn
         1,952,375      branches:u                #  483.298 M/sec
            40,313      branch-misses:u           #    2.06% of all branches          (33.71%)

       0.002695165 seconds time elapsed

       0.002980000 seconds user
       0.001480000 seconds sys

go-bb has about 86% less cycles than the go test binary.

License

MIT (LICENSE)