Documentation ¶
Overview ¶
Package benchparse provides utilities for parsing benchmark results. Parsed results are split by sub-benchmarks, with support for sub-benchmarks with names of the form 'var_name=var_value'
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNotMeasured = errors.New("not measured")
ErrNotMeasured indicates that a specific output was not measured.
Functions ¶
This section is empty.
Types ¶
type BenchInputs ¶
type BenchInputs struct { VarValues []BenchVarValue // sub-benchmark names of the form some_var=some_val Subs []BenchSub // remaining components of a sub-benchmark MaxProcs int // the value of GOMAXPROCS when the benchmark was run }
BenchInputs define a sub-benchmark. For example a benchmark with a full name 'BenchmarkMyType/some_method/foo=2/bar=baz-4' would be defined by the Subs=[some_method], the VarValues=[foo=2 bar=baz], and MaxProcs=4.
func (BenchInputs) String ¶
func (b BenchInputs) String() string
String returns the string representation of the BenchInputs. This should be equivalent to the portion of the benchmark name following the name of the top-level benchmark, but formatting of VarValues may vary slightly.
type BenchOutputs ¶
type BenchOutputs interface { GetIterations() int GetNsPerOp() (float64, error) GetAllocedBytesPerOp() (uint64, error) // measured if either '-test.benchmem' is set of if testing.B.ReportAllocs() is called GetAllocsPerOp() (uint64, error) // measured if either '-test.benchmem' is set of if testing.B.ReportAllocs() is called GetMBPerS() (float64, error) // measured if testing.B.SetBytes() is called }
BenchOutputs are the outputs of a single benchmark run.
Since not all output values are measured on each benchmark run, the getter for these values will return ErrNotMeasured if this is the case.
type BenchRes ¶
type BenchRes struct { Inputs BenchInputs // the input variables Outputs BenchOutputs // the output result }
BenchRes represents a result from a single benchmark run. This corresponds to one line from the testing.B output.
type BenchResults ¶ added in v0.4.0
type BenchResults []BenchRes
BenchResults represents a list of benchmark results
func (BenchResults) Filter ¶ added in v0.4.0
func (b BenchResults) Filter(filterExpr string) (BenchResults, error)
Filter returns a subset of the BenchResults matching the provided filter expr. For example filtering by the expression 'var1<=2' will return the results where the input variable named 'var1' has a value less than or equal to 2.
Example ¶
r := strings.NewReader(` BenchmarkMath/areaUnder/y=sin(x)/delta=0.001000/start_x=-2/end_x=1/abs_val=true-4 21801 55357 ns/op 0 B/op 0 allocs/op BenchmarkMath/areaUnder/y=2x+3/delta=1.000000/start_x=-1/end_x=2/abs_val=false-4 88335925 13.3 ns/op 0 B/op 0 allocs/op BenchmarkMath/max/y=2x+3/delta=0.001000/start_x=-2/end_x=1-4 56282 20361 ns/op 0 B/op 0 allocs/op BenchmarkMath/max/y=sin(x)/delta=1.000000/start_x=-1/end_x=2-4 16381138 62.7 ns/op 0 B/op 0 allocs/op `) benches, err := ParseBenchmarks(r) if err != nil { log.Fatal(err) } filtered, err := benches[0].Results.Filter("y==sin(x)") if err != nil { log.Fatal(err) } for _, res := range filtered { nsPerOp, err := res.Outputs.GetNsPerOp() if err != nil { log.Fatal(err) } fmt.Printf("ns per op = %v\n", nsPerOp) }
Output: ns per op = 55357 ns per op = 62.7
func (BenchResults) Group ¶ added in v0.4.0
func (b BenchResults) Group(groupBy []string) GroupedResults
Group groups a benchmarks results by a specified set of input variable names. For example a Benchmark with Results corresponding to the cases [/foo=1/bar=baz /foo=2/bar=baz /foo=1/bar=qux /foo=2/bar=qux] grouped by ['foo'] would have 2 groups of results (those with Inputs where
Example ¶
r := strings.NewReader(` BenchmarkMath/areaUnder/y=sin(x)/delta=0.001000/start_x=-2/end_x=1/abs_val=true-4 21801 55357 ns/op 0 B/op 0 allocs/op BenchmarkMath/areaUnder/y=2x+3/delta=1.000000/start_x=-1/end_x=2/abs_val=false-4 88335925 13.3 ns/op 0 B/op 0 allocs/op BenchmarkMath/max/y=2x+3/delta=0.001000/start_x=-2/end_x=1-4 56282 20361 ns/op 0 B/op 0 allocs/op BenchmarkMath/max/y=sin(x)/delta=1.000000/start_x=-1/end_x=2-4 16381138 62.7 ns/op 0 B/op 0 allocs/op `) benches, err := ParseBenchmarks(r) if err != nil { log.Fatal(err) } groupedResults := benches[0].Results.Group([]string{"y"}) // sort by key names to ensure consistent iteration order groupNames := make([]string, len(groupedResults)) i := 0 for k := range groupedResults { groupNames[i] = k i++ } sort.Strings(groupNames) for _, k := range groupNames { fmt.Println(k) v := groupedResults[k] times := make([]float64, len(v)) for i, res := range v { nsPerOp, err := res.Outputs.GetNsPerOp() if err != nil { log.Fatal(err) } times[i] = nsPerOp } fmt.Printf("ns per op = %v\n", times) }
Output: y=2x+3 ns per op = [13.3 20361] y=sin(x) ns per op = [55357 62.7]
type BenchSub ¶
type BenchSub struct { Name string // contains filtered or unexported fields }
BenchSub represents an input to the benchmark represented by a sub-benchmark with a name NOT of the form 'var_name=var_value'.
type BenchVarValue ¶
type BenchVarValue struct { Name string Value interface{} // contains filtered or unexported fields }
BenchVarValue represents an input to the benchmark represented by a sub-benchmark with a name of the form 'var_name=var_value'.
func (BenchVarValue) String ¶
func (b BenchVarValue) String() string
String returns the string representation of the BenchVarValue with the form 'var_name=var_value'.
The string representation of a BenchVarValue may vary slightly from the original input due to things like floating point precision and alternate string representations of various types.
Currently the '%f' verb is used for floating point values in order to guarantee that they can be distinguished from integer values. For everything else the default '%v' verb is used for simplicities sake.
type Benchmark ¶
type Benchmark struct { Name string Results BenchResults }
Benchmark represents a single top-level benchmark and it's results.
func ParseBenchmarks ¶
ParseBenchmarks extracts a list of Benchmarks from testing.B output.
Example ¶
r := strings.NewReader(` BenchmarkMath/areaUnder/y=sin(x)/delta=0.001000/start_x=-2/end_x=1/abs_val=true-4 21801 55357 ns/op 0 B/op 0 allocs/op BenchmarkMath/areaUnder/y=2x+3/delta=1.000000/start_x=-1/end_x=2/abs_val=false-4 88335925 13.3 ns/op 0 B/op 0 allocs/op BenchmarkMath/max/y=2x+3/delta=0.001000/start_x=-2/end_x=1-4 56282 20361 ns/op 0 B/op 0 allocs/op BenchmarkMath/max/y=sin(x)/delta=1.000000/start_x=-1/end_x=2-4 16381138 62.7 ns/op 0 B/op 0 allocs/op `) benches, err := ParseBenchmarks(r) if err != nil { log.Fatal(err) } for _, bench := range benches { fmt.Printf("bench name: %s\n", bench.Name) for _, res := range bench.Results { var ( varValues = make([]string, len(res.Inputs.VarValues)) otherSubs = make([]string, len(res.Inputs.Subs)) ) for i, varVal := range res.Inputs.VarValues { varValues[i] = varVal.String() } for i, sub := range res.Inputs.Subs { otherSubs[i] = sub.String() } fmt.Printf("var values = %q\n", varValues) fmt.Printf("other subs = %q\n", otherSubs) nsPerOp, err := res.Outputs.GetNsPerOp() if err != nil { log.Fatal(err) } fmt.Printf("ns per op = %.2f\n", nsPerOp) } }
Output: bench name: BenchmarkMath var values = ["y=sin(x)" "delta=0.001000" "start_x=-2" "end_x=1" "abs_val=true"] other subs = ["areaUnder"] ns per op = 55357.00 var values = ["y=2x+3" "delta=1.000000" "start_x=-1" "end_x=2" "abs_val=false"] other subs = ["areaUnder"] ns per op = 13.30 var values = ["y=2x+3" "delta=0.001000" "start_x=-2" "end_x=1"] other subs = ["max"] ns per op = 20361.00 var values = ["y=sin(x)" "delta=1.000000" "start_x=-1" "end_x=2"] other subs = ["max"] ns per op = 62.70
type Comparison ¶ added in v0.4.0
type Comparison string
Comparison represents a comparison operation.
const ( Eq Comparison = "==" Ne Comparison = "!=" Lt Comparison = "<" Gt Comparison = ">" Le Comparison = "<=" Ge Comparison = ">=" )
The available comparison operations.
type GroupedResults ¶
type GroupedResults map[string]BenchResults
GroupedResults represents a grouping of benchmark results.