perf: Index | Files

package benchstat

import ""


Package Files

data.go delta.go html.go scaler.go sort.go table.go text.go


var (
    ErrSamplesEqual = errors.New("all equal")
    ErrSampleSize   = errors.New("too few samples")
    ErrZeroVariance = errors.New("zero variance")

Errors returned by DeltaTest.

func ByDelta Uses

func ByDelta(t *Table, i, j int) bool

ByDelta sorts tables by the Delta column, reversing the order when larger is better (for "speed" results).

func ByName Uses

func ByName(t *Table, i, j int) bool

ByName sorts tables by the Benchmark name column

func FormatCSV Uses

func FormatCSV(w io.Writer, tables []*Table, norange bool)

FormatCSV appends a CSV formatting of the tables to w. norange suppresses the range columns.

func FormatHTML Uses

func FormatHTML(buf *bytes.Buffer, tables []*Table)

FormatHTML appends an HTML formatting of the tables to buf.

func FormatText Uses

func FormatText(w io.Writer, tables []*Table)

FormatText appends a fixed-width text formatting of the tables to w.

func NoDeltaTest Uses

func NoDeltaTest(old, new *Metrics) (pval float64, err error)

NoDeltaTest applies no delta test; it returns -1, nil.

func Sort Uses

func Sort(t *Table, order Order)

Sort sorts a Table t (in place) by the given order.

func TTest Uses

func TTest(old, new *Metrics) (pval float64, err error)

TTest is a DeltaTest using the two-sample Welch t-test.

func UTest Uses

func UTest(old, new *Metrics) (pval float64, err error)

UTest is a DeltaTest using the Mann-Whitney U test.

type Collection Uses

type Collection struct {
    // Configs, Groups, and Units give the set of configs,
    // groups, and units from the keys in Stats in an order
    // meant to match the order the benchmarks were read in.
    Configs, Groups, Units []string

    // Benchmarks gives the set of benchmarks from the keys in
    // Stats by group in an order meant to match the order
    // benchmarks were read in.
    Benchmarks map[string][]string

    // Metrics holds the accumulated metrics for each key.
    Metrics map[Key]*Metrics

    // DeltaTest is the test to use to decide if a change is significant.
    // If nil, it defaults to UTest.
    DeltaTest DeltaTest

    // Alpha is the p-value cutoff to report a change as significant.
    // If zero, it defaults to 0.05.
    Alpha float64

    // AddGeoMean specifies whether to add a line to the table
    // showing the geometric mean of all the benchmark results.
    AddGeoMean bool

    // SplitBy specifies the labels to split results by.
    // By default, results will only be split by full name.
    SplitBy []string

    // Order specifies the row display order for this table.
    // If Order is nil, the table rows are printed in order of
    // first appearance in the input.
    Order Order

A Collection is a collection of benchmark results.

func (*Collection) AddConfig Uses

func (c *Collection) AddConfig(config string, data []byte)

AddConfig adds the benchmark results in the formatted data to the named configuration. If the input is large, AddFile should be preferred, since it avoids the need to read a copy of the entire raw input into memory.

func (*Collection) AddFile Uses

func (c *Collection) AddFile(config string, r io.Reader) error

AddFile adds the benchmark results in the formatted data (read from the reader r) to the named configuration.

func (*Collection) AddResults Uses

func (c *Collection) AddResults(config string, results []*benchfmt.Result)

AddResults adds the benchmark results to the named configuration.

func (*Collection) Tables Uses

func (c *Collection) Tables() []*Table

Tables returns tables comparing the benchmarks in the collection.

type DeltaTest Uses

type DeltaTest func(old, new *Metrics) (float64, error)

A DeltaTest compares the old and new metrics and returns the expected probability that they are drawn from the same distribution.

If a probability cannot be computed, the DeltaTest returns an error explaining why. Common errors include ErrSamplesEqual (all samples are equal), ErrSampleSize (there aren't enough samples), and ErrZeroVariance (the sample has zero variance).

As a special case, the missing test NoDeltaTest returns -1, nil.

type Key Uses

type Key struct {
    Config, Group, Benchmark, Unit string

A Key identifies one metric (e.g., "ns/op", "B/op") from one benchmark (function name sans "Benchmark" prefix) and optional group in one configuration (input file name).

type Metrics Uses

type Metrics struct {
    Unit    string    // unit being measured
    Values  []float64 // measured values
    RValues []float64 // Values with outliers removed
    Min     float64   // min of RValues
    Mean    float64   // mean of RValues
    Max     float64   // max of RValues

A Metrics holds the measurements of a single metric (for example, ns/op or MB/s) for all runs of a particular benchmark.

func (*Metrics) Format Uses

func (m *Metrics) Format(scaler Scaler) string

Format returns a textual formatting of "Mean ±Diff" using scaler.

func (*Metrics) FormatDiff Uses

func (m *Metrics) FormatDiff() string

FormatDiff computes and formats the percent variation of max and min compared to mean. If b.Mean or b.Max is zero, FormatDiff returns an empty string.

func (*Metrics) FormatMean Uses

func (m *Metrics) FormatMean(scaler Scaler) string

FormatMean formats m.Mean using scaler.

type Order Uses

type Order func(t *Table, i, j int) bool

An Order defines a sort order for a table. It reports whether t.Rows[i] should appear before t.Rows[j].

func Reverse Uses

func Reverse(order Order) Order

Reverse returns the reverse of the given order.

type Row Uses

type Row struct {
    Benchmark string     // benchmark name
    Group     string     // group name
    Scaler    Scaler     // formatter for stats means
    Metrics   []*Metrics // columns of statistics
    PctDelta  float64    // unformatted percent change
    Delta     string     // formatted percent change
    Note      string     // additional information
    Change    int        // +1 better, -1 worse, 0 unchanged

A Row is a table row for display in the benchstat output.

type Scaler Uses

type Scaler func(float64) string

A Scaler is a function that scales and formats a measurement. All measurements within a given table row are formatted using the same scaler, so that the units are consistent across the row.

func NewScaler Uses

func NewScaler(val float64, unit string) Scaler

NewScaler returns a Scaler appropriate for formatting the measurement val, which has the given unit.

type Table Uses

type Table struct {
    Metric      string
    OldNewDelta bool // is this an old-new-delta table?
    Configs     []string
    Groups      []string
    Rows        []*Row

A Table is a table for display in the benchstat output.

Package benchstat imports 14 packages (graph) and is imported by 3 packages. Updated 2019-06-21. Refresh now. Tools for package owners.