tinystat: github.com/codahale/tinystat/cmd/tinystat Files

Command tinystat

tinystat is used to compare two or more sets of measurements (e.g., runs of a multiple runs of benchmarks of two possible implementations) and determine if they are statistically different. It's inspired largely by FreeBSD's ministat (written by Poul-Henning Kamp).

Imagine we have the results of different animals' SAT scores. Each animal took the SATs multiple times, and we're assuming that differences between each animal's attempts are measurement error (i.e., normally distributed). We can test for differences as follows:

$ tinystat iguana chameleon leopard

 1.5 k  +
        |
        |
        |
        |
        |
  1000  +                                             |
        |                              |              |
        |                        +-----------+  +-----------+
        |              |         |           |  |           |
        |              |         |           |  +-----*-----+
        |              |         |     *     |  |           |
   500  +              |         +-----------+  +-----------+
        |        +-----------+   |           |        |
        |        |     *     |   +-----------+
        |        +-----------+         |
        |        +-----------+         |
     0  +--------------|-----------------------------------------------
                    iguana         chameleon       leopard

Experiment  Results
chameleon   No difference proven at 95% confidence.
leopard     Difference at 95% confidence!
              343.5 +/- 292.63453863922877
              114.5% +/- 97.54484621307626%
              (Student's t, pooled s = 238.9799344943192)

As you can see, despite the superficial differences between the iguana's scores and the chameleon's scores, there is no statistically significant difference between the two at a 95% confidence level. The leopard, on the other hand, has statistically significantly different scores.

Package Files

main.go

Package main imports 10 packages (graph). Updated 2018-12-08. Refresh now. Tools for package owners.