Documentation ¶
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type TDigest ¶
type TDigest struct {
// contains filtered or unexported fields
}
A TDigest is an efficient data structure for computing streaming approximate quantiles of a dataset.
Example ¶
rand.Seed(5678) values := make(chan float64) // Generate 100k uniform random data between 0 and 100 var ( n int = 100000 min, max float64 = 0, 100 ) go func() { for i := 0; i < n; i++ { values <- min + rand.Float64()*(max-min) } close(values) }() // Pass the values through a TDigest, compression parameter 100 td := New() for val := range values { // Add the value with weight 1 td.Add(val, 1) } // Print the 50th, 90th, 99th, 99.9th, and 99.99th percentiles fmt.Printf("50th: %.5f\n", td.Quantile(0.5)) fmt.Printf("90th: %.5f\n", td.Quantile(0.9)) fmt.Printf("99th: %.5f\n", td.Quantile(0.99)) fmt.Printf("99.9th: %.5f\n", td.Quantile(0.999)) fmt.Printf("99.99th: %.5f\n", td.Quantile(0.9999))
Output:
func New ¶
func New() *TDigest
New produces a new TDigest using the default compression level of 100.
func NewWithCompression ¶
NewWithCompression produces a new TDigest with a specific compression level. The input compression value, which should be >= 1.0, will control how aggressively the TDigest compresses data together.
The original TDigest paper suggests using a value of 100 for a good balance between precision and efficiency. It will land at very small (think like 1e-6 percentile points) errors at extreme points in the distribution, and compression ratios of around 500 for large data sets (1 millionish datapoints).
func (*TDigest) Add ¶
Add will add a value to the TDigest, updating all quantiles. A weight can be specified; use weight of 1 if you don't care about weighting your dataset.
Add will ignore input values of NaN or Inf.
func (*TDigest) MarshalBinary ¶
MarshalBinary serializes d as a sequence of bytes, suitable to be deserialized later with UnmarshalBinary.
func (*TDigest) MergeInto ¶
MergeInto(other) will add all of the data within a TDigest into other, combining them into one larger TDigest.
func (*TDigest) Quantile ¶
Quantile(q) will estimate the qth quantile value of the dataset. The input value of q should be in the range [0.0, 1.0]; if it is outside that range, it will be clipped into it automatically.
Calling Quantile on a TDigest with no data will return NaN.
func (*TDigest) UnmarshalBinary ¶
UnmarshalBinary populates d with the parsed contents of p, which should have been created with a call to MarshalBinary.