metrictank: github.com/grafana/metrictank/mdata/chunk Index | Files | Directories

package chunk

import "github.com/grafana/metrictank/mdata/chunk"

package chunk encodes timeseries in chunks of data see devdocs/chunk-format.md for more information.

Index

Package Files

chunk.go encode.go format.go format_string.go itergen.go itergen_gen.go spans.go

Variables

var ChunkSpans = [32]uint32{
    1,
    5,
    10,
    15,
    20,
    30,
    60,
    90,
    2 * 60,
    3 * 60,
    5 * 60,
    10 * 60,
    15 * 60,
    20 * 60,
    30 * 60,
    45 * 60,
    3600,
    90 * 60,
    2 * 3600,
    150 * 60,
    3 * 3600,
    4 * 3600,
    5 * 3600,
    6 * 3600,
    7 * 3600,
    8 * 3600,
    9 * 3600,
    10 * 3600,
    12 * 3600,
    15 * 3600,
    18 * 3600,
    24 * 3600,
}
var RevChunkSpans = make(map[uint32]SpanCode, len(ChunkSpans))

func ExtractChunkSpan Uses

func ExtractChunkSpan(chunk []byte) uint32

SpanOfChunk takes a chunk and tries to determine its span. It returns 0 if it failed to determine the span, this could fail either because the given chunk is invalid or because it has an old format

func MaxConfigurableSpan Uses

func MaxConfigurableSpan() uint32

MaxConfigurableSpan returns the largest configurable span.

type Chunk Uses

type Chunk struct {
    Series    tsz.SeriesLong
    NumPoints uint32
    First     bool
}

Chunk is a chunk of data. not concurrency safe. last check that the methods are being called safely by Dieter on 20/11/2018 checked: String, Push, Finish, Encode and properties Series, NumPoints, First for the most part, confirming serialized access is easy by tracking all callers/references. The main exception is the ChunkWriteRequest mechanism. any CWR created is processed asynchronously, and chunk properties will be read (the series.T0, chunk.Encode(), etc) But it can be proven that we only call NewChunkWriteRequest() on chunks that are no longer being modified.

func New Uses

func New(t0 uint32) *Chunk

func NewFirst Uses

func NewFirst(t0 uint32) *Chunk

func (*Chunk) Encode Uses

func (c *Chunk) Encode(span uint32) []byte

Encode encodes the chunk note: chunks don't know their own span, the caller/owner manages that, so for formats that encode it, it needs to be passed in. the returned value contains no references to the chunk. data is copied.

func (*Chunk) Finish Uses

func (c *Chunk) Finish()

func (*Chunk) Push Uses

func (c *Chunk) Push(t uint32, v float64) error

func (*Chunk) String Uses

func (c *Chunk) String() string

type Format Uses

type Format uint8
const (
    FormatStandardGoTsz Format = iota
    FormatStandardGoTszWithSpan
    FormatGoTszLongWithSpan // like FormatStandardGoTszWithSpan but using tsz.SeriesLong
)

identifier of message format

func (Format) String Uses

func (i Format) String() string

type IterGen Uses

type IterGen struct {
    T0           uint32
    IntervalHint uint32 // a hint wrt expected alignment of points. useful to recover delta overflows in tsz.Series4h, not used for other formats
    B            []byte
}

go:generate msgp

func NewIterGen Uses

func NewIterGen(t0, intervalHint uint32, b []byte) (IterGen, error)

NewIterGen creates an IterGen and performs crude validation of the data note: it's ok for intervalHint to be 0 or 1 to mean unknown. it just means that series4h corruptions can't be remediated in single-point-per-chunk scenarios

func (*IterGen) DecodeMsg Uses

func (z *IterGen) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*IterGen) EncodeMsg Uses

func (z *IterGen) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (IterGen) EndTs Uses

func (ig IterGen) EndTs() uint32

end of itergen (exclusive). next t0

func (IterGen) Format Uses

func (ig IterGen) Format() Format

func (*IterGen) Get Uses

func (ig *IterGen) Get() (tsz.Iter, error)

func (*IterGen) MarshalMsg Uses

func (z *IterGen) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*IterGen) Msgsize Uses

func (z *IterGen) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*IterGen) Size Uses

func (ig *IterGen) Size() uint64

func (*IterGen) Span Uses

func (ig *IterGen) Span() uint32

func (*IterGen) UnmarshalMsg Uses

func (z *IterGen) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

type IterGensAsc Uses

type IterGensAsc []IterGen

msgp:ignore IterGensAsc

func (IterGensAsc) Len Uses

func (a IterGensAsc) Len() int

func (IterGensAsc) Less Uses

func (a IterGensAsc) Less(i, j int) bool

func (IterGensAsc) Swap Uses

func (a IterGensAsc) Swap(i, j int)

type SpanCode Uses

type SpanCode uint8

Directories

PathSynopsis
archive
tszPackage tsz implements time-series compression it is a fork of https://github.com/dgryski/go-tsz which implements http://www.vldb.org/pvldb/vol8/p1816-teller.pdf see devdocs/chunk-format.md for more info

Package chunk imports 8 packages (graph) and is imported by 27 packages. Updated 2020-07-15. Refresh now. Tools for package owners.