chunk

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 30, 2020 License: AGPL-3.0 Imports: 8 Imported by: 67

Documentation

Overview

package chunk encodes timeseries in chunks of data see devdocs/chunk-format.md for more information.

Index

Constants

This section is empty.

Variables

View Source
var ChunkSpans = [32]uint32{
	1,
	5,
	10,
	15,
	20,
	30,
	60,
	90,
	2 * 60,
	3 * 60,
	5 * 60,
	10 * 60,
	15 * 60,
	20 * 60,
	30 * 60,
	45 * 60,
	3600,
	90 * 60,
	2 * 3600,
	150 * 60,
	3 * 3600,
	4 * 3600,
	5 * 3600,
	6 * 3600,
	7 * 3600,
	8 * 3600,
	9 * 3600,
	10 * 3600,
	12 * 3600,
	15 * 3600,
	18 * 3600,
	24 * 3600,
}
View Source
var RevChunkSpans = make(map[uint32]SpanCode, len(ChunkSpans))

Functions

func ExtractChunkSpan added in v0.13.0

func ExtractChunkSpan(chunk []byte) uint32

SpanOfChunk takes a chunk and tries to determine its span. It returns 0 if it failed to determine the span, this could fail either because the given chunk is invalid or because it has an old format

func MaxConfigurableSpan added in v1.0.0

func MaxConfigurableSpan() uint32

MaxConfigurableSpan returns the largest configurable span.

Types

type Chunk

type Chunk struct {
	Series    tsz.SeriesLong
	NumPoints uint32
	First     bool
}

Chunk is a chunk of data. not concurrency safe. last check that the methods are being called safely by Dieter on 20/11/2018 checked: String, Push, Finish, Encode and properties Series, NumPoints, First for the most part, confirming serialized access is easy by tracking all callers/references. The main exception is the ChunkWriteRequest mechanism. any CWR created is processed asynchronously, and chunk properties will be read (the series.T0, chunk.Encode(), etc) But it can be proven that we only call NewChunkWriteRequest() on chunks that are no longer being modified.

func New

func New(t0 uint32) *Chunk

func NewFirst

func NewFirst(t0 uint32) *Chunk

func (*Chunk) Encode

func (c *Chunk) Encode(span uint32) []byte

Encode encodes the chunk note: chunks don't know their own span, the caller/owner manages that, so for formats that encode it, it needs to be passed in. the returned value contains no references to the chunk. data is copied.

func (*Chunk) Finish

func (c *Chunk) Finish()

func (*Chunk) Push

func (c *Chunk) Push(t uint32, v float64) error

func (*Chunk) String

func (c *Chunk) String() string

type Format

type Format uint8
const (
	FormatStandardGoTsz Format = iota
	FormatStandardGoTszWithSpan
	FormatGoTszLongWithSpan // like FormatStandardGoTszWithSpan but using tsz.SeriesLong
)

identifier of message format

func (Format) String

func (i Format) String() string

type IterGen

type IterGen struct {
	T0           uint32
	IntervalHint uint32 // a hint wrt expected alignment of points. useful to recover delta overflows in tsz.Series4h, not used for other formats
	B            []byte
}

func NewIterGen

func NewIterGen(t0, intervalHint uint32, b []byte) (IterGen, error)

NewIterGen creates an IterGen and performs crude validation of the data note: it's ok for intervalHint to be 0 or 1 to mean unknown. it just means that series4h corruptions can't be remediated in single-point-per-chunk scenarios

func (*IterGen) DecodeMsg

func (z *IterGen) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*IterGen) EncodeMsg

func (z *IterGen) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (IterGen) EndTs

func (ig IterGen) EndTs() uint32

end of itergen (exclusive). next t0

func (IterGen) Format

func (ig IterGen) Format() Format

func (*IterGen) Get

func (ig *IterGen) Get() (tsz.Iter, error)

func (*IterGen) MarshalMsg

func (z *IterGen) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*IterGen) Msgsize

func (z *IterGen) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*IterGen) Size

func (ig *IterGen) Size() uint64

func (*IterGen) Span

func (ig *IterGen) Span() uint32

func (*IterGen) UnmarshalMsg

func (z *IterGen) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

type IterGensAsc

type IterGensAsc []IterGen

func (IterGensAsc) Len

func (a IterGensAsc) Len() int

func (IterGensAsc) Less

func (a IterGensAsc) Less(i, j int) bool

func (IterGensAsc) Swap

func (a IterGensAsc) Swap(i, j int)

type SpanCode

type SpanCode uint8

Directories

Path Synopsis
Package tsz implements time-series compression it is a fork of https://github.com/dgryski/go-tsz which implements http://www.vldb.org/pvldb/vol8/p1816-teller.pdf see devdocs/chunk-format.md for more info Package tsz implements time-series compression it is a fork of https://github.com/dgryski/go-tsz which implements http://www.vldb.org/pvldb/vol8/p1816-teller.pdf see devdocs/chunk-format.md for more info
Package tsz implements time-series compression it is a fork of https://github.com/dgryski/go-tsz which implements http://www.vldb.org/pvldb/vol8/p1816-teller.pdf see devdocs/chunk-format.md for more info Package tsz implements time-series compression it is a fork of https://github.com/dgryski/go-tsz which implements http://www.vldb.org/pvldb/vol8/p1816-teller.pdf see devdocs/chunk-format.md for more info

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL