encoding

package
v0.3.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 21, 2024 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	BitmapMarshal   = bitmapMarshal
	BitmapUnmarshal = bitmapUnmarshal
)

for testing

View Source 
var (
	TSDEncodeFunc = GetTSDEncoder
)

for testing

Functions

func ByteSlice2Uint32 

func ByteSlice2Uint32(slice []byte) uint32

func DecodeTSDTime 

func DecodeTSDTime(data []byte) (startTime, endTime uint16)

DecodeTSDTime decodes start-time-slot and end-time-slot of tsd. a simple method extracted from NewTSDDecoder to reduce gc pressure.

func HighBits 

func HighBits(x uint32) uint16

HighBits returns the high 16 bits of value

func JSONMarshal 

func JSONMarshal(v interface{}) []byte

JSONMarshal returns the JSON encoding of v.

func JSONUnmarshal 

func JSONUnmarshal(data []byte, v interface{}) error

JSONUnmarshal parses the JSON-encoded data and stores the result

func LowBits 

func LowBits(x uint32) uint16

LowBits returns the low 16 bits of value

func ReleaseFixedOffsetDecoder 

func ReleaseFixedOffsetDecoder(decoder *FixedOffsetDecoder)

ReleaseFixedOffsetDecoder pushes back FixedOffsetDecoder instance to pool.

func ReleaseTSDDecoder 

func ReleaseTSDDecoder(decoder *TSDDecoder)

func ReleaseTSDEncoder 

func ReleaseTSDEncoder(encoder *TSDEncoder)

func Uint32MinWidth 

func Uint32MinWidth(value uint32) int

Uint32MinWidth returns the min length of uint32

func ValueWithHighLowBits 

func ValueWithHighLowBits(high uint32, low uint16) uint32

ValueWithHighLowBits returns the value with high/low 16 bits

func ZigZagDecode 

func ZigZagDecode(v uint64) int64

ZigZagDecode converts a previously zigzag encoded uint64 back to a int64.

func ZigZagEncode 

func ZigZagEncode(x int64) uint64

ZigZagEncode converts a int64 to a uint64 by zig zagging negative and positive values across even and odd numbers. Eg. [0,-1,1,-2] becomes [0, 1, 2, 3].

Types

type DeltaBitPackingDecoder 

type DeltaBitPackingDecoder struct {
	// contains filtered or unexported fields
}

DeltaBitPackingDecoder represents a delta decoding for int32

func NewDeltaBitPackingDecoder 

func NewDeltaBitPackingDecoder(buf []byte) *DeltaBitPackingDecoder

NewDeltaBitPackingDecoder creates a delta decoder

func (*DeltaBitPackingDecoder) HasNext 

func (d *DeltaBitPackingDecoder) HasNext() bool

HasNext tests if has more int32 value

func (*DeltaBitPackingDecoder) Next 

func (d *DeltaBitPackingDecoder) Next() int32

Next returns next value if exist

func (*DeltaBitPackingDecoder) Reset 

func (d *DeltaBitPackingDecoder) Reset(buf []byte)

type DeltaBitPackingEncoder 

type DeltaBitPackingEncoder struct {
	// contains filtered or unexported fields
}

DeltaBitPackingEncoder represents a delta encoding for int32

func NewDeltaBitPackingEncoder 

func NewDeltaBitPackingEncoder() *DeltaBitPackingEncoder

NewDeltaBitPackingEncoder creates a delta encoder

func (*DeltaBitPackingEncoder) Add 

func (p *DeltaBitPackingEncoder) Add(v int32)

Add adds a new int value

func (*DeltaBitPackingEncoder) Bytes 

func (p *DeltaBitPackingEncoder) Bytes() []byte

Bytes returns binary data

func (*DeltaBitPackingEncoder) Reset 

func (p *DeltaBitPackingEncoder) Reset()

Reset clears the underlying data structure to prepare for next use

type FixedOffsetDecoder 

type FixedOffsetDecoder struct {
	// contains filtered or unexported fields
}

FixedOffsetDecoder represents the fixed offset decoder, supports random reads offset by index

func GetFixedOffsetDecoder 

func GetFixedOffsetDecoder() *FixedOffsetDecoder

GetFixedOffsetDecoder returns a FixedOffsetDecoder instance from pool.

func NewFixedOffsetDecoder 

func NewFixedOffsetDecoder() *FixedOffsetDecoder

NewFixedOffsetDecoder creates the fixed offset decoder

func (*FixedOffsetDecoder) Get 

func (d *FixedOffsetDecoder) Get(index int) (int, bool)

func (*FixedOffsetDecoder) GetBlock 

func (d *FixedOffsetDecoder) GetBlock(index int, dataBlock []byte) (block []byte, err error)

GetBlock returns the block by offset range(start -> end) with index GetBlock is only supported when Offsets are increasing encoded.

func (*FixedOffsetDecoder) Size 

func (d *FixedOffsetDecoder) Size() int

Size returns the size of offset values

func (*FixedOffsetDecoder) Unmarshal 

func (d *FixedOffsetDecoder) Unmarshal(data []byte) (left []byte, err error)

Unmarshal unmarshals from data block, then return the remaining buffer.

func (*FixedOffsetDecoder) ValueWidth 

func (d *FixedOffsetDecoder) ValueWidth() int

ValueWidth returns the width of all stored values

type FixedOffsetEncoder 

type FixedOffsetEncoder struct {
	// contains filtered or unexported fields
}

FixedOffsetEncoder represents the offset encoder with fixed length Make sure that added offset is increasing

func NewFixedOffsetEncoder 

func NewFixedOffsetEncoder(ensureIncreasing bool) *FixedOffsetEncoder

NewFixedOffsetEncoder creates the fixed length offset encoder ensureIncreasing=true ensure that added offsets are increasing, panic when value is smaller than before ensureIncreasing=false suppresses the increasing check. Offset must >= 0

func (*FixedOffsetEncoder) Add 

func (e *FixedOffsetEncoder) Add(v int)

Add adds the start offset value,

func (*FixedOffsetEncoder) FromValues 

func (e *FixedOffsetEncoder) FromValues(values []int)

FromValues resets the encoder, then init it with multi values.

func (*FixedOffsetEncoder) IsEmpty 

func (e *FixedOffsetEncoder) IsEmpty() bool

IsEmpty returns if is empty

func (*FixedOffsetEncoder) MarshalBinary 

func (e *FixedOffsetEncoder) MarshalBinary() []byte

MarshalBinary marshals the values to binary

func (*FixedOffsetEncoder) MarshalSize 

func (e *FixedOffsetEncoder) MarshalSize() int

func (*FixedOffsetEncoder) Reset 

func (e *FixedOffsetEncoder) Reset()

Reset resets the encoder context for reuse

func (*FixedOffsetEncoder) Size 

func (e *FixedOffsetEncoder) Size() int

Size returns the size

func (*FixedOffsetEncoder) Write 

func (e *FixedOffsetEncoder) Write(writer io.Writer) error

Write writes the data to the writer.

type TSDDecoder 

type TSDDecoder struct {
	// contains filtered or unexported fields
}

TSDDecoder decodes time series compress data

func GetTSDDecoder 

func GetTSDDecoder() *TSDDecoder

func NewTSDDecoder 

func NewTSDDecoder(data []byte) *TSDDecoder

NewTSDDecoder create tsd decoder instance

func (*TSDDecoder) EndTime 

func (d *TSDDecoder) EndTime() uint16

EndTime returns tsd end time slot

func (*TSDDecoder) Error 

func (d *TSDDecoder) Error() error

Error returns decode error

func (*TSDDecoder) GetValue 

func (d *TSDDecoder) GetValue(slot uint16) (float64, bool)

GetValue returns value by time slot, if it hasn't, return false.

func (*TSDDecoder) HasValue 

func (d *TSDDecoder) HasValue() bool

HasValue returns slot value if exist

func (*TSDDecoder) HasValueWithSlot 

func (d *TSDDecoder) HasValueWithSlot(slot uint16) bool

HasValueWithSlot returns value if exist by given time slot

func (*TSDDecoder) Next 

func (d *TSDDecoder) Next() bool

Next returns if has next slot data

func (*TSDDecoder) Reset 

func (d *TSDDecoder) Reset(data []byte)

Reset resets tsd data and reads the meta info from the data

func (*TSDDecoder) ResetWithTimeRange 

func (d *TSDDecoder) ResetWithTimeRange(data []byte, start, end uint16)

ResetWithTimeRange resets tsd data and reads the meta info from the data with time range

func (*TSDDecoder) Seek 

func (d *TSDDecoder) Seek(slot uint16) bool

Seek seeks and reads at the specified slot

func (*TSDDecoder) Slot 

func (d *TSDDecoder) Slot() uint16

func (*TSDDecoder) StartTime 

func (d *TSDDecoder) StartTime() uint16

StartTime returns tsd start time slot

func (*TSDDecoder) Value 

func (d *TSDDecoder) Value() uint64

Value returns value of time slot

type TSDEncoder 

type TSDEncoder struct {
	// contains filtered or unexported fields
}

TSDEncoder encodes time series data point

func GetTSDEncoder 

func GetTSDEncoder(startTime uint16) *TSDEncoder

func NewTSDEncoder 

func NewTSDEncoder(startTime uint16) *TSDEncoder

NewTSDEncoder creates tsd encoder instance

func (*TSDEncoder) AppendTime 

func (e *TSDEncoder) AppendTime(slot bit.Bit)

AppendTime appends time slot, marks time slot if has data point

func (*TSDEncoder) AppendValue 

func (e *TSDEncoder) AppendValue(value uint64)

AppendValue appends data point value

func (*TSDEncoder) Bytes 

func (e *TSDEncoder) Bytes() ([]byte, error)

Bytes returns binary which compress time series data point

func (*TSDEncoder) BytesWithoutTime 

func (e *TSDEncoder) BytesWithoutTime() ([]byte, error)

BytesWithoutTime returns binary which compress time series data point without time slot range

func (*TSDEncoder) EmitDownSamplingValue 

func (e *TSDEncoder) EmitDownSamplingValue(pos int, value float64)

EmitDownSamplingValue appends the value after down sampling Inf value symbols an empty value to omit

func (*TSDEncoder) Reset 

func (e *TSDEncoder) Reset()

Reset resets the underlying bytes.Buffer

func (*TSDEncoder) RestWithStartTime 

func (e *TSDEncoder) RestWithStartTime(startTime uint16)

RestWithStartTime resets the buffer and slot info

type TSDStreamReader 

type TSDStreamReader interface {
	// TimeRange returns the time slot range
	TimeRange() (start, end uint16)
	// HasNext returns the if has more field data
	HasNext() bool
	// Next returns the fieldID and field data
	Next() (pFieldID uint16, fieldData *TSDDecoder)
	// Close closes the reader and releases the resource
	Close()
}

TSDStreamReader represents the tsd data points reader for multi-fields

func NewTSDStreamReader 

func NewTSDStreamReader(data []byte) TSDStreamReader

NewTSDStreamReader creates the tsd stream reader

type TSDStreamWriter 

type TSDStreamWriter interface {
	// WriteField writes the field data
	WriteField(pFieldID uint16, data []byte)
	// Bytes returns the binary data with time range and fields' data
	Bytes() ([]byte, error)
}

TSDStreamWriter represents the tsd data points stream writer for multi-fields

func NewTSDStreamWriter 

func NewTSDStreamWriter(start, end uint16) TSDStreamWriter

NewTSDStreamWriter creates the tsd stream writer

type TSDValueGetter 

type TSDValueGetter interface {
	// GetValue returns value by time slot, if it hasn't, return false.
	GetValue(slot uint16) (float64, bool)
}

TSDValueGetter represents value getter from tsd.

type XORDecoder 

type XORDecoder struct {
	// contains filtered or unexported fields
}

XORDecoder decodes buffer to uint64 values using xor compress

func NewXORDecoder 

func NewXORDecoder(br *bit.Reader) *XORDecoder

NewXORDecoder create decoder uncompress buffer using xor

func (*XORDecoder) Next 

func (d *XORDecoder) Next() bool

Next return if it has value in buffer using xor, do uncompress logic in next method, data format reference encoder format

func (*XORDecoder) Reset 

func (d *XORDecoder) Reset()

Reset resets the underlying buffer to decode

func (*XORDecoder) Value 

func (d *XORDecoder) Value() uint64

Value returns uint64 from buffer

type XOREncoder 

type XOREncoder struct {
	// contains filtered or unexported fields
}

XOREncoder encodes uint64 value using xor compress

func NewXOREncoder 

func NewXOREncoder(bw *bit.Writer) *XOREncoder

NewXOREncoder creates xor encoder for compressing uint64 data

func (*XOREncoder) Reset 

func (e *XOREncoder) Reset()

func (*XOREncoder) Write 

func (e *XOREncoder) Write(val uint64) error

Write writs uint64 v to underlying buffer, using xor compress Value is encoded by XOR then with previous value. If XOR result is a zero value(value is the same as the previous value), only a single '0' bit is stored, otherwise '1' bit is stored. For non-zero XOR result, there are two choices: 1). If the block of meaningful bits falls in between the block of previous meaningful bits, 

i.e., there are at least as many leading and trailing zeros as with the previous value,
      use that information for the block position and just store the XOR value.

2). Length of the number of leading zeros is stored in the next 6 bits, 

then length of the XOR value is stored in the next 6 bits,
finally the XOR value is stored.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.