cockroach: github.com/cockroachdb/cockroach/pkg/util/encoding Index | Files | Directories

package encoding

import "github.com/cockroachdb/cockroach/pkg/util/encoding"

Index

Package Files

complement_fast.go decimal.go encoding.go float.go type_string.go

Constants

const (

    // IntMin is chosen such that the range of int tags does not overlap the
    // ascii character set that is frequently used in testing.
    IntMin = 0x80 // 128

    // IntMax is the maximum int tag value.
    IntMax = 0xfd // 253

)
const (
    // EncodedDurationMaxLen is the largest number of bytes used when encoding a
    // Duration.
    EncodedDurationMaxLen = 1 + 3*binary.MaxVarintLen64 // 3 varints are encoded.
    // EncodedTimeTZMaxLen is the largest number of bytes used when encoding a
    // TimeTZ.
    EncodedTimeTZMaxLen = 1 + binary.MaxVarintLen64 + binary.MaxVarintLen32
)
const NoColumnID uint32 = 0

NoColumnID is a sentinel for the EncodeFooValue methods representing an invalid column id.

const NonsortingUvarintMaxLen = 10

NonsortingUvarintMaxLen is the maximum length of an EncodeNonsortingUvarint encoded value.

const NonsortingVarintMaxLen = binary.MaxVarintLen64

NonsortingVarintMaxLen is the maximum length of an EncodeNonsortingVarint encoded value.

func AddJSONPathTerminator Uses

func AddJSONPathTerminator(b []byte) []byte

AddJSONPathTerminator adds a json path terminator to a byte array.

func DecodeBitArrayAscending Uses

func DecodeBitArrayAscending(b []byte) ([]byte, bitarray.BitArray, error)

DecodeBitArrayAscending decodes a bit array which was encoded using EncodeBitArrayAscending. The remainder of the input buffer and the decoded bit array are returned.

func DecodeBitArrayDescending Uses

func DecodeBitArrayDescending(b []byte) ([]byte, bitarray.BitArray, error)

DecodeBitArrayDescending is the descending version of DecodeBitArrayAscending.

func DecodeBitArrayValue Uses

func DecodeBitArrayValue(b []byte) (remaining []byte, d bitarray.BitArray, err error)

DecodeBitArrayValue decodes a value encoded by EncodeUntaggedBitArrayValue.

func DecodeBoolValue Uses

func DecodeBoolValue(buf []byte) (remaining []byte, b bool, err error)

DecodeBoolValue decodes a value encoded by EncodeBoolValue.

func DecodeBytesAscending Uses

func DecodeBytesAscending(b []byte, r []byte) ([]byte, []byte, error)

DecodeBytesAscending decodes a []byte value from the input buffer which was encoded using EncodeBytesAscending. The decoded bytes are appended to r. The remainder of the input buffer and the decoded []byte are returned.

func DecodeBytesDescending Uses

func DecodeBytesDescending(b []byte, r []byte) ([]byte, []byte, error)

DecodeBytesDescending decodes a []byte value from the input buffer which was encoded using EncodeBytesDescending. The decoded bytes are appended to r. The remainder of the input buffer and the decoded []byte are returned.

func DecodeBytesValue Uses

func DecodeBytesValue(b []byte) (remaining []byte, data []byte, err error)

DecodeBytesValue decodes a value encoded by EncodeBytesValue.

func DecodeDecimalAscending Uses

func DecodeDecimalAscending(buf []byte, tmp []byte) ([]byte, apd.Decimal, error)

DecodeDecimalAscending returns the remaining byte slice after decoding and the decoded decimal from buf.

func DecodeDecimalDescending Uses

func DecodeDecimalDescending(buf []byte, tmp []byte) ([]byte, apd.Decimal, error)

DecodeDecimalDescending decodes decimals encoded with EncodeDecimalDescending.

func DecodeDecimalValue Uses

func DecodeDecimalValue(b []byte) (remaining []byte, d apd.Decimal, err error)

DecodeDecimalValue decodes a value encoded by EncodeDecimalValue.

func DecodeDurationAscending Uses

func DecodeDurationAscending(b []byte) ([]byte, duration.Duration, error)

DecodeDurationAscending decodes a duration.Duration value which was encoded using EncodeDurationAscending. The remainder of the input buffer and the decoded duration.Duration are returned.

func DecodeDurationDescending Uses

func DecodeDurationDescending(b []byte) ([]byte, duration.Duration, error)

DecodeDurationDescending is the descending version of DecodeDurationAscending.

func DecodeDurationValue Uses

func DecodeDurationValue(b []byte) (remaining []byte, d duration.Duration, err error)

DecodeDurationValue decodes a value encoded by EncodeUntaggedDurationValue.

func DecodeFloatAscending Uses

func DecodeFloatAscending(buf []byte) ([]byte, float64, error)

DecodeFloatAscending returns the remaining byte slice after decoding and the decoded float64 from buf.

func DecodeFloatDescending Uses

func DecodeFloatDescending(buf []byte) ([]byte, float64, error)

DecodeFloatDescending decodes floats encoded with EncodeFloatDescending.

func DecodeFloatValue Uses

func DecodeFloatValue(b []byte) (remaining []byte, f float64, err error)

DecodeFloatValue decodes a value encoded by EncodeFloatValue.

func DecodeIPAddrValue Uses

func DecodeIPAddrValue(b []byte) (remaining []byte, u ipaddr.IPAddr, err error)

DecodeIPAddrValue decodes a value encoded by EncodeIPAddrValue.

func DecodeIfInterleavedSentinel Uses

func DecodeIfInterleavedSentinel(b []byte) ([]byte, bool)

DecodeIfInterleavedSentinel decodes the interleavedSentinel from the input buffer and returns the remaining buffer without the sentinel if the interleavedSentinel is the first byte. Otherwise, the buffer is returned unchanged and false is returned.

func DecodeIfNotNull Uses

func DecodeIfNotNull(b []byte) ([]byte, bool)

DecodeIfNotNull decodes a not-NULL value from the input buffer. If the input buffer contains a not-NULL marker at the start of the buffer then it is removed from the buffer and true is returned for the second result. Otherwise, the buffer is returned unchanged and false is returned for the second result. Note that the not-NULL marker is identical to the empty string encoding, so do not use this routine where it is necessary to distinguish not-NULL from the empty string. This function handles both ascendingly and descendingly encoded NULLs.

func DecodeIfNotNullDescending Uses

func DecodeIfNotNullDescending(b []byte) ([]byte, bool)

DecodeIfNotNullDescending decodes encodedNotNullDesc from the input buffer and returns the remaining buffer without the sentinel if encodedNotNullDesc is the first byte. Otherwise, the buffer is returned unchanged and false is returned.

func DecodeIfNull Uses

func DecodeIfNull(b []byte) ([]byte, bool)

DecodeIfNull decodes a NULL value from the input buffer. If the input buffer contains a null at the start of the buffer then it is removed from the buffer and true is returned for the second result. Otherwise, the buffer is returned unchanged and false is returned for the second result. Since the NULL value encoding is guaranteed to never occur as the prefix for the EncodeVarint, EncodeFloat, EncodeBytes and EncodeString encodings, it is safe to call DecodeIfNull on their encoded values. This function handles both ascendingly and descendingly encoded NULLs.

func DecodeIntValue Uses

func DecodeIntValue(b []byte) (remaining []byte, i int64, err error)

DecodeIntValue decodes a value encoded by EncodeIntValue.

func DecodeIntoNonsortingDecimal Uses

func DecodeIntoNonsortingDecimal(dec *apd.Decimal, buf []byte, tmp []byte) error

DecodeIntoNonsortingDecimal is like DecodeNonsortingDecimal, but it operates on the passed-in *apd.Decimal instead of producing a new one.

func DecodeIntoUntaggedDecimalValue Uses

func DecodeIntoUntaggedDecimalValue(d *apd.Decimal, b []byte) (remaining []byte, err error)

DecodeIntoUntaggedDecimalValue is like DecodeUntaggedDecimalValue except it writes the new Decimal into the input apd.Decimal pointer, which must be non-nil.

func DecodeNonsortingDecimal Uses

func DecodeNonsortingDecimal(buf []byte, tmp []byte) (apd.Decimal, error)

DecodeNonsortingDecimal returns the decoded decimal from buf encoded with EncodeNonsortingDecimal. buf is assumed to contain only the encoded decimal, as the function does not know from the encoding itself what the length of the encoded value is.

func DecodeNonsortingStdlibUvarint Uses

func DecodeNonsortingStdlibUvarint(
    buf []byte,
) (remaining []byte, length int, value uint64, err error)

DecodeNonsortingStdlibUvarint decodes a value encoded with binary.PutUvarint. It returns the length of the encoded varint and value.

func DecodeNonsortingStdlibVarint Uses

func DecodeNonsortingStdlibVarint(b []byte) (remaining []byte, length int, value int64, err error)

DecodeNonsortingStdlibVarint decodes a value encoded by EncodeNonsortingVarint. It returns the length of the encoded varint and value.

func DecodeNonsortingUvarint Uses

func DecodeNonsortingUvarint(buf []byte) (remaining []byte, length int, value uint64, err error)

DecodeNonsortingUvarint decodes a value encoded by EncodeNonsortingUvarint. It returns the length of the encoded varint and value.

func DecodeTimeAscending Uses

func DecodeTimeAscending(b []byte) ([]byte, time.Time, error)

DecodeTimeAscending decodes a time.Time value which was encoded using EncodeTime. The remainder of the input buffer and the decoded time.Time are returned.

func DecodeTimeDescending Uses

func DecodeTimeDescending(b []byte) ([]byte, time.Time, error)

DecodeTimeDescending is the descending version of DecodeTimeAscending.

func DecodeTimeTZAscending Uses

func DecodeTimeTZAscending(b []byte) ([]byte, timetz.TimeTZ, error)

DecodeTimeTZAscending decodes a timetz.TimeTZ value which was encoded using encodeTimeTZ. The remainder of the input buffer and the decoded timetz.TimeTZ are returned.

func DecodeTimeTZDescending Uses

func DecodeTimeTZDescending(b []byte) ([]byte, timetz.TimeTZ, error)

DecodeTimeTZDescending is the descending version of DecodeTimeTZAscending.

func DecodeTimeTZValue Uses

func DecodeTimeTZValue(b []byte) (remaining []byte, t timetz.TimeTZ, err error)

DecodeTimeTZValue decodes a value encoded by EncodeTimeTZValue.

func DecodeTimeValue Uses

func DecodeTimeValue(b []byte) (remaining []byte, t time.Time, err error)

DecodeTimeValue decodes a value encoded by EncodeTimeValue.

func DecodeUUIDValue Uses

func DecodeUUIDValue(b []byte) (remaining []byte, u uuid.UUID, err error)

DecodeUUIDValue decodes a value encoded by EncodeUUIDValue.

func DecodeUint32Ascending Uses

func DecodeUint32Ascending(b []byte) ([]byte, uint32, error)

DecodeUint32Ascending decodes a uint32 from the input buffer, treating the input as a big-endian 4 byte uint32 representation. The remainder of the input buffer and the decoded uint32 are returned.

func DecodeUint32Descending Uses

func DecodeUint32Descending(b []byte) ([]byte, uint32, error)

DecodeUint32Descending decodes a uint32 value which was encoded using EncodeUint32Descending.

func DecodeUint64Ascending Uses

func DecodeUint64Ascending(b []byte) ([]byte, uint64, error)

DecodeUint64Ascending decodes a uint64 from the input buffer, treating the input as a big-endian 8 byte uint64 representation. The remainder of the input buffer and the decoded uint64 are returned.

func DecodeUint64Descending Uses

func DecodeUint64Descending(b []byte) ([]byte, uint64, error)

DecodeUint64Descending decodes a uint64 value which was encoded using EncodeUint64Descending.

func DecodeUnsafeStringAscending Uses

func DecodeUnsafeStringAscending(b []byte, r []byte) ([]byte, string, error)

DecodeUnsafeStringAscending decodes a string value from the input buffer which was encoded using EncodeString or EncodeBytes. The r []byte is used as a temporary buffer in order to avoid memory allocations. The remainder of the input buffer and the decoded string are returned. Note that the returned string may share storage with the input buffer.

func DecodeUnsafeStringDescending Uses

func DecodeUnsafeStringDescending(b []byte, r []byte) ([]byte, string, error)

DecodeUnsafeStringDescending decodes a string value from the input buffer which was encoded using EncodeStringDescending or EncodeBytesDescending. The r []byte is used as a temporary buffer in order to avoid memory allocations. The remainder of the input buffer and the decoded string are returned. Note that the returned string may share storage with the input buffer.

func DecodeUntaggedBitArrayValue Uses

func DecodeUntaggedBitArrayValue(b []byte) (remaining []byte, d bitarray.BitArray, err error)

DecodeUntaggedBitArrayValue decodes a value encoded by EncodeUntaggedBitArrayValue.

func DecodeUntaggedBytesValue Uses

func DecodeUntaggedBytesValue(b []byte) (remaining, data []byte, err error)

DecodeUntaggedBytesValue decodes a value encoded by EncodeUntaggedBytesValue.

func DecodeUntaggedDecimalValue Uses

func DecodeUntaggedDecimalValue(b []byte) (remaining []byte, d apd.Decimal, err error)

DecodeUntaggedDecimalValue decodes a value encoded by EncodeUntaggedDecimalValue.

func DecodeUntaggedDurationValue Uses

func DecodeUntaggedDurationValue(b []byte) (remaining []byte, d duration.Duration, err error)

DecodeUntaggedDurationValue decodes a value encoded by EncodeUntaggedDurationValue.

func DecodeUntaggedFloatValue Uses

func DecodeUntaggedFloatValue(b []byte) (remaining []byte, f float64, err error)

DecodeUntaggedFloatValue decodes a value encoded by EncodeUntaggedFloatValue.

func DecodeUntaggedIPAddrValue Uses

func DecodeUntaggedIPAddrValue(b []byte) (remaining []byte, u ipaddr.IPAddr, err error)

DecodeUntaggedIPAddrValue decodes a value encoded by EncodeUntaggedIPAddrValue.

func DecodeUntaggedIntValue Uses

func DecodeUntaggedIntValue(b []byte) (remaining []byte, i int64, err error)

DecodeUntaggedIntValue decodes a value encoded by EncodeUntaggedIntValue.

func DecodeUntaggedTimeTZValue Uses

func DecodeUntaggedTimeTZValue(b []byte) (remaining []byte, t timetz.TimeTZ, err error)

DecodeUntaggedTimeTZValue decodes a value encoded by EncodeUntaggedTimeTZValue.

func DecodeUntaggedTimeValue Uses

func DecodeUntaggedTimeValue(b []byte) (remaining []byte, t time.Time, err error)

DecodeUntaggedTimeValue decodes a value encoded by EncodeUntaggedTimeValue.

func DecodeUntaggedUUIDValue Uses

func DecodeUntaggedUUIDValue(b []byte) (remaining []byte, u uuid.UUID, err error)

DecodeUntaggedUUIDValue decodes a value encoded by EncodeUntaggedUUIDValue.

func DecodeUvarintAscending Uses

func DecodeUvarintAscending(b []byte) ([]byte, uint64, error)

DecodeUvarintAscending decodes a varint encoded uint64 from the input buffer. The remainder of the input buffer and the decoded uint64 are returned.

func DecodeUvarintDescending Uses

func DecodeUvarintDescending(b []byte) ([]byte, uint64, error)

DecodeUvarintDescending decodes a uint64 value which was encoded using EncodeUvarintDescending.

func DecodeVarintAscending Uses

func DecodeVarintAscending(b []byte) ([]byte, int64, error)

DecodeVarintAscending decodes a value encoded by EncodeVaringAscending.

func DecodeVarintDescending Uses

func DecodeVarintDescending(b []byte) ([]byte, int64, error)

DecodeVarintDescending decodes a uint64 value which was encoded using EncodeVarintDescending.

func DecomposeKeyTokens Uses

func DecomposeKeyTokens(b []byte) (tokens [][]byte, containsNull bool, err error)

DecomposeKeyTokens breaks apart a key into its individual key-encoded values and returns a slice of byte slices, one for each key-encoded value. It also returns whether the key contains a NULL value.

func EncLenUvarintAscending Uses

func EncLenUvarintAscending(v uint64) int

EncLenUvarintAscending returns the encoding length for EncodeUvarintAscending without actually encoding.

func EncLenUvarintDescending Uses

func EncLenUvarintDescending(v uint64) int

EncLenUvarintDescending returns the encoding length for EncodeUvarintDescending without actually encoding.

func EncodeArrayAscending Uses

func EncodeArrayAscending(b []byte) []byte

EncodeArrayAscending encodes a value used to signify membership of an array for JSON objects.

func EncodeArrayValue Uses

func EncodeArrayValue(appendTo []byte, colID uint32, data []byte) []byte

EncodeArrayValue encodes a byte array value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeBitArrayAscending Uses

func EncodeBitArrayAscending(b []byte, d bitarray.BitArray) []byte

EncodeBitArrayAscending encodes a bitarray.BitArray value, appends it to the supplied buffer, and returns the final buffer. The encoding is guaranteed to be ordered such that if t1.Compare(t2) < 0 (or = 0 or > 0) then bytes.Compare will order them the same way after encoding.

The encoding uses varint encoding for each word of the backing array. This is a trade-off. The alternative is to encode the entire backing word array as a byte array, using byte array encoding and escaped special bytes (via `encodeBytesAscendingWithoutTerminatorOrPrefix`). There are two arguments against this alternative: - the bytes must be encoded big endian, but the most common architectures

running CockroachDB are little-endian, so the bytes would need
to be reordered prior to encoding.

- when decoding or skipping over a value, the decoding/sizing loop

would need to look at every byte of the encoding to find the
terminator.

In contrast, the chosen encoding using varints is endianness-agnostic and enables fast decoding/skipping thanks ot the tag bytes.

func EncodeBitArrayDescending Uses

func EncodeBitArrayDescending(b []byte, d bitarray.BitArray) []byte

EncodeBitArrayDescending is the descending version of EncodeBitArrayAscending.

func EncodeBitArrayValue Uses

func EncodeBitArrayValue(appendTo []byte, colID uint32, d bitarray.BitArray) []byte

EncodeBitArrayValue encodes a bit array value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeBoolValue Uses

func EncodeBoolValue(appendTo []byte, colID uint32, b bool) []byte

EncodeBoolValue encodes a bool value, appends it to the supplied buffer, and returns the final buffer.

func EncodeBytesAscending Uses

func EncodeBytesAscending(b []byte, data []byte) []byte

EncodeBytesAscending encodes the []byte value using an escape-based encoding. The encoded value is terminated with the sequence "\x00\x01" which is guaranteed to not occur elsewhere in the encoded value. The encoded bytes are append to the supplied buffer and the resulting buffer is returned.

func EncodeBytesDescending Uses

func EncodeBytesDescending(b []byte, data []byte) []byte

EncodeBytesDescending encodes the []byte value using an escape-based encoding and then inverts (ones complement) the result so that it sorts in reverse order, from larger to smaller lexicographically.

func EncodeBytesValue Uses

func EncodeBytesValue(appendTo []byte, colID uint32, data []byte) []byte

EncodeBytesValue encodes a byte array value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeDecimalAscending Uses

func EncodeDecimalAscending(appendTo []byte, d *apd.Decimal) []byte

EncodeDecimalAscending returns the resulting byte slice with the encoded decimal appended to the given buffer.

Values are classified as large, medium, or small according to the value of E. If E is 11 or more, the value is large. For E between 0 and 10, the value is medium. For E less than zero, the value is small.

Large positive values are encoded as a single byte 0x34 followed by E as a varint and then M. Medium positive values are a single byte of 0x29+E followed by M. Small positive values are encoded as a single byte 0x28 followed by a descending varint encoding for -E followed by M.

Small negative values are encoded as a single byte 0x26 followed by -E as a varint and then the ones-complement of M. Medium negative values are encoded as a byte 0x25-E followed by the ones-complement of M. Large negative values consist of the single byte 0x1a followed by a descending varint encoding of E followed by the ones-complement of M.

func EncodeDecimalDescending Uses

func EncodeDecimalDescending(appendTo []byte, d *apd.Decimal) []byte

EncodeDecimalDescending is the descending version of EncodeDecimalAscending.

func EncodeDecimalValue Uses

func EncodeDecimalValue(appendTo []byte, colID uint32, d *apd.Decimal) []byte

EncodeDecimalValue encodes an apd.Decimal value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeDurationAscending Uses

func EncodeDurationAscending(b []byte, d duration.Duration) ([]byte, error)

EncodeDurationAscending encodes a duration.Duration value, appends it to the supplied buffer, and returns the final buffer. The encoding is guaranteed to be ordered such that if t1.Compare(t2) < 0 (or = 0 or > 0) then bytes.Compare will order them the same way after encoding.

func EncodeDurationDescending Uses

func EncodeDurationDescending(b []byte, d duration.Duration) ([]byte, error)

EncodeDurationDescending is the descending version of EncodeDurationAscending.

func EncodeDurationValue Uses

func EncodeDurationValue(appendTo []byte, colID uint32, d duration.Duration) []byte

EncodeDurationValue encodes a duration.Duration value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeFalseAscending Uses

func EncodeFalseAscending(b []byte) []byte

EncodeFalseAscending encodes the boolean value false for use with JSON inverted indexes.

func EncodeFloatAscending Uses

func EncodeFloatAscending(b []byte, f float64) []byte

EncodeFloatAscending returns the resulting byte slice with the encoded float64 appended to b. The encoded format for a float64 value f is, for positive f, the encoding of the 64 bits (in IEEE 754 format) re-interpreted as an int64 and encoded using EncodeUint64Ascending. For negative f, we keep the sign bit and invert all other bits, encoding this value using EncodeUint64Descending. This approach was inspired by in github.com/google/orderedcode/orderedcode.go.

One of five single-byte prefix tags are appended to the front of the encoding. These tags enforce logical ordering of keys for both ascending and descending encoding directions. The tags split the encoded floats into five categories: - NaN for an ascending encoding direction - Negative valued floats - Zero (positive and negative) - Positive valued floats - NaN for a descending encoding direction This ordering ensures that NaNs are always sorted first in either encoding direction, and that after them a logical ordering is followed.

func EncodeFloatDescending Uses

func EncodeFloatDescending(b []byte, f float64) []byte

EncodeFloatDescending is the descending version of EncodeFloatAscending.

func EncodeFloatValue Uses

func EncodeFloatValue(appendTo []byte, colID uint32, f float64) []byte

EncodeFloatValue encodes a float value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeIPAddrValue Uses

func EncodeIPAddrValue(appendTo []byte, colID uint32, u ipaddr.IPAddr) []byte

EncodeIPAddrValue encodes a ipaddr.IPAddr value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeIntValue Uses

func EncodeIntValue(appendTo []byte, colID uint32, i int64) []byte

EncodeIntValue encodes an int value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeInterleavedSentinel Uses

func EncodeInterleavedSentinel(b []byte) []byte

EncodeInterleavedSentinel encodes an interleavedSentinel that is necessary for interleaved indexes and their index keys. The interleavedSentinel has a byte value 0xfe and is equivalent to encodedNotNullDesc.

func EncodeJSONAscending Uses

func EncodeJSONAscending(b []byte) []byte

EncodeJSONAscending encodes a JSON Type. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeJSONEmptyArray Uses

func EncodeJSONEmptyArray(b []byte) []byte

EncodeJSONEmptyArray returns a byte array b with a byte to signify an empty JSON array.

func EncodeJSONEmptyObject Uses

func EncodeJSONEmptyObject(b []byte) []byte

EncodeJSONEmptyObject returns a byte array b with a byte to signify an empty JSON object.

func EncodeJSONKeyStringAscending Uses

func EncodeJSONKeyStringAscending(b []byte, s string, end bool) []byte

EncodeJSONKeyStringAscending encodes the JSON key string value with a JSON specific escaped terminator. This allows us to encode keys in the same number of bytes as a string, while at the same time giving us a sentinel to identify JSON keys. The end parameter is used to determine if this is the last key in a a JSON path. If it is we don't add a separator after it.

func EncodeJSONValue Uses

func EncodeJSONValue(appendTo []byte, colID uint32, data []byte) []byte

EncodeJSONValue encodes an already-byte-encoded JSON value with no value tag but with a length prefix, appends it to the supplied buffer, and returns the final buffer.

func EncodeNonsortingDecimal Uses

func EncodeNonsortingDecimal(b []byte, d *apd.Decimal) []byte

EncodeNonsortingDecimal returns the resulting byte slice with the encoded decimal appended to b. The encoding is limited compared to standard encodings in this package in that - It will not sort lexicographically - It does not encode its length or terminate itself, so decoding

functions must be provided the exact encoded bytes

The encoding assumes that any number can be written as ±0.xyz... * 10^exp, where xyz is a digit string, x != 0, and the last decimal in xyz is also not 0.

The encoding uses its first byte to split decimals into 7 distinct ordered groups (no NaN or Infinity support yet). The groups can be seen in encoding.go's const definition. Following this, the absolute value of the exponent of the decimal (as defined above) is encoded as an unsigned varint. Second, the absolute value of the digit string is added as a big-endian byte slice.

All together, the encoding looks like:

<marker><uvarint exponent><big-endian encoded big.Int>.

The markers are shared with the sorting decimal encoding as follows:

decimalNaN              -> decimalNaN
decimalNegativeInfinity -> decimalNegativeInfinity
decimalNegLarge         -> decimalNegValPosExp
decimalNegMedium        -> decimalNegValZeroExp
decimalNegSmall         -> decimalNegValNegExp
decimalZero             -> decimalZero
decimalPosSmall         -> decimalPosValNegExp
decimalPosMedium        -> decimalPosValZeroExp
decimalPosLarge         -> decimalPosValPosExp
decimalInfinity         -> decimalInfinity
decimalNaNDesc          -> decimalNaNDesc

func EncodeNonsortingStdlibVarint Uses

func EncodeNonsortingStdlibVarint(appendTo []byte, x int64) []byte

EncodeNonsortingStdlibVarint encodes an int value using encoding/binary, appends it to the supplied buffer, and returns the final buffer.

func EncodeNonsortingUvarint Uses

func EncodeNonsortingUvarint(appendTo []byte, x uint64) []byte

EncodeNonsortingUvarint encodes a uint64, appends it to the supplied buffer, and returns the final buffer. The encoding used is similar to encoding/binary, but with the most significant bits first: - Unsigned integers are serialized 7 bits at a time, starting with the

most significant bits.

- The most significant bit (msb) in each output byte indicates if there

is a continuation byte (msb = 1).

func EncodeNotNullAscending Uses

func EncodeNotNullAscending(b []byte) []byte

EncodeNotNullAscending encodes a value that is larger than the NULL marker encoded by EncodeNull but less than any encoded value returned by EncodeVarint, EncodeFloat, EncodeBytes or EncodeString.

func EncodeNotNullDescending Uses

func EncodeNotNullDescending(b []byte) []byte

EncodeNotNullDescending is the descending equivalent of EncodeNotNullAscending.

func EncodeNotNullValue Uses

func EncodeNotNullValue(appendTo []byte, colID uint32) []byte

EncodeNotNullValue encodes a not null value, appends it to the supplied buffer, and returns the final buffer.

func EncodeNullAscending Uses

func EncodeNullAscending(b []byte) []byte

EncodeNullAscending encodes a NULL value. The encodes bytes are appended to the supplied buffer and the final buffer is returned. The encoded value for a NULL is guaranteed to not be a prefix for the EncodeVarint, EncodeFloat, EncodeBytes and EncodeString encodings.

func EncodeNullDescending Uses

func EncodeNullDescending(b []byte) []byte

EncodeNullDescending is the descending equivalent of EncodeNullAscending.

func EncodeNullValue Uses

func EncodeNullValue(appendTo []byte, colID uint32) []byte

EncodeNullValue encodes a null value, appends it to the supplied buffer, and returns the final buffer.

func EncodeStringAscending Uses

func EncodeStringAscending(b []byte, s string) []byte

EncodeStringAscending encodes the string value using an escape-based encoding. See EncodeBytes for details. The encoded bytes are append to the supplied buffer and the resulting buffer is returned.

func EncodeStringDescending Uses

func EncodeStringDescending(b []byte, s string) []byte

EncodeStringDescending is the descending version of EncodeStringAscending.

func EncodeTimeAscending Uses

func EncodeTimeAscending(b []byte, t time.Time) []byte

EncodeTimeAscending encodes a time value, appends it to the supplied buffer, and returns the final buffer. The encoding is guaranteed to be ordered Such that if t1.Before(t2) then after EncodeTime(b1, t1), and EncodeTime(b2, t2), Compare(b1, b2) < 0. The time zone offset not included in the encoding.

func EncodeTimeDescending Uses

func EncodeTimeDescending(b []byte, t time.Time) []byte

EncodeTimeDescending is the descending version of EncodeTimeAscending.

func EncodeTimeTZAscending Uses

func EncodeTimeTZAscending(b []byte, t timetz.TimeTZ) []byte

EncodeTimeTZAscending encodes a timetz.TimeTZ value and appends it to the supplied buffer and returns the final buffer. The encoding is guaranteed to be ordered such that if t1.Before(t2) then after encodeTimeTZ(b1, t1) and encodeTimeTZ(b2, t2), Compare(b1, b2) < 0. The time zone offset is included in the encoding.

func EncodeTimeTZDescending Uses

func EncodeTimeTZDescending(b []byte, t timetz.TimeTZ) []byte

EncodeTimeTZDescending is the descending version of EncodeTimeTZAscending.

func EncodeTimeTZValue Uses

func EncodeTimeTZValue(appendTo []byte, colID uint32, t timetz.TimeTZ) []byte

EncodeTimeTZValue encodes a timetz.TimeTZ value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeTimeValue Uses

func EncodeTimeValue(appendTo []byte, colID uint32, t time.Time) []byte

EncodeTimeValue encodes a time.Time value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeTrueAscending Uses

func EncodeTrueAscending(b []byte) []byte

EncodeTrueAscending encodes the boolean value true for use with JSON inverted indexes.

func EncodeUUIDValue Uses

func EncodeUUIDValue(appendTo []byte, colID uint32, u uuid.UUID) []byte

EncodeUUIDValue encodes a uuid.UUID value with its value tag, appends it to the supplied buffer, and returns the final buffer.

func EncodeUint32Ascending Uses

func EncodeUint32Ascending(b []byte, v uint32) []byte

EncodeUint32Ascending encodes the uint32 value using a big-endian 4 byte representation. The bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUint32Descending Uses

func EncodeUint32Descending(b []byte, v uint32) []byte

EncodeUint32Descending encodes the uint32 value so that it sorts in reverse order, from largest to smallest.

func EncodeUint64Ascending Uses

func EncodeUint64Ascending(b []byte, v uint64) []byte

EncodeUint64Ascending encodes the uint64 value using a big-endian 8 byte representation. The bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUint64Descending Uses

func EncodeUint64Descending(b []byte, v uint64) []byte

EncodeUint64Descending encodes the uint64 value so that it sorts in reverse order, from largest to smallest.

func EncodeUntaggedBitArrayValue Uses

func EncodeUntaggedBitArrayValue(appendTo []byte, d bitarray.BitArray) []byte

EncodeUntaggedBitArrayValue encodes a bit array value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedBytesValue Uses

func EncodeUntaggedBytesValue(appendTo []byte, data []byte) []byte

EncodeUntaggedBytesValue encodes a byte array value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedDecimalValue Uses

func EncodeUntaggedDecimalValue(appendTo []byte, d *apd.Decimal) []byte

EncodeUntaggedDecimalValue encodes an apd.Decimal value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedDurationValue Uses

func EncodeUntaggedDurationValue(appendTo []byte, d duration.Duration) []byte

EncodeUntaggedDurationValue encodes a duration.Duration value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedFloatValue Uses

func EncodeUntaggedFloatValue(appendTo []byte, f float64) []byte

EncodeUntaggedFloatValue encodes a float value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedIPAddrValue Uses

func EncodeUntaggedIPAddrValue(appendTo []byte, u ipaddr.IPAddr) []byte

EncodeUntaggedIPAddrValue encodes a ipaddr.IPAddr value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedIntValue Uses

func EncodeUntaggedIntValue(appendTo []byte, i int64) []byte

EncodeUntaggedIntValue encodes an int value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedTimeTZValue Uses

func EncodeUntaggedTimeTZValue(appendTo []byte, t timetz.TimeTZ) []byte

EncodeUntaggedTimeTZValue encodes a time.Time value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedTimeValue Uses

func EncodeUntaggedTimeValue(appendTo []byte, t time.Time) []byte

EncodeUntaggedTimeValue encodes a time.Time value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUntaggedUUIDValue Uses

func EncodeUntaggedUUIDValue(appendTo []byte, u uuid.UUID) []byte

EncodeUntaggedUUIDValue encodes a uuid.UUID value, appends it to the supplied buffer, and returns the final buffer.

func EncodeUvarintAscending Uses

func EncodeUvarintAscending(b []byte, v uint64) []byte

EncodeUvarintAscending encodes the uint64 value using a variable length (length-prefixed) representation. The length is encoded as a single byte indicating the number of encoded bytes (-8) to follow. See EncodeVarintAscending for rationale. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeUvarintDescending Uses

func EncodeUvarintDescending(b []byte, v uint64) []byte

EncodeUvarintDescending encodes the uint64 value so that it sorts in reverse order, from largest to smallest.

func EncodeValueTag Uses

func EncodeValueTag(appendTo []byte, colID uint32, typ Type) []byte

EncodeValueTag encodes the prefix that is used by each of the EncodeFooValue methods.

The prefix uses varints to encode a column id and type, packing them into a single byte when they're small (colID < 8 and typ < 15). This works by shifting the colID "left" by 4 and putting any type less than 15 in the low bytes. The result is uvarint encoded and fits in one byte if the original column id fit in 3 bits. If it doesn't fit in one byte, the most significant bits spill to the "left", leaving the type bits always at the very "right".

If the type is > 15, the reserved sentinel of 15 is placed in the type bits and a uvarint follows with the type value. This means that there are always one or two uvarints.

Together, this means the everything but the last byte of the first uvarint can be dropped if the column id isn't needed.

func EncodeVarintAscending Uses

func EncodeVarintAscending(b []byte, v int64) []byte

EncodeVarintAscending encodes the int64 value using a variable length (length-prefixed) representation. The length is encoded as a single byte. If the value to be encoded is negative the length is encoded as 8-numBytes. If the value is positive it is encoded as 8+numBytes. The encoded bytes are appended to the supplied buffer and the final buffer is returned.

func EncodeVarintDescending Uses

func EncodeVarintDescending(b []byte, v int64) []byte

EncodeVarintDescending encodes the int64 value so that it sorts in reverse order, from largest to smallest.

func GetMultiVarintLen Uses

func GetMultiVarintLen(b []byte, num int) (int, error)

GetMultiVarintLen find the length of <num> encoded varints that follow a 1-byte tag.

func PeekLength Uses

func PeekLength(b []byte) (int, error)

PeekLength returns the length of the encoded value at the start of b. Note: if this function succeeds, it's not a guarantee that decoding the value will succeed.

func PeekLengthNonsortingUvarint Uses

func PeekLengthNonsortingUvarint(buf []byte) int

PeekLengthNonsortingUvarint returns the length of the value that starts at the beginning of buf and was encoded by EncodeNonsortingUvarint.

func PeekValueLength Uses

func PeekValueLength(b []byte) (typeOffset int, length int, err error)

PeekValueLength returns the length of the encoded value at the start of b. Note: If this function succeeds, it's not a guarantee that decoding the value will succeed.

`b` can point either at beginning of the "full tag" with the column id, or it can point to the beginning of the type part of the tag, as indicated by the `typeOffset` returned by this or DecodeValueTag.

The length returned is the full length of the encoded value, including the entire tag.

func PeekValueLengthWithOffsetsAndType Uses

func PeekValueLengthWithOffsetsAndType(b []byte, dataOffset int, typ Type) (length int, err error)

PeekValueLengthWithOffsetsAndType is the same as PeekValueLength, except it expects a dataOffset and typ value from a previous call to DecodeValueTag on its input byte slice. Use this if you've already called DecodeValueTag on the input for another reason, to avoid it getting called twice.

func PrettyPrintValue Uses

func PrettyPrintValue(valDirs []Direction, b []byte, sep string) string

PrettyPrintValue returns the string representation of all contiguous decodable values in the provided byte slice, separated by a provided separator. The directions each value is encoded may be provided. If valDirs is nil, all values are decoded and printed with the default direction (ascending).

func PrettyPrintValueEncoded Uses

func PrettyPrintValueEncoded(b []byte) ([]byte, string, error)

PrettyPrintValueEncoded returns a string representation of the first decodable value in the provided byte slice, along with the remaining byte slice after decoding.

func PrintableBytes Uses

func PrintableBytes(b []byte) bool

PrintableBytes returns true iff the given byte array is a valid UTF-8 sequence and it is printable.

func PutUint32Ascending Uses

func PutUint32Ascending(b []byte, v uint32, idx int) []byte

PutUint32Ascending encodes the uint32 value using a big-endian 4 byte representation at the specified index, lengthening the input slice if necessary.

func UndoPrefixEnd Uses

func UndoPrefixEnd(b []byte) (_ []byte, ok bool)

UndoPrefixEnd is a partial inverse for roachpb.Key.PrefixEnd.

In general, we can't undo PrefixEnd because it is lossy; we don't know how many FFs were stripped from the original key. For example:

- key:            01 02 03 FF FF
- PrefixEnd:      01 02 04
- UndoPrefixEnd:  01 02 03

Some keys are not possible results of PrefixEnd; in particular, PrefixEnd keys never end in 00. If an impossible key is passed, the second return value is false.

Specifically, calling UndoPrefixEnd will reverse the effects of calling a PrefixEnd on a byte sequence, except when the byte sequence represents a maximal prefix (i.e., 0xff...). This is because PrefixEnd is a lossy operation: PrefixEnd(0xff) returns 0xff rather than wrapping around to the minimal prefix 0x00. For consistency, UndoPrefixEnd is also lossy: UndoPrefixEnd(0x00) returns 0x00 rather than wrapping around to the maximal prefix 0xff.

Formally:

PrefixEnd(UndoPrefixEnd(p)) = p for all non-minimal prefixes p
UndoPrefixEnd(PrefixEnd(p)) = p for all non-maximal prefixes p

A minimal prefix is any prefix that consists only of one or more 0x00 bytes; analogously, a maximal prefix is any prefix that consists only of one or more 0xff bytes.

UndoPrefixEnd is implemented here to avoid a circular dependency on roachpb, but arguably belongs in a byte-manipulation utility package.

func UnsafeConvertStringToBytes Uses

func UnsafeConvertStringToBytes(s string) []byte

UnsafeConvertStringToBytes converts a string to a byte array to be used with string encoding functions. Note that the output byte array should not be modified if the input string is expected to be used again - doing so could violate Go semantics.

func UpperBoundNonsortingDecimalSize Uses

func UpperBoundNonsortingDecimalSize(d *apd.Decimal) int

UpperBoundNonsortingDecimalSize returns the upper bound number of bytes that the decimal will need for the non-sorting encoding.

func WordLen Uses

func WordLen(nat []big.Word) int

WordLen returns the size in bytes of the given array of Words.

type Direction Uses

type Direction int

Direction for ordering results.

const (
    Ascending Direction
    Descending
)

Direction values.

type Type Uses

type Type int

Type represents the type of a value encoded by Encode{Null,NotNull,Varint,Uvarint,Float,Bytes}. go:generate stringer -type=Type

const (
    Unknown   Type = 0
    Null      Type = 1
    NotNull   Type = 2
    Int       Type = 3
    Float     Type = 4
    Decimal   Type = 5
    Bytes     Type = 6
    BytesDesc Type = 7 // Bytes encoded descendingly
    Time      Type = 8
    Duration  Type = 9
    True      Type = 10
    False     Type = 11
    UUID      Type = 12
    Array     Type = 13
    IPAddr    Type = 14
    // SentinelType is used for bit manipulation to check if the encoded type
    // value requires more than 4 bits, and thus will be encoded in two bytes. It
    // is not used as a type value, and thus intentionally overlaps with the
    // subsequent type value. The 'Type' annotation is intentionally omitted here.
    SentinelType      = 15
    JSON         Type = 15
    Tuple        Type = 16
    BitArray     Type = 17
    BitArrayDesc Type = 18 // BitArray encoded descendingly
    TimeTZ       Type = 19
)

Type values. TODO(dan, arjun): Make this into a proto enum. The 'Type' annotations are necessary for producing stringer-generated values.

func DecodeValueTag Uses

func DecodeValueTag(b []byte) (typeOffset int, dataOffset int, colID uint32, typ Type, err error)

DecodeValueTag decodes a value encoded by EncodeValueTag, used as a prefix in each of the other EncodeFooValue methods.

The tag is structured such that the encoded column id can be dropped from the front by removing the first `typeOffset` bytes. DecodeValueTag, PeekValueLength and each of the DecodeFooValue methods will still work as expected with `b[typeOffset:]`. (Except, obviously, the column id is no longer encoded so if this suffix is passed back to DecodeValueTag, the returned colID should be discarded.)

Concretely:

b := ...
typeOffset, _, colID, typ, err := DecodeValueTag(b)
_, _, _, typ, err := DecodeValueTag(b[typeOffset:])

will return the same typ and err and

DecodeFooValue(b)
DecodeFooValue(b[typeOffset:])

will return the same thing. PeekValueLength works as expected with either of `b` or `b[typeOffset:]`.

func PeekType Uses

func PeekType(b []byte) Type

PeekType peeks at the type of the value encoded at the start of b.

func (Type) String Uses

func (i Type) String() string

Directories

PathSynopsis
csvPackage csv reads and writes comma-separated values (CSV) files.

Package encoding imports 22 packages (graph) and is imported by 369 packages. Updated 2019-12-11. Refresh now. Tools for package owners.