Documentation
¶
Overview ¶
Package serialize provides methods for reading and writing concatenable, bytewise-sortable forms of the datatypes defined in the datastore package.
Index ¶
- Constants
- Variables
- func Increment(bstr []byte) ([]byte, bool)
- func Invert(bs []byte) []byte
- func Join(itms ...[]byte) []byte
- func ReadGeoPoint(buf ReadBuffer) (gp ds.GeoPoint, err error)
- func ReadIndexColumn(buf ReadBuffer) (c ds.IndexColumn, err error)
- func ReadIndexDefinition(buf ReadBuffer) (i ds.IndexDefinition, err error)
- func ReadKey(buf ReadBuffer, context KeyContext, inKC ds.KeyContext) (ret *ds.Key, err error)
- func ReadKeyTok(buf ReadBuffer) (ret ds.KeyTok, err error)
- func ReadProperty(buf ReadBuffer, context KeyContext, kc ds.KeyContext) (p ds.Property, err error)
- func ReadPropertyMap(buf ReadBuffer, context KeyContext, kc ds.KeyContext) (pm ds.PropertyMap, err error)
- func ReadTime(buf ReadBuffer) (time.Time, error)
- func ToBytes(i interface{}) []byte
- func ToBytesErr(i interface{}) ([]byte, error)
- func ToBytesWithContext(i interface{}) []byte
- func ToBytesWithContextErr(i interface{}) ([]byte, error)
- func WriteGeoPoint(buf WriteBuffer, gp ds.GeoPoint) (err error)
- func WriteIndexColumn(buf WriteBuffer, c ds.IndexColumn) (err error)
- func WriteIndexDefinition(buf WriteBuffer, i ds.IndexDefinition) (err error)
- func WriteIndexProperty(buf WriteBuffer, context KeyContext, p ds.Property) error
- func WriteKey(buf WriteBuffer, context KeyContext, k *ds.Key) (err error)
- func WriteKeyTok(buf WriteBuffer, tok ds.KeyTok) (err error)
- func WriteProperty(buf WriteBuffer, context KeyContext, p ds.Property) error
- func WritePropertyMap(buf WriteBuffer, context KeyContext, pm ds.PropertyMap) (err error)
- func WriteTime(buf WriteBuffer, t time.Time) error
- type InvertibleBuffer
- type KeyContext
- type ReadBuffer
- type SerializedPmap
- type SerializedPslice
- type WriteBuffer
Constants ¶
const MaxIndexColumns = 64
MaxIndexColumns is the maximum number of sort columns (e.g. sort orders) that ReadIndexDefinition is willing to deserialize. 64 was chosen as a likely-astronomical number.
const ReadKeyNumToksReasonableLimit = 50
ReadKeyNumToksReasonableLimit is the maximum number of Key tokens that ReadKey is willing to read for a single key.
const ReadPropertyMapReasonableLimit uint64 = 30000
ReadPropertyMapReasonableLimit sets a limit on the number of rows and number of properties per row which can be read by ReadPropertyMap. The total number of Property objects readable by this method is this number squared (e.g. Limit rows * Limit properties)
Variables ¶
var WritePropertyMapDeterministic = false
WritePropertyMapDeterministic allows tests to make WritePropertyMap deterministic.
Functions ¶
func Increment ¶
Increment attempts to increment a copy of bstr as if adding 1 to an integer.
If it overflows, the returned []byte will be all 0's, and the overflow bool will be true.
func ReadGeoPoint ¶
func ReadGeoPoint(buf ReadBuffer) (gp ds.GeoPoint, err error)
ReadGeoPoint reads a GeoPoint from the buffer.
func ReadIndexColumn ¶
func ReadIndexColumn(buf ReadBuffer) (c ds.IndexColumn, err error)
ReadIndexColumn reads an IndexColumn from the buffer.
func ReadIndexDefinition ¶
func ReadIndexDefinition(buf ReadBuffer) (i ds.IndexDefinition, err error)
ReadIndexDefinition reads an IndexDefinition from the buffer.
func ReadKey ¶
func ReadKey(buf ReadBuffer, context KeyContext, inKC ds.KeyContext) (ret *ds.Key, err error)
ReadKey deserializes a key from the buffer. The value of context must match the value of context that was passed to WriteKey when the key was encoded. If context == WithoutContext, then the appid and namespace parameters are used in the decoded Key. Otherwise they're ignored.
func ReadKeyTok ¶
func ReadKeyTok(buf ReadBuffer) (ret ds.KeyTok, err error)
ReadKeyTok reads a KeyTok from the buffer. You usually want ReadKey instead of this.
func ReadProperty ¶
func ReadProperty(buf ReadBuffer, context KeyContext, kc ds.KeyContext) (p ds.Property, err error)
ReadProperty reads a Property from the buffer. `context` and `kc` behave the same way they do for ReadKey, but only have an effect if the decoded property has a Key value.
func ReadPropertyMap ¶
func ReadPropertyMap(buf ReadBuffer, context KeyContext, kc ds.KeyContext) (pm ds.PropertyMap, err error)
ReadPropertyMap reads a PropertyMap from the buffer. `context` and friends behave the same way that they do for ReadKey.
func ReadTime ¶
func ReadTime(buf ReadBuffer) (time.Time, error)
ReadTime reads a time.Time from the buffer.
func ToBytes ¶
func ToBytes(i interface{}) []byte
ToBytes serializes i to a byte slice, if it's one of the type supported by this library. If an error is encountered (e.g. `i` is not a supported type), this method panics.
Key types will be serialized using the 'WithoutContext' option (e.g. their encoded forms will not contain AppID or Namespace).
func ToBytesErr ¶
ToBytesErr serializes i to a byte slice, if it's one of the type supported by this library, otherwise it returns an error.
Key types will be serialized using the 'WithoutContext' option (e.g. their encoded forms will not contain AppID or Namespace).
func ToBytesWithContext ¶
func ToBytesWithContext(i interface{}) []byte
ToBytesWithContext serializes i to a byte slice, if it's one of the type supported by this library. If an error is encountered (e.g. `i` is not a supported type), this method panics.
Key types will be serialized using the 'WithContext' option (e.g. their encoded forms will not contain AppID or Namespace).
func ToBytesWithContextErr ¶
ToBytesWithContextErr serializes i to a byte slice, if it's one of the type supported by this library, otherwise it returns an error.
Key types will be serialized using the 'WithContext' option (e.g. their encoded forms will contain AppID and Namespace).
func WriteGeoPoint ¶
func WriteGeoPoint(buf WriteBuffer, gp ds.GeoPoint) (err error)
WriteGeoPoint writes a GeoPoint to the buffer.
func WriteIndexColumn ¶
func WriteIndexColumn(buf WriteBuffer, c ds.IndexColumn) (err error)
WriteIndexColumn writes an IndexColumn to the buffer.
func WriteIndexDefinition ¶
func WriteIndexDefinition(buf WriteBuffer, i ds.IndexDefinition) (err error)
WriteIndexDefinition writes an IndexDefinition to the buffer
func WriteIndexProperty ¶
func WriteIndexProperty(buf WriteBuffer, context KeyContext, p ds.Property) error
WriteIndexProperty writes a Property to the buffer as its native index type. `context` behaves the same way that it does for WriteKey, but only has an effect if `p` contains a Key as its IndexValue.
func WriteKey ¶
func WriteKey(buf WriteBuffer, context KeyContext, k *ds.Key) (err error)
WriteKey encodes a key to the buffer. If context is WithContext, then this encoded value will include the appid and namespace of the key.
func WriteKeyTok ¶
func WriteKeyTok(buf WriteBuffer, tok ds.KeyTok) (err error)
WriteKeyTok writes a KeyTok to the buffer. You usually want WriteKey instead of this.
func WriteProperty ¶
func WriteProperty(buf WriteBuffer, context KeyContext, p ds.Property) error
WriteProperty writes a Property to the buffer. `context` behaves the same way that it does for WriteKey, but only has an effect if `p` contains a Key as its IndexValue.
func WritePropertyMap ¶
func WritePropertyMap(buf WriteBuffer, context KeyContext, pm ds.PropertyMap) (err error)
WritePropertyMap writes an entire PropertyMap to the buffer. `context` behaves the same way that it does for WriteKey.
If WritePropertyMapDeterministic is true, then the rows will be sorted by property name before they're serialized to buf (mostly useful for testing, but also potentially useful if you need to make a hash of the property data).
Write skips metadata keys.
Types ¶
type InvertibleBuffer ¶
type InvertibleBuffer interface {
WriteBuffer
SetInvert(inverted bool)
}
InvertibleBuffer is just like Buffer, except that it also has a stateful Invert() method, which will cause all reads and writes to/from it to be inverted (e.g. every byte XOR 0xFF).
Implementing queries requires manipulating the index entries (e.g. synthesizing them, parsing them, etc.). In particular, when you have a reverse-sorted field (e.g. high to low instead of low to high), it's achieved by having all the bits inverted.
All the serialization formats include delimiter information, which the parsers only know to parse non-inverted. If we don't have this buffer, we'd basically have to invert every byte in the []byte array when we're trying to decode a reverse-ordered field (including the bytes of all fields after the one we intend to parse) so that the parser can consume as many bytes as it needs (and it only knows the number of bytes it needs as it decodes them). This InvertibleBuffer lets that happen on the fly without having to flip the whole []byte.
If you know you need it, you'll know it's the right thing. If you're not sure then you definitely don't need it!
func Invertible ¶
func Invertible(b WriteBuffer) InvertibleBuffer
Invertible returns an InvertibleBuffer based on the Buffer.
type KeyContext ¶
type KeyContext bool
KeyContext controls whether the various Write and Read serializtion routines should encode the context of Keys (read: the appid and namespace). Frequently the appid and namespace of keys are known in advance and so there's no reason to redundantly encode them.
const ( WithContext KeyContext = true WithoutContext = false )
With- and WithoutContext indicate if the serialization method should include context for Keys. See KeyContext for more information.
type ReadBuffer ¶
ReadBuffer is the interface which corresponds to the subset of *bytes.Reader that this package requires.
type SerializedPmap ¶
type SerializedPmap map[string]SerializedPslice
SerializedPmap maps from
prop name -> [<serialized DSProperty>, ...]
includes special values '__key__' and '__ancestor__' which contains all of the ancestor entries for this key.
func PropertyMapPartially ¶
func PropertyMapPartially(k *ds.Key, pm ds.PropertyMap) (ret SerializedPmap)
PropertyMapPartially turns a regular PropertyMap into a SerializedPmap. Essentially all the []Property's become SerializedPslice, using cmpbin and datastore/serialize's encodings.
type SerializedPslice ¶
type SerializedPslice [][]byte
SerializedPslice is all of the serialized DSProperty values in ASC order.
func PropertySlice ¶
func PropertySlice(vals ds.PropertySlice) SerializedPslice
PropertySlice serializes a single row of a DSProperty map.
It does not differentiate between single- and multi- properties.
func (SerializedPslice) Len ¶
func (s SerializedPslice) Len() int
func (SerializedPslice) Less ¶
func (s SerializedPslice) Less(i, j int) bool
func (SerializedPslice) Swap ¶
func (s SerializedPslice) Swap(i, j int)
Source Files