README
¶
cork
An efficient binary serialisation format for Go (Golang).
Features
- Simple and efficient encoding
- Based on MsgPack serialization algorithm
- Stores go type information inside encoded data
- Faster serialization than gob binary encoding
- More efficient output data size than gob binary encoding
- Serializes native go types, and arbritary structs or interfaces
- Enables predetermined encoding for structs without run-time reflection
- Allows serialization to and from maps, structs, slices, and nil interfaces
Installation
go get github.com/surrealdb/cork
Documentation
¶
Overview ¶
CORK is a dynamic and static binary object serialization specification similar to GOB and MessagePack. CORK manages encoding and decoding binary data streams, for use in transmission of, or storage of data.
Basics ¶
A CORK data stream is self-describing. Each data item in the stream is preceded by a description of its type, expressed in terms of a small set of predefined types. Pointers to values are not transmitted, but the contents are flattened and transmitted.
To use cork, create an Encoder and present it with a series of data items as values or addresses that can be dereferenced to values. The Encoder makes sure all type information is sent before it is needed. At the receive side, a Decoder retrieves values from the encoded stream and unpacks them into local variables.
Types ¶
CORK has built in support for the built-in Golang types
nil
bool
string
[]byte
int
int8
int16
int32
int64
uint
uint8
uint16
uint32
uint64
float32
float64
complex64
complex128
time.Time
interface{}
[]<T>
map[<T>]<T>
Structs ¶
When a struct is encountered whilst encoding (and that struct does not satisfy the Corker interface) then the struct will be encoded into the stream as a map with the keys encoded as strings, and the values as the relevant type. Any struct tags describing how the struct should be encoded will be used.
Corkers ¶
CORK allows applications to define application-specific types to be added to the encoding format. Each extended type must be assigned a unique byte from 0x00 upto 0xFF. Application-specific types (otherwise known as Corkers) are able to encode themselves into a binary data value, and are able to decode themselves from that same binary data value.
To define a custom type, an application must ensure that the type satisfies the Corker interface, and must then register the type using the Register method.
If a custom type is found in the stream when decoding, but no type with the specified unique byte is registered, then the binary data value will be decoded as a raw binary data value.
Types and Values ¶
The source and destination values/types need not correspond exactly. For structs, fields (identified by name) that are in the source but absent from the receiving variable will be ignored. Fields that are in the receiving variable but missing from the transmitted type or value will be ignored in the destination. If a field with the same name is present in both, their types must be compatible. Both the receiver and transmitter will do all necessary indirection and dereferencing to convert between cork encoded data and actual Go values. For instance, a cork type that is schematically,
struct { A, B int }
can be sent from or received into any of these Go types:
struct { A, B int } // the same
*struct { A, B int } // extra indirection of the struct
struct { *A, **B int } // extra indirection of the fields
struct { A, B int64 } // different concrete value type; see below
It may also be received into any of these:
struct { A, B int } // the same
struct { B, A int } // ordering doesn't matter; matching is by name
struct { A, B, C int } // extra field (C) ignored
struct { B int } // missing field (A) ignored; data will be dropped
struct { B, C int } // missing field (A) ignored; extra field (C) ignored.
Attempting to receive into these types will draw a decode error:
struct { A int; B uint } // change of signedness for B
struct { A int; B float } // change of type for B
struct { } // no field names in common
struct { C, D int } // no field names in common
Integers can be serialized using two different methods: arbritrary precision or full precision. When using arbritrary precision, integers (int, int8, int16, int32, int64) and unsinged integers (uint, uint8, uint16, uint32, uint64) are encoded with a variable-length encoding using as few bytes as possible, and are decoded into the destination variable, or an integer with the necessary capacity (when decoding into a nil interface). When using full precision, all integers (int8, int16, int32, int64) and unsinged integers (uint8, uint16, uint32, uint64) are encoded with a fixed-length encoding format, and are able to be decoded into the corresponding variable type when decoding into a nil interface.
Signed integers may be received into any signed integer variable: int, int16, etc.; unsigned integers may be received into any unsigned integer variable; and floating point values may be received into any floating point variable. However, the destination variable must be able to represent the value or the decode operation will fail.
Structs, arrays and slices are also supported. Structs encode and decode only exported fields. Struct tags (using the 'cork' descriptor) can specify custom key names to be used when serailizing, and can be omitted entirely, or when empty using the 'omitempty' tag keyword. Strings and arrays of bytes are supported with a special, efficient representation. When a slice is decoded, if the existing slice has capacity the slice will be extended in place; if not, a new array is allocated. Regardless, the length of the resulting slice reports the number of elements decoded.
In general, if allocation is required, the decoder will allocate memory. If not, it will update the destination variables with values read from the stream. It does not initialize them first, so if the destination is a compound value such as a map, struct, or slice, the decoded values will be merged elementwise into the existing variables.
Functions and channels will not be encoded into a CORK. Attempting to encode such a value at the top level will fail. A struct field of chan or func type is treated exactly like an unexported field and is ignored.
Specification ¶
The full specification can be found at http://github.com/surrealdb/cork/SPEC.md
Index ¶
- func Decode(src []byte) (dst interface{})
- func DecodeInto(src []byte, dst interface{})
- func Encode(src interface{}) (dst []byte)
- func EncodeInto(src interface{}, dst *[]byte)
- func Register(value interface{})
- type Corker
- type Decoder
- type Encoder
- type Handle
- type Reader
- func (r *Reader) DecodeAny(v interface{})
- func (r *Reader) DecodeArr(v interface{})
- func (r *Reader) DecodeBool(v *bool)
- func (r *Reader) DecodeByte(v *byte)
- func (r *Reader) DecodeBytes(v *[]byte)
- func (r *Reader) DecodeComplex128(v *complex128)
- func (r *Reader) DecodeComplex64(v *complex64)
- func (r *Reader) DecodeCorker(v Corker)
- func (r *Reader) DecodeFloat32(v *float32)
- func (r *Reader) DecodeFloat64(v *float64)
- func (r *Reader) DecodeInt(v *int)
- func (r *Reader) DecodeInt16(v *int16)
- func (r *Reader) DecodeInt32(v *int32)
- func (r *Reader) DecodeInt64(v *int64)
- func (r *Reader) DecodeInt8(v *int8)
- func (r *Reader) DecodeInterface(v *interface{})
- func (r *Reader) DecodeMap(v interface{})
- func (r *Reader) DecodeReflect(v reflect.Value)
- func (r *Reader) DecodeSelfer(v Selfer)
- func (r *Reader) DecodeString(v *string)
- func (r *Reader) DecodeTime(v *time.Time)
- func (r *Reader) DecodeUint(v *uint)
- func (r *Reader) DecodeUint16(v *uint16)
- func (r *Reader) DecodeUint32(v *uint32)
- func (r *Reader) DecodeUint64(v *uint64)
- func (r *Reader) DecodeUint8(v *uint8)
- type Selfer
- type Writer
- func (w *Writer) EncodeAny(v interface{})
- func (w *Writer) EncodeArr(v interface{})
- func (w *Writer) EncodeBool(v bool)
- func (w *Writer) EncodeByte(v byte)
- func (w *Writer) EncodeBytes(v []byte)
- func (w *Writer) EncodeComplex128(v complex128)
- func (w *Writer) EncodeComplex64(v complex64)
- func (w *Writer) EncodeCorker(v Corker)
- func (w *Writer) EncodeFloat32(v float32)
- func (w *Writer) EncodeFloat64(v float64)
- func (w *Writer) EncodeInt(v int)
- func (w *Writer) EncodeInt16(v int16)
- func (w *Writer) EncodeInt32(v int32)
- func (w *Writer) EncodeInt64(v int64)
- func (w *Writer) EncodeInt8(v int8)
- func (w *Writer) EncodeMap(v interface{})
- func (w *Writer) EncodeNil()
- func (w *Writer) EncodeReflect(v reflect.Value)
- func (w *Writer) EncodeSelfer(v Selfer)
- func (w *Writer) EncodeString(v string)
- func (w *Writer) EncodeTime(v time.Time)
- func (w *Writer) EncodeUint(v uint)
- func (w *Writer) EncodeUint16(v uint16)
- func (w *Writer) EncodeUint32(v uint32)
- func (w *Writer) EncodeUint64(v uint64)
- func (w *Writer) EncodeUint8(v uint8)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DecodeInto ¶
func DecodeInto(src []byte, dst interface{})
DecodeInto decodes a byte slice into a Go object.
func EncodeInto ¶
func EncodeInto(src interface{}, dst *[]byte)
EncodeInto encodes a Go object into a byte slice.
Types ¶
type Corker ¶
type Corker interface {
ExtendCORK() byte
MarshalCORK() ([]byte, error)
UnmarshalCORK([]byte) error
}
Corker represents an object which can encode and decode itself.
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
Decoder represents a CORK decoder.
func NewDecoder ¶
NewDecoder returns a Decoder for decoding from an io.Reader.
func NewDecoderBytes ¶
NewDecoderBytes returns a Decoder for decoding directly from a byte slice.
func NewDecoderBytesFromPool ¶
NewDecoderBytesFromPool returns a Decoder for decoding directly from a byte slice. The Decoder is taken from a pool of decoders, and must be put back when finished, using d.Reset().
func NewDecoderFromPool ¶
NewDecoderFromPool returns a Decoder for decoding into an io.Reader. The Decoder is taken from a pool of decoders, and must be put back when finished, using d.Reset().
func (*Decoder) Decode ¶
Decode decodes the stream into the 'dst' object.
The decoder can not decode into a nil pointer, but can decode into a nil interface. If you do not know what type of stream it is, pass in a pointer to a nil interface. We will decode and store a value in that nil interface.
When decoding into a nil interface{}, we will decode into an appropriate value based on the contents of the stream. When decoding into a non-nil interface{} value, the mode of encoding is based on the type of the value.
Example:
// Decoding into a non-nil typed value
var s string
buf := bytes.NewBuffer(src)
err = cork.NewDecoder(buf).Decode(&s)
// Decoding into nil interface
var v interface{}
buf := bytes.NewBuffer(src)
err := cork.NewDecoder(buf).Decode(&v)
type Encoder ¶
type Encoder struct {
// contains filtered or unexported fields
}
Encoder represents a CORK encoder.
func NewEncoder ¶
NewEncoder returns an Encoder for encoding into an io.Writer.
func NewEncoderBytes ¶
NewEncoderBytes returns an Encoder for encoding directly into a byte slice.
func NewEncoderBytesFromPool ¶
NewEncoderBytesFromPool returns an Encoder for encoding directly into a byte slice. The Encoder is taken from a pool of encoders, and must be put back when finished, using e.Reset().
func NewEncoderFromPool ¶
NewEncoderFromPool returns an Encoder for encoding into an io.Writer. The Encoder is taken from a pool of encoders, and must be put back when finished, using e.Reset().
func (*Encoder) Encode ¶
Encode encodes the 'src' object into the stream.
The decoder can be configured using struct tags. The 'cork' key if found will be analysed for any configuration options when encoding.
Each exported struct field is encoded unless:
- the field's tag is "-"
- the field is empty and its tag specifies the "omitempty" option.
When encoding a struct as a map, the first string in the tag (before the comma) will be used for the map key, and if not specified will default to the struct key name.
The empty values (for omitempty option) are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero.
type Tester struct {
Test bool `cork:"-"` // Skip this field
Name string `cork:"name"` // Use key "name" in encode stream
Size int32 `cork:"size"` // Use key "size" in encode stream
Data []byte `cork:"data,omitempty"` // Use key data in encode stream, and omit if empty
}
Example:
// Encoding a typed value
var s string = "Hello"
buf := bytes.NewBuffer(nil)
err = cork.NewEncoder(buf).Encode(s)
// Encoding a struct
var t &Tester{Name: "Temp", Size: 0}
buf := bytes.NewBuffer(nil)
err = cork.NewEncoder(buf).Encode(t)
type Handle ¶
type Handle struct {
// SortMaps specifies whether maps should be sorted before
// being encoded into CORK. This guarantees that the same
// input data is always encoded into the same binary data.
SortMaps bool
// ArrType specifies the type of slice to use when decoding
// into a nil interface during schema-less decoding of a
// slice in the stream.
//
// If not specified, we use []interface{}
ArrType interface{}
// MapType specifies the type of map to use when decoding
// into a nil interface during schema-less decoding of a
// map in the stream.
//
// If not specified, we use map[interface{}]interface{}
MapType interface{}
}
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader is used when self-decoding a cork.Selfer item from binary form.
func (*Reader) DecodeAny ¶
func (r *Reader) DecodeAny(v interface{})
DecodeAny decodes a value from the Reader.
func (*Reader) DecodeArr ¶
func (r *Reader) DecodeArr(v interface{})
DecodeArr decodes an array from the Reader.
func (*Reader) DecodeBool ¶
DecodeBool decodes a boolean value from the Reader.
func (*Reader) DecodeByte ¶
DecodeByte decodes a byte value from the Reader.
func (*Reader) DecodeBytes ¶
DecodeBytes decodes a byte slice value from the Reader.
func (*Reader) DecodeComplex128 ¶
func (r *Reader) DecodeComplex128(v *complex128)
DecodeComplex128 decodes a complex128 value from the Reader.
func (*Reader) DecodeComplex64 ¶
DecodeComplex64 decodes a complex64 value from the Reader.
func (*Reader) DecodeCorker ¶
DecodeCorker decodes a cork.Corker value from the Reader.
func (*Reader) DecodeFloat32 ¶
DecodeFloat32 decodes a float32 value from the Reader.
func (*Reader) DecodeFloat64 ¶
DecodeFloat64 decodes a float64 value from the Reader.
func (*Reader) DecodeInt16 ¶
DecodeInt16 decodes an int16 value from the Reader.
func (*Reader) DecodeInt32 ¶
DecodeInt32 decodes an int32 value from the Reader.
func (*Reader) DecodeInt64 ¶
DecodeInt64 decodes an int64 value from the Reader.
func (*Reader) DecodeInt8 ¶
DecodeInt8 decodes an int8 value from the Reader.
func (*Reader) DecodeInterface ¶
func (r *Reader) DecodeInterface(v *interface{})
DecodeInterface decodes any value from the Reader, with whatever type is next in the stream.
func (*Reader) DecodeMap ¶
func (r *Reader) DecodeMap(v interface{})
DecodeMap decodes a map from the Reader.
func (*Reader) DecodeReflect ¶
DecodeReflect decodes a reflect.Value value from the Reader.
func (*Reader) DecodeSelfer ¶
DecodeSelfer decodes a cork.Selfer value from the Reader.
func (*Reader) DecodeString ¶
DecodeString decodes a string value from the Reader.
func (*Reader) DecodeTime ¶
DecodeTime decodes a time.Time value from the Reader.
func (*Reader) DecodeUint ¶
DecodeUint decodes a uint value from the Reader.
func (*Reader) DecodeUint16 ¶
DecodeUint16 decodes a uint16 value from the Reader.
func (*Reader) DecodeUint32 ¶
DecodeUint32 decodes a uint32 value from the Reader.
func (*Reader) DecodeUint64 ¶
DecodeUint64 decodes a uint64 value from the Reader.
func (*Reader) DecodeUint8 ¶
DecodeUint8 decodes a uint8 value from the Reader.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer is used when self-encoding a cork.Selfer item into binary form.
func (*Writer) EncodeArr ¶
func (w *Writer) EncodeArr(v interface{})
EncodeArr encodes an array to the Writer.
func (*Writer) EncodeBool ¶
EncodeBool encodes a boolean value to the Writer.
func (*Writer) EncodeByte ¶
EncodeByte encodes a byte value to the Writer.
func (*Writer) EncodeBytes ¶
EncodeBytes encodes a byte slice value to the Writer.
func (*Writer) EncodeComplex128 ¶
func (w *Writer) EncodeComplex128(v complex128)
EncodeComplex128 encodes a complex128 value to the Writer.
func (*Writer) EncodeComplex64 ¶
EncodeComplex64 encodes a complex64 value to the Writer.
func (*Writer) EncodeCorker ¶
EncodeCorker encodes a cork.Corker value to the Writer.
func (*Writer) EncodeFloat32 ¶
EncodeFloat32 encodes a float32 value to the Writer.
func (*Writer) EncodeFloat64 ¶
EncodeFloat64 encodes a float64 value to the Writer.
func (*Writer) EncodeInt16 ¶
EncodeInt16 encodes an int16 value to the Writer.
func (*Writer) EncodeInt32 ¶
EncodeInt32 encodes an int32 value to the Writer.
func (*Writer) EncodeInt64 ¶
EncodeInt64 encodes an int64 value to the Writer.
func (*Writer) EncodeInt8 ¶
EncodeInt8 encodes an int8 value to the Writer.
func (*Writer) EncodeMap ¶
func (w *Writer) EncodeMap(v interface{})
EncodeMap encodes a map to the Writer.
func (*Writer) EncodeNil ¶
func (w *Writer) EncodeNil()
EncodeBool writes a nil value to the Writer.
func (*Writer) EncodeReflect ¶
EncodeReflect encodes a reflect.Value value to the Writer.
func (*Writer) EncodeSelfer ¶
EncodeSelfer encodes a cork.Selfer value to the Writer.
func (*Writer) EncodeString ¶
EncodeString encodes a string value to the Writer.
func (*Writer) EncodeTime ¶
EncodeInt encodes a time.Time value to the Writer.
func (*Writer) EncodeUint ¶
EncodeUint encodes a uint value to the Writer.
func (*Writer) EncodeUint16 ¶
EncodeUint16 encodes a uint16 value to the Writer.
func (*Writer) EncodeUint32 ¶
EncodeUint32 encodes a uint32 value to the Writer.
func (*Writer) EncodeUint64 ¶
EncodeUint64 encodes a uint64 value to the Writer.
func (*Writer) EncodeUint8 ¶
EncodeUint8 encodes a uint8 value to the Writer.
Source Files