go-amino: github.com/tendermint/go-amino Index | Examples | Files | Directories

package amino

import "github.com/tendermint/go-amino"

Amino is an encoding library that can handle interfaces (like protobuf "oneof") well. This is achieved by prefixing bytes before each "concrete type".

A concrete type is some non-interface value (generally a struct) which implements the interface to be (de)serialized. Not all structures need to be registered as concrete types -- only when they will be stored in interface type fields (or interface type slices) do they need to be registered.

Registering types

All interfaces and the concrete types that implement them must be registered.

amino.RegisterInterface((*MyInterface1)(nil), nil)
amino.RegisterInterface((*MyInterface2)(nil), nil)
amino.RegisterConcrete(MyStruct1{}, "com.tendermint/MyStruct1", nil)
amino.RegisterConcrete(MyStruct2{}, "com.tendermint/MyStruct2", nil)
amino.RegisterConcrete(&MyStruct3{}, "anythingcangoinhereifitsunique", nil)

Notice that an interface is represented by a nil pointer.

Structures that must be deserialized as pointer values must be registered with a pointer value as well. It's OK to (de)serialize such structures in non-pointer (value) form, but when deserializing such structures into an interface field, they will always be deserialized as pointers.

How it works

All registered concrete types are encoded with leading 4 bytes (called "prefix bytes"), even when it's not held in an interface field/element. In this way, Amino ensures that concrete types (almost) always have the same canonical representation. The first byte of the prefix bytes must not be a zero byte, so there are 2**(8*4)-2**(8*3) possible values.

When there are 4096 types registered at once, the probability of there being a conflict is ~ 0.2%. See https://instacalc.com/51189 for estimation. This is assuming that all registered concrete types have unique natural names (e.g. prefixed by a unique entity name such as "com.tendermint/", and not "mined/grinded" to produce a particular sequence of "prefix bytes").

TODO Update instacalc.com link with 255/256 since 0x00 is an escape.

Do not mine/grind to produce a particular sequence of prefix bytes, and avoid using dependencies that do so.

Since 4 bytes are not sufficient to ensure no conflicts, sometimes it is necessary to prepend more than the 4 prefix bytes for disambiguation. Like the prefix bytes, the disambiguation bytes are also computed from the registered name of the concrete type. There are 3 disambiguation bytes, and in binary form they always precede the prefix bytes. The first byte of the disambiguation bytes must not be a zero byte, so there are 2**(8*3)-2**(8*2) possible values.

// Sample Amino encoded binary bytes with 4 prefix bytes.
> [0xBB 0x9C 0x83 0xDD] [...]

// Sample Amino encoded binary bytes with 3 disambiguation bytes and 4
// prefix bytes.
> 0x00 <0xA8 0xFC 0x54> [0xBB 0x9C 0x83 0xDD] [...]

The prefix bytes never start with a zero byte, so the disambiguation bytes are escaped with 0x00.

Notice that the 4 prefix bytes always immediately precede the binary encoding of the concrete type.

Computing prefix bytes

To compute the disambiguation bytes, we take `hash := sha256(concreteTypeName)`, and drop the leading 0x00 bytes.

> hash := sha256("com.tendermint.consensus/MyConcreteName")
> hex.EncodeBytes(hash) // 0x{00 00 A8 FC 54 00 00 00 BB 9C 83 DD ...} (example)

In the example above, hash has two leading 0x00 bytes, so we drop them.

> rest = dropLeadingZeroBytes(hash) // 0x{A8 FC 54 00 00 BB 9C 83 DD ...}
> disamb = rest[0:3]
> rest = dropLeadingZeroBytes(rest[3:])
> prefix = rest[0:4]

The first 3 bytes are called the "disambiguation bytes" (in angle brackets). The next 4 bytes are called the "prefix bytes" (in square brackets).

> <0xA8 0xFC 0x54> [0xBB 0x9C 9x83 9xDD]

Code:

defer func() {
    if e := recover(); e != nil {
        fmt.Println("Recovered:", e)
    }
}()

type Message interface{}

type bcMessage struct {
    Message string
    Height  int
}

type bcResponse struct {
    Status  int
    Message string
}

type bcStatus struct {
    Peers int
}

var cdc = amino.NewCodec()
cdc.RegisterInterface((*Message)(nil), nil)
cdc.RegisterConcrete(&bcMessage{}, "bcMessage", nil)
cdc.RegisterConcrete(&bcResponse{}, "bcResponse", nil)
cdc.RegisterConcrete(&bcStatus{}, "bcStatus", nil)

var bm = &bcMessage{Message: "ABC", Height: 100}
var msg = bm

var bz []byte // the marshalled bytes.
var err error
bz, err = cdc.MarshalBinaryLengthPrefixed(msg)
fmt.Printf("Encoded: %X (err: %v)\n", bz, err)

var msg2 Message
err = cdc.UnmarshalBinaryLengthPrefixed(bz, &msg2)
fmt.Printf("Decoded: %v (err: %v)\n", msg2, err)
var bm2 = msg2.(*bcMessage)
fmt.Printf("Decoded successfully: %v\n", *bm == *bm2)

Output:

Encoded: 0B740613650A034142431064 (err: <nil>)
Decoded: &{ABC 100} (err: <nil>)
Decoded successfully: true

Index

Examples

Package Files

amino.go binary-decode.go binary-encode.go codec.go decoder.go deep_copy.go doc.go encoder.go json-decode.go json-encode.go reflect.go version.go

Constants

const (
    // Typ3 types
    Typ3Varint     = Typ3(0)
    Typ38Byte      = Typ3(1)
    Typ3ByteLength = Typ3(2)
    //Typ3_Struct     = Typ3(3)
    //Typ3_StructTerm = Typ3(4)
    Typ3_4Byte = Typ3(5)
)
const (
    PrefixBytesLen = 4
    DisambBytesLen = 3
    DisfixBytesLen = PrefixBytesLen + DisambBytesLen
)

Lengths

const Version = "0.15.0"

Version

Variables

var (

    // ErrNoPointer is thrown when you call a method that expects a pointer, e.g. Unmarshal
    ErrNoPointer = errors.New("expected a pointer")
)
var (
    ErrOverflowInt = errors.New("encoded integer value overflows int(32)")
)

func ByteSliceSize Uses

func ByteSliceSize(bz []byte) int

func DecodeBool Uses

func DecodeBool(bz []byte) (b bool, n int, err error)

func DecodeByte Uses

func DecodeByte(bz []byte) (b byte, n int, err error)

func DecodeByteSlice Uses

func DecodeByteSlice(bz []byte) (bz2 []byte, n int, err error)

func DecodeDisambPrefixBytes Uses

func DecodeDisambPrefixBytes(bz []byte) (db DisambBytes, hasDb bool, pb PrefixBytes, hasPb bool, n int, err error)

func DecodeFloat32 Uses

func DecodeFloat32(bz []byte) (f float32, n int, err error)

NOTE: UNSAFE

func DecodeFloat64 Uses

func DecodeFloat64(bz []byte) (f float64, n int, err error)

NOTE: UNSAFE

func DecodeInt16 Uses

func DecodeInt16(bz []byte) (i int16, n int, err error)

func DecodeInt32 Uses

func DecodeInt32(bz []byte) (i int32, n int, err error)

func DecodeInt64 Uses

func DecodeInt64(bz []byte) (i int64, n int, err error)

func DecodeInt8 Uses

func DecodeInt8(bz []byte) (i int8, n int, err error)

func DecodeString Uses

func DecodeString(bz []byte) (s string, n int, err error)

func DecodeTime Uses

func DecodeTime(bz []byte) (t time.Time, n int, err error)

DecodeTime decodes seconds (int64) and nanoseconds (int32) since January 1, 1970 UTC, and returns the corresponding time. If nanoseconds is not in the range [0, 999999999], or if seconds is too large, the behavior is undefined. TODO return error if behavior is undefined.

func DecodeUint16 Uses

func DecodeUint16(bz []byte) (u uint16, n int, err error)

func DecodeUint32 Uses

func DecodeUint32(bz []byte) (u uint32, n int, err error)

func DecodeUint64 Uses

func DecodeUint64(bz []byte) (u uint64, n int, err error)

func DecodeUint8 Uses

func DecodeUint8(bz []byte) (u uint8, n int, err error)

func DecodeUvarint Uses

func DecodeUvarint(bz []byte) (u uint64, n int, err error)

func DecodeVarint Uses

func DecodeVarint(bz []byte) (i int64, n int, err error)

func DeepCopy Uses

func DeepCopy(o interface{}) (r interface{})

Deeply copies an object. If anything implements `.DeepCopy() <any>` along the way, the result of that function will be used. Otherwise, if it implements `.MarshalAmino() (<any>, error)` and `.UnmarshalAmino(<any>) error`, the pair will be used to copy. If .MarshalAmino() or .UnmarshalAmino() returns an error, this function will panic.

func EncodeBool Uses

func EncodeBool(w io.Writer, b bool) (err error)

func EncodeByte Uses

func EncodeByte(w io.Writer, b byte) (err error)

func EncodeByteSlice Uses

func EncodeByteSlice(w io.Writer, bz []byte) (err error)

func EncodeFloat32 Uses

func EncodeFloat32(w io.Writer, f float32) (err error)

NOTE: UNSAFE

func EncodeFloat64 Uses

func EncodeFloat64(w io.Writer, f float64) (err error)

NOTE: UNSAFE

func EncodeInt16 Uses

func EncodeInt16(w io.Writer, i int16) (err error)

func EncodeInt32 Uses

func EncodeInt32(w io.Writer, i int32) (err error)

func EncodeInt64 Uses

func EncodeInt64(w io.Writer, i int64) (err error)

func EncodeInt8 Uses

func EncodeInt8(w io.Writer, i int8) (err error)

func EncodeString Uses

func EncodeString(w io.Writer, s string) (err error)

func EncodeTime Uses

func EncodeTime(w io.Writer, t time.Time) (err error)

EncodeTime writes the number of seconds (int64) and nanoseconds (int32), with millisecond resolution since January 1, 1970 UTC to the Writer as an UInt64. Milliseconds are used to ease compatibility with Javascript, which does not support finer resolution.

func EncodeUint16 Uses

func EncodeUint16(w io.Writer, u uint16) (err error)

func EncodeUint32 Uses

func EncodeUint32(w io.Writer, u uint32) (err error)

func EncodeUint64 Uses

func EncodeUint64(w io.Writer, u uint64) (err error)

func EncodeUint8 Uses

func EncodeUint8(w io.Writer, u uint8) (err error)

func EncodeUvarint Uses

func EncodeUvarint(w io.Writer, u uint64) (err error)

EncodeUvarint is used to encode golang's int, int32, int64 by default. unless specified differently by the `binary:"fixed32"`, `binary:"fixed64"`, or `binary:"zigzag32"` `binary:"zigzag64"` tags. It matches protobufs varint encoding.

func EncodeVarint Uses

func EncodeVarint(w io.Writer, i int64) (err error)

func MarshalBinaryBare Uses

func MarshalBinaryBare(o interface{}) ([]byte, error)

func MarshalBinaryLengthPrefixed Uses

func MarshalBinaryLengthPrefixed(o interface{}) ([]byte, error)

func MarshalBinaryLengthPrefixedWriter Uses

func MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)

func MarshalJSON Uses

func MarshalJSON(o interface{}) ([]byte, error)

func MarshalJSONIndent Uses

func MarshalJSONIndent(o interface{}, prefix, indent string) ([]byte, error)

func MustMarshalBinaryBare Uses

func MustMarshalBinaryBare(o interface{}) []byte

func MustMarshalBinaryLengthPrefixed Uses

func MustMarshalBinaryLengthPrefixed(o interface{}) []byte

func MustUnmarshalBinaryBare Uses

func MustUnmarshalBinaryBare(bz []byte, ptr interface{})

func MustUnmarshalBinaryLengthPrefixed Uses

func MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})

func NameToDisfix Uses

func NameToDisfix(name string) (db DisambBytes, pb PrefixBytes)

Return the DisambBytes and the PrefixBytes for a given name.

func UnmarshalBinaryBare Uses

func UnmarshalBinaryBare(bz []byte, ptr interface{}) error

func UnmarshalBinaryLengthPrefixed Uses

func UnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{}) error

func UnmarshalBinaryLengthPrefixedReader Uses

func UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{}, maxSize int64) (n int64, err error)

func UnmarshalJSON Uses

func UnmarshalJSON(bz []byte, ptr interface{}) error

func UvarintSize Uses

func UvarintSize(u uint64) int

func VarintSize Uses

func VarintSize(i int64) int

type Codec Uses

type Codec struct {
    // contains filtered or unexported fields
}

func NewCodec Uses

func NewCodec() *Codec

func (*Codec) MarshalBinaryBare Uses

func (cdc *Codec) MarshalBinaryBare(o interface{}) ([]byte, error)

MarshalBinaryBare encodes the object o according to the Amino spec. MarshalBinaryBare doesn't prefix the byte-length of the encoding, so the caller must handle framing.

func (*Codec) MarshalBinaryLengthPrefixed Uses

func (cdc *Codec) MarshalBinaryLengthPrefixed(o interface{}) ([]byte, error)

MarshalBinaryLengthPrefixed encodes the object o according to the Amino spec, but prefixed by a uvarint encoding of the object to encode. Use MarshalBinaryBare if you don't want byte-length prefixing.

For consistency, MarshalBinaryLengthPrefixed will first dereference pointers before encoding. MarshalBinaryLengthPrefixed will panic if o is a nil-pointer, or if o is invalid.

func (*Codec) MarshalBinaryLengthPrefixedWriter Uses

func (cdc *Codec) MarshalBinaryLengthPrefixedWriter(w io.Writer, o interface{}) (n int64, err error)

MarshalBinaryLengthPrefixedWriter writes the bytes as would be returned from MarshalBinaryLengthPrefixed to the writer w.

func (*Codec) MarshalJSON Uses

func (cdc *Codec) MarshalJSON(o interface{}) ([]byte, error)

func (*Codec) MarshalJSONIndent Uses

func (cdc *Codec) MarshalJSONIndent(o interface{}, prefix, indent string) ([]byte, error)

MarshalJSONIndent calls json.Indent on the output of cdc.MarshalJSON using the given prefix and indent string.

func (*Codec) MustMarshalBinaryBare Uses

func (cdc *Codec) MustMarshalBinaryBare(o interface{}) []byte

Panics if error.

func (*Codec) MustMarshalBinaryLengthPrefixed Uses

func (cdc *Codec) MustMarshalBinaryLengthPrefixed(o interface{}) []byte

Panics if error.

func (*Codec) MustMarshalJSON Uses

func (cdc *Codec) MustMarshalJSON(o interface{}) []byte

MustMarshalJSON panics if an error occurs. Besides tha behaves exactly like MarshalJSON.

func (*Codec) MustUnmarshalBinaryBare Uses

func (cdc *Codec) MustUnmarshalBinaryBare(bz []byte, ptr interface{})

Panics if error.

func (*Codec) MustUnmarshalBinaryLengthPrefixed Uses

func (cdc *Codec) MustUnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{})

Panics if error.

func (*Codec) MustUnmarshalJSON Uses

func (cdc *Codec) MustUnmarshalJSON(bz []byte, ptr interface{})

MustUnmarshalJSON panics if an error occurs. Besides tha behaves exactly like UnmarshalJSON.

func (*Codec) PrintTypes Uses

func (cdc *Codec) PrintTypes(out io.Writer) error

PrintTypes writes all registered types in a markdown-style table. The table's header is:

| Type | Name | Prefix | Notes |

Where Type is the golang type name and Name is the name the type was registered with.

func (*Codec) RegisterConcrete Uses

func (cdc *Codec) RegisterConcrete(o interface{}, name string, copts *ConcreteOptions)

This function should be used to register concrete types that will appear in interface fields/elements to be encoded/decoded by go-amino. Usage: `amino.RegisterConcrete(MyStruct1{}, "com.tendermint/MyStruct1", nil)`

func (*Codec) RegisterInterface Uses

func (cdc *Codec) RegisterInterface(ptr interface{}, iopts *InterfaceOptions)

This function should be used to register all interfaces that will be encoded/decoded by go-amino. Usage: `amino.RegisterInterface((*MyInterface1)(nil), nil)`

func (*Codec) Seal Uses

func (cdc *Codec) Seal() *Codec

func (*Codec) UnmarshalBinaryBare Uses

func (cdc *Codec) UnmarshalBinaryBare(bz []byte, ptr interface{}) error

UnmarshalBinaryBare will panic if ptr is a nil-pointer.

func (*Codec) UnmarshalBinaryLengthPrefixed Uses

func (cdc *Codec) UnmarshalBinaryLengthPrefixed(bz []byte, ptr interface{}) error

Like UnmarshalBinaryBare, but will first decode the byte-length prefix. UnmarshalBinaryLengthPrefixed will panic if ptr is a nil-pointer. Returns an error if not all of bz is consumed.

func (*Codec) UnmarshalBinaryLengthPrefixedReader Uses

func (cdc *Codec) UnmarshalBinaryLengthPrefixedReader(r io.Reader, ptr interface{},
    maxSize int64) (n int64, err error)

Like UnmarshalBinaryBare, but will first read the byte-length prefix. UnmarshalBinaryLengthPrefixedReader will panic if ptr is a nil-pointer. If maxSize is 0, there is no limit (not recommended).

func (*Codec) UnmarshalJSON Uses

func (cdc *Codec) UnmarshalJSON(bz []byte, ptr interface{}) error

type ConcreteInfo Uses

type ConcreteInfo struct {

    // These fields are only set when registered (as implementing an interface).
    Registered       bool // Registered with RegisterConcrete().
    PointerPreferred bool // Deserialize to pointer type if possible.
    // NilPreferred     bool        // Deserialize to nil for empty structs if PointerPreferred.
    Name            string      // Registered name.
    Disamb          DisambBytes // Disambiguation bytes derived from name.
    Prefix          PrefixBytes // Prefix bytes derived from name.
    ConcreteOptions             // Registration options.

    // These fields get set for all concrete types,
    // even those not manually registered (e.g. are never interface values).
    IsAminoMarshaler       bool         // Implements MarshalAmino() (<ReprObject>, error).
    AminoMarshalReprType   reflect.Type // <ReprType>
    IsAminoUnmarshaler     bool         // Implements UnmarshalAmino(<ReprObject>) (error).
    AminoUnmarshalReprType reflect.Type // <ReprType>
}

func (ConcreteInfo) GetDisfix Uses

func (cinfo ConcreteInfo) GetDisfix() DisfixBytes

type ConcreteOptions Uses

type ConcreteOptions struct {
}

type DisambBytes Uses

type DisambBytes [DisambBytesLen]byte

Prefix types

func (DisambBytes) Bytes Uses

func (db DisambBytes) Bytes() []byte

func (DisambBytes) EqualBytes Uses

func (db DisambBytes) EqualBytes(bz []byte) bool

type DisfixBytes Uses

type DisfixBytes [DisfixBytesLen]byte // Disamb+Prefix

Prefix types

func (DisfixBytes) Bytes Uses

func (df DisfixBytes) Bytes() []byte

func (DisfixBytes) EqualBytes Uses

func (df DisfixBytes) EqualBytes(bz []byte) bool

type FieldInfo Uses

type FieldInfo struct {
    Name         string        // Struct field name
    Type         reflect.Type  // Struct field type
    Index        int           // Struct field index
    ZeroValue    reflect.Value // Could be nil pointer unlike TypeInfo.ZeroValue.
    UnpackedList bool          // True iff this field should be encoded as an unpacked list.
    FieldOptions               // Encoding options
}

type FieldOptions Uses

type FieldOptions struct {
    JSONName      string // (JSON) field name
    JSONOmitEmpty bool   // (JSON) omitempty
    BinFixed64    bool   // (Binary) Encode as fixed64
    BinFixed32    bool   // (Binary) Encode as fixed32
    BinFieldNum   uint32 // (Binary) max 1<<29-1

    Unsafe        bool // e.g. if this field is a float.
    WriteEmpty    bool // write empty structs and lists (default false except for pointers)
    EmptyElements bool // Slice and Array elements are never nil, decode 0x00 as empty struct.
}

type InterfaceInfo Uses

type InterfaceInfo struct {
    Priority     []DisfixBytes               // Disfix priority.
    Implementers map[PrefixBytes][]*TypeInfo // Mutated over time.
    InterfaceOptions
}

type InterfaceOptions Uses

type InterfaceOptions struct {
    Priority           []string // Disamb priority.
    AlwaysDisambiguate bool     // If true, include disamb for all types.
}

type InvalidTimeErr Uses

type InvalidTimeErr string

func (InvalidTimeErr) Error Uses

func (e InvalidTimeErr) Error() string

type PrefixBytes Uses

type PrefixBytes [PrefixBytesLen]byte

Prefix types

func NewPrefixBytes Uses

func NewPrefixBytes(prefixBytes []byte) PrefixBytes

Copy into PrefixBytes

func (PrefixBytes) Bytes Uses

func (pb PrefixBytes) Bytes() []byte

func (PrefixBytes) EqualBytes Uses

func (pb PrefixBytes) EqualBytes(bz []byte) bool

type StructInfo Uses

type StructInfo struct {
    Fields []FieldInfo // If a struct.
}

type Typ3 Uses

type Typ3 uint8

func (Typ3) String Uses

func (typ Typ3) String() string

type TypeInfo Uses

type TypeInfo struct {
    Type      reflect.Type // Interface type.
    PtrToType reflect.Type
    ZeroValue reflect.Value
    ZeroProto interface{}
    InterfaceInfo
    ConcreteInfo
    StructInfo
}

func (TypeInfo) String Uses

func (ti TypeInfo) String() string

Directories

PathSynopsis
tests

Package amino imports 16 packages (graph) and is imported by 262 packages. Updated 2019-09-12. Refresh now. Tools for package owners.