molecule: github.com/richardartoul/molecule Index | Examples | Files | Directories

package molecule

import "github.com/richardartoul/molecule"

Package molecule is a Go library for parsing protobufs in an efficient and zero-allocation manner.

Example demonstrates how the molecule library can be used to parse a protobuf message.

Code:

// Proto definitions:
//
//   message Test {
//     string string_field = 1;
//     int64 int64_field = 2;
//     repeated int64 repeated_int64_field = 3;
//   }

m := &simple.Test{
    StringField: "hello world!",
    Int64Field:  10,
}
marshaled, err := proto.Marshal(m)
if err != nil {
    panic(err)
}

var (
    buffer   = codec.NewBuffer(marshaled)
    strVal   Value
    int64Val Value
)
err = MessageEach(buffer, func(fieldNum int32, value Value) (bool, error) {
    if fieldNum == 1 {
        strVal = value
    }
    if fieldNum == 2 {
        int64Val = value
    }

    // Continue scanning.
    return true, nil
})
if err != nil {
    panic(err)
}

str, err := strVal.AsStringUnsafe()
if err != nil {
    panic(err)
}
int64V, err := int64Val.AsInt64()
if err != nil {
    panic(err)
}

fmt.Println("StringField:", str)
fmt.Println("Int64Field:", int64V)

Output:

StringField: hello world!
Int64Field: 10

Example_nested demonstrates how to use the MessageEach function to decode a nested message.

Code:

// Proto definitions:
//
//   message Test {
//       string string_field = 1;
//       int64 int64_field = 2;
//       repeated int64 repeated_int64_field = 3;
//   }
//
//   message Nested {
//       Test nested_message = 1;
//   }

var (
    test   = &simple.Test{StringField: "Hello world!"}
    nested = &simple.Nested{NestedMessage: test}
)
marshaled, err := proto.Marshal(nested)
if err != nil {
    panic(err)
}

var (
    buffer = codec.NewBuffer(marshaled)
    strVal Value
)
err = MessageEach(buffer, func(fieldNum int32, value Value) (bool, error) {
    if fieldNum == 1 {
        packedArr, err := value.AsBytesUnsafe()
        if err != nil {
            return false, err
        }

        buffer := codec.NewBuffer(packedArr)
        err = MessageEach(buffer, func(fieldNum int32, value Value) (bool, error) {
            if fieldNum == 1 {
                strVal = value
            }
            // Found it, stop scanning.
            return false, nil
        })
        if err != nil {
            return false, err
        }

        // Found it, stop scanning.
        return false, nil
    }
    // Continue scanning.
    return true, nil
})
if err != nil {
    panic(err)
}

str, err := strVal.AsStringUnsafe()
if err != nil {
    panic(err)
}

fmt.Println("NestedMessage.StringField:", str)

Output:

NestedMessage.StringField: Hello world!

Example_repeated demonstrates how to use the PackedRepeatedEach function to decode a repeated field encoded in the packed (proto 3) format.

Code:

// Proto definitions:
//
//   message Test {
//     string string_field = 1;
//     int64 int64_field = 2;
//     repeated int64 repeated_int64_field = 3;
//   }

int64s := []int64{1, 2, 3, 4, 5, 6, 7}
m := &simple.Test{RepeatedInt64Field: int64s}
marshaled, err := proto.Marshal(m)
if err != nil {
    panic(err)
}

var (
    buffer          = codec.NewBuffer(marshaled)
    unmarshaledInts = []int64{}
)
err = MessageEach(buffer, func(fieldNum int32, value Value) (bool, error) {
    if fieldNum == 3 {
        packedArr, err := value.AsBytesUnsafe()
        if err != nil {
            return false, err
        }

        buffer := codec.NewBuffer(packedArr)
        if err := PackedRepeatedEach(buffer, codec.FieldType_INT64, func(v Value) (bool, error) {
            vInt64, err := v.AsInt64()
            if err != nil {
                return false, err
            }
            unmarshaledInts = append(unmarshaledInts, vInt64)
            return true, nil
        }); err != nil {
            return false, err
        }

        // Found it, stop scanning.
        return false, nil
    }
    // Continue scanning.
    return true, nil
})
if err != nil {
    panic(err)
}

fmt.Println("Int64s:", unmarshaledInts)

Output:

Int64s: [1 2 3 4 5 6 7]

Index

Examples

Package Files

molecule.go value.go

func MessageEach Uses

func MessageEach(buffer *codec.Buffer, fn MessageEachFn) error

MessageEach iterates over each top-level field in the message stored in buffer and calls fn on each one.

ExampleMessageEach_SelectAField desmonates how the MessageEach function can be used to select an individual field.

Code:

// Proto definitions:
//
//   message Test {
//     string string_field = 1;
//     int64 int64_field = 2;
//     repeated int64 repeated_int64_field = 3;
//   }

m := &simple.Test{StringField: "hello world!"}
marshaled, err := proto.Marshal(m)
if err != nil {
    panic(err)
}

var (
    buffer = codec.NewBuffer(marshaled)
    strVal Value
)
err = MessageEach(buffer, func(fieldNum int32, value Value) (bool, error) {
    if fieldNum == 1 {
        strVal = value
        // Found it, stop scanning.
        return false, nil
    }
    // Continue scanning.
    return true, nil
})
if err != nil {
    panic(err)
}

str, err := strVal.AsStringUnsafe()
if err != nil {
    panic(err)
}

fmt.Println("StringField:", str)

Output:

StringField: hello world!

func PackedRepeatedEach Uses

func PackedRepeatedEach(buffer *codec.Buffer, fieldType codec.FieldType, fn PackedRepeatedEachFn) error

PackedRepeatedEach iterates over each value in the packed repeated field stored in buffer and calls fn on each one.

The fieldType argument should match the type of the value stored in the repeated field.

PackedRepeatedEach only supports repeated fields encoded using packed encoding.

ExamplePackedRepeatedEach demonstrates how to use the PackedRepeatedEach function to decode a repeated field encoded in the packed (proto 3) format.

Code:

// Proto definitions:
//
//   message Test {
//     string string_field = 1;
//     int64 int64_field = 2;
//     repeated int64 repeated_int64_field = 3;
//   }

int64s := []int64{1, 2, 3, 4, 5, 6, 7}
m := &simple.Test{RepeatedInt64Field: int64s}
marshaled, err := proto.Marshal(m)
if err != nil {
    panic(err)
}

var (
    buffer          = codec.NewBuffer(marshaled)
    unmarshaledInts = []int64{}
)
err = MessageEach(buffer, func(fieldNum int32, value Value) (bool, error) {
    if fieldNum == 3 {
        packedArr, err := value.AsBytesUnsafe()
        if err != nil {
            panic(err)
        }

        buffer := codec.NewBuffer(packedArr)
        err = PackedRepeatedEach(buffer, codec.FieldType_INT64, func(v Value) (bool, error) {
            vInt64, err := v.AsInt64()
            if err != nil {
                return false, err
            }
            unmarshaledInts = append(unmarshaledInts, vInt64)
            return true, nil
        })
        if err != nil {
            return false, err
        }

        // Found it, stop scanning.
        return false, nil
    }
    // Continue scanning.
    return true, nil
})
if err != nil {
    panic(err)
}

fmt.Println("Int64s:", unmarshaledInts)

Output:

Int64s: [1 2 3 4 5 6 7]

type MessageEachFn Uses

type MessageEachFn func(fieldNum int32, value Value) (bool, error)

MessageEachFn is a function that will be called for each top-level field in a message passed to MessageEach.

type PackedRepeatedEachFn Uses

type PackedRepeatedEachFn func(value Value) (bool, error)

PackedRepeatedEachFn is a function that is called for each value in a repeated field.

type Value Uses

type Value struct {
    // WireType is the protobuf wire type that was used to encode the field.
    WireType codec.WireType
    // Number will contain the value for any fields encoded with the
    // following wire types:
    //
    // 1. varint
    // 2. Fixed32
    // 3. Fixed64
    Number uint64
    // Bytes will contain the value for any fields encoded with the
    // following wire types:
    //
    // 1. bytes
    //
    // Bytes is an unsafe view over the bytes in the buffer. To obtain a "safe" copy
    // call value.AsSafeBytes() or copy Bytes directly.
    Bytes []byte
}

Value represents a protobuf value. It contains the original wiretype that the value was encoded with as well as a variety of helper methods for interpreting the raw value based on the field's actual type.

func (*Value) AsBool Uses

func (v *Value) AsBool() (bool, error)

AsBool interprets the value as a bool.

func (*Value) AsBytesSafe Uses

func (v *Value) AsBytesSafe() ([]byte, error)

AsBytesSafe interprets the value as a byte slice by allocating a safe copy of the underlying data.

func (*Value) AsBytesUnsafe Uses

func (v *Value) AsBytesUnsafe() ([]byte, error)

AsBytesUnsafe interprets the value as a byte slice. The returned []byte is an unsafe view over the underlying bytes. Use AsBytesSafe() to obtain a "safe" [] that is a copy of the underlying data.

func (*Value) AsDouble Uses

func (v *Value) AsDouble() (float64, error)

AsDouble interprets the value as a double.

func (*Value) AsFixed32 Uses

func (v *Value) AsFixed32() (uint32, error)

AsFixed32 interprets the value as a fixed32.

func (*Value) AsFixed64 Uses

func (v *Value) AsFixed64() (uint64, error)

AsFixed64 interprets the value as a fixed64.

func (*Value) AsFloat Uses

func (v *Value) AsFloat() (float32, error)

AsFloat interprets the value as a float.

func (*Value) AsInt32 Uses

func (v *Value) AsInt32() (int32, error)

AsInt32 interprets the value as an int32.

func (*Value) AsInt64 Uses

func (v *Value) AsInt64() (int64, error)

AsInt64 interprets the value as an int64.

func (*Value) AsSFixed32 Uses

func (v *Value) AsSFixed32() (int32, error)

AsSFixed32 interprets the value as a SFixed32.

func (*Value) AsSFixed64 Uses

func (v *Value) AsSFixed64() (int64, error)

AsSFixed64 interprets the value as a SFixed64.

func (*Value) AsSint32 Uses

func (v *Value) AsSint32() (int32, error)

AsSint32 interprets the value as a sint32.

func (*Value) AsSint64 Uses

func (v *Value) AsSint64() (int64, error)

AsSint64 interprets the value as a sint64.

func (*Value) AsStringSafe Uses

func (v *Value) AsStringSafe() (string, error)

AsStringSafe interprets the value as a string by allocating a safe copy of the underlying data.

func (*Value) AsStringUnsafe Uses

func (v *Value) AsStringUnsafe() (string, error)

AsStringUnsafe interprets the value as a string. The returned string is an unsafe view over the underlying bytes. Use AsStringSafe() to obtain a "safe" string that is a copy of the underlying data.

func (*Value) AsUint32 Uses

func (v *Value) AsUint32() (uint32, error)

AsUint32 interprets the value as a uint32.

func (*Value) AsUint64 Uses

func (v *Value) AsUint64() (uint64, error)

AsUint64 interprets the value as a uint64.

Directories

PathSynopsis
src/codecPackage codec contains all the logic required for interacting with the protobuf raw encoding.
src/protoPackage simple is a generated protocol buffer package.

Package molecule imports 6 packages (graph). Updated 2020-05-17. Refresh now. Tools for package owners.