fastproto

Protocol Buffers for Go with faster Marshal and Unmarshal methods

FastProto is a go support for adding extra Marshal and Unmarshal methods to standard generated protobuf file inspired by gogo/protobuf. For now, FastProto support proto 3 only.

Getting Started

Installation

Install the standard protocol buffer implementation from https://github.com/protocolbuffers/protobuf first.

Then install the protoc-gen-go-fast binary

go install github.com/billyplus/fastproto/cmd/protoc-gen-go-fast@latest

How to use

Generate standard .pb.go with protoc-gen-go first, then generate an extra _fast.pb.go file with Marshal and Unmarshal methods.

protoc --go_out=./ ./test/msg.proto
protoc --go-fast_out=./ ./test/msg.proto

Marshal message

    msg := &pb.SomeProtoMsg{}
    if err := fastproto.Marshal(msg); err!=nil {}

    // you can allocate []byte first
    data := make([]byte, msg.Size())
    if n, err := fastproto.MarshalTo(data, msg); err!=nil {}
    // result is data[:n]

Unmarshal message

    msg := &pb.SomeProtoMsg{}
    if err := fastproto.Unmarshal(data, &msg); err!=nil {}

GRPC

It works with grpc.

Option 1. Replace the default codec for proto

import "google.golang.org/grpc/encoding"

func main() {
    // replace the default codec.
	encoding.RegisterCodec(fastproto.ProtoCodec())
}

Option 2. Not recommended. Use grpc.ForceServerCodec option or grpc.CustomCodec option to create grpc server. Notice: This API is may be changed or removed in a later release.

    import "google.golang.org/grpc"
    s := grpc.NewServer(grpc.CustomCodec(fastproto.ProtoCodec()))

    // or

    s := grpc.NewServer(grpc.ForceServerCodec(fastproto.ProtoCodec()))

Options to control code generation

if you want to ignore marshal method

    import "github.com/billyplus/fastproto/options/options.proto"

    // if true, Marshal interface will not be generated for this file
    option (options.fastproto_no_marshaler) = true;

    message XXXX {
        // if true, Marshal interface will be generated for this message whatever fastproto_no_marshaler is.
        option (options.fastproto_msg_marshaler) = true;
        // if true, Marshal interface will not be generated for this message
        option (options.fastproto_msg_no_marshaler) = true;
    }

if you want to ignore unmarshal method

    import "github.com/billyplus/fastproto/options/options.proto"

    // if true, Unmarshal interface will not be generated for this file
    option (options.fastproto_no_unmarshaler) = true;

    message XXXX {
        // if true, Unmarshal interface will be generated for this message whatever fastproto_no_unmarshaler is.
        option (options.fastproto_msg_unmarshaler) = true;
        // if true, Unmarshal interface will not be generated for this message
        option (options.fastproto_msg_no_unmarshaler) = true;
    }

please check options/options.proto for other options.

Benchmarks

code

goos: linux
goarch: amd64
pkg: github.com/billyplus/fastproto/test
cpu: AMD Ryzen 9 5950X 16-Core Processor