protobuf3

package
v1.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 1, 2024 License: BSD-3-Clause Imports: 14 Imported by: 0

Documentation

Overview

Package protobuf3 encodes (marshals) Go structs which contain fields tagged with 'protobuf' encoding information into protobuf version 3.

It can operate on the output generated by the protobuf compiler. However it exists to operate on hand crafted types which are easier/more efficient for the rest of the code to handle than the struct the protobuf compiler emits would be.

Index

Constants

View Source 
const (
	WireVarint     = WireType(0)
	WireFixed64    = WireType(1)
	WireBytes      = WireType(2)
	WireStartGroup = WireType(3) // legacy from protobuf v2. Groups are not used in protobuf v3
	WireEndGroup   = WireType(4) // legacy...
	WireFixed32    = WireType(5)
)

Constants that identify the encoding of a value on the wire.

Variables

View Source 
var (

	// ErrNil is the error returned if Marshal is called with nil.
	ErrNil = errors.New("protobuf3: [Un]Marshal called with nil")

	ErrNotPointerToStruct = errors.New("protobuf3: Unmarshal called with argument which is not a pointer to a struct")
)
View Source 
var ErrNotFound = errors.New("ID not found in protobuf buffer")

error returned by (*Buffer).Find when the id is not present in the buffer

View Source 
var MakeFieldName func(f string, t reflect.Type) string = MakeLowercaseFieldName

MakeFieldName is a pointer to a function which returns what should be the name of field f in the protobuf definition of type t. You can replace this with your own function before calling AsProtobuf[Full]() to control the field names yourself.

View Source 
var MakePackageName func(pkgpath string) string = MakeSamePackageName

MakePackageName is a pointer to a function which returns what should be the name of the protobuf package given the go package path. By default it simply returns the last component of the pkgpath.

View Source 
var MakeTypeName func(t reflect.Type, f string) string = MakeUppercaseTypeName

MakeTypeName is a pointer to a function which returns what should be the name of the protobuf message of type t, which is the type of a field named f.

View Source 
var XXXHack = false

XXXHack enables a backwards compatibility hack to match the canonical golang.go/protobuf error behavior for fields whose names start with XXX_ This isn't needed unless you are dealing with old protobuf v2 generated types like some unit tests do

Functions

func AsProtobuf 

func AsProtobuf(t reflect.Type) (string, error)

returns the type expressed in protobuf v3 format, suitable for feeding back into the protobuf compiler.

func AsProtobufFull 

func AsProtobufFull(t reflect.Type, more ...reflect.Type) (string, error)

returns the type expressed in protobuf v3 format, including all dependent types and imports

func AsProtobufFull2 added in v1.5.0 

func AsProtobufFull2(t reflect.Type, extra_package_headers []string, more ...reflect.Type) (string, error)

returns the type expressed in protobuf v3 format, including all dependent types and imports extra_headers allow the caller to specify headers they want inserted after the `package` line.

func DebugPrint 

func DebugPrint(b []byte) string

DebugPrint dumps the encoded data in b in a debugging format with a header including the string s. Used in testing but made available for general debugging.

func DecodeVarint 

func DecodeVarint(buf []byte) (x uint64, n int)

DecodeVarint reads a varint-encoded integer from the slice. It returns the integer and the number of bytes consumed, or zero if there is not enough. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func MakeLowercaseFieldName 

func MakeLowercaseFieldName(f string, t reflect.Type) string

MakeLowercaseFieldName returns a reasonable lowercase field name

func MakeSamePackageName 

func MakeSamePackageName(pkgpath string) string

given the full path of the package of the 1st type passed to AsProtobufFull(), return the last component of the package path to be used as the package name.

func MakeUppercaseTypeName 

func MakeUppercaseTypeName(t reflect.Type, f string) string

MakeUppercaseTypeName makes an uppercase message type name for type t, which is the type of a field named f. Since the field is visible to us it is public, and thus it is uppercase. And since the type is similarly visible it is almost certainly uppercased too. So there isn't much to do except pick whichever is appropriate.

func Marshal 

func Marshal(pb Message) ([]byte, error)

Marshal takes the protocol buffer and encodes it into the wire format, returning the data.

func SizeVarint 

func SizeVarint(x uint64) (n int)

SizeVarint returns the varint encoding size of an integer.

func Unmarshal 

func Unmarshal(bytes []byte, pb Message) error

Unmarshal parses the protocol buffer representation in buf and writes the decoded result to pb. If the struct underlying pb does not match the data in buf, the results can be unpredictable.

Unmarshal merges into existing data in pb. If that's not what you wanted then you ought to zero pb before calling Unmarshal. NOTE WELL this differs from the behavior of the golang/proto.Unmarshal(), but matches the standard go encoding/json.Unmarshal() Since we're used to json, and since having the caller do the zeroing is more efficient (both because they know the type (making it more efficient for the CPU), and it avoids forcing everyone to define a Reset() method for the Message interface (making it more efficient for the developer, me!)), our Unmarshal() matches the behavior of encoding/json.Unmarshal()

Types

type AsProtobuf3er 

type AsProtobuf3er interface {
	AsProtobuf3() (name string, definition string, imports []string)
}

AsProtobuf3er is the interface which returns the protobuf v3 type equivalent to what the MarshalProtobuf3() method encodes. This is optional, but useful when using AsProtobufFull() against types implementing Marshaler. `definition` can be "" if the datatype doesn't need a custom definition. `imports` is the list of files to import when using this type. The order of imports in the slice is not important, and is not respected.

type AsV1Protobuf3er added in v1.8.0 

type AsV1Protobuf3er interface {
	AsProtobuf3() (name string, definition string)
}

legacy AsProtobuf3() which didn't support imports

type Buffer 

type Buffer struct {
	Immutable bool // true if we the caller promises the contents of buf[] are immutable, and thus we can retain references to it for types which decode into []byte
	// contains filtered or unexported fields
}

A Buffer is a buffer manager for marshaling and unmarshaling protocol buffers. It may be reused between invocations to reduce memory usage. It is not necessary to use a Buffer; the global functions Marshal and Unmarshal create a temporary Buffer and are fine for most applications. However if you are decoding objects with large []bytes, creating your own Buffer and setting Immutable=true will result in the decode []bytes references directly into the []byte passed to NewBuffer() rather than being expensive copies.

func NewBuffer 

func NewBuffer(e []byte) *Buffer

NewBuffer allocates a new Buffer and initializes its internal data to the contents of the argument slice.

func (*Buffer) Bytes 

func (p *Buffer) Bytes() []byte

Bytes returns the contents of the Buffer.

func (*Buffer) DecodeFixed32 

func (p *Buffer) DecodeFixed32() (uint64, error)

DecodeFixed32 reads a 32-bit integer from the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) DecodeFixed64 

func (p *Buffer) DecodeFixed64() (uint64, error)

DecodeFixed64 reads a 64-bit integer from the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) DecodeRawBytes 

func (p *Buffer) DecodeRawBytes() ([]byte, error)

DecodeRawBytes reads a count-delimited byte buffer from the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages. The returned slice points to shared memory. Treat as read-only.

func (*Buffer) DecodeStringBytes 

func (p *Buffer) DecodeStringBytes() (string, error)

DecodeStringBytes reads an encoded string from the Buffer. This is the format used for the proto3 string type.

func (*Buffer) DecodeTimestamp added in v1.7.4 

func (o *Buffer) DecodeTimestamp() (time.Time, error)

DecodeTimstamp decodes a google.protobuf.Timestamp as a time.Time

func (*Buffer) DecodeVarint 

func (p *Buffer) DecodeVarint() (x uint64, err error)

DecodeVarint reads a varint-encoded integer from the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types, as well as the tags.

func (*Buffer) DecodeZigzag32 

func (p *Buffer) DecodeZigzag32() (x uint64, err error)

DecodeZigzag32 reads a zigzag-encoded 32-bit integer from the Buffer. This is the format used for the sint32 protocol buffer type. Since I might cast the result to 'int', I want this to return a signed 64-bit value, rather than a signed 32-bit value embedded in an unsigned 64-bit value. Hence the cast to int32 before extending to uint64 which does not appear in the proto package.

func (*Buffer) DecodeZigzag64 

func (p *Buffer) DecodeZigzag64() (x uint64, err error)

DecodeZigzag64 reads a zigzag-encoded 64-bit integer from the Buffer. This is the format used for the sint64 protocol buffer type.

func (*Buffer) EOF 

func (p *Buffer) EOF() bool

returns true if we've decoded to the end of the buffer

func (*Buffer) EncodeBytes added in v1.0.2 

func (p *Buffer) EncodeBytes(tag uint32, b []byte)

EncodeBytes writes a bytes tag and count-delimited byte slice to the Buffer. This is equivalent to encoding a 'b []byte `protobuf:"bytes,tag"` field.

func (*Buffer) EncodeFixed32 

func (p *Buffer) EncodeFixed32(x uint64)

EncodeFixed32 writes a 32-bit integer to the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) EncodeFixed64 

func (p *Buffer) EncodeFixed64(x uint64)

EncodeFixed64 writes a 64-bit integer to the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) EncodeRawBytes 

func (p *Buffer) EncodeRawBytes(b []byte)

EncodeRawBytes writes a count-delimited byte buffer to the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) EncodeStringBytes 

func (p *Buffer) EncodeStringBytes(s string)

EncodeStringBytes writes an encoded string to the Buffer. This is the format used for the proto2 string type.

func (*Buffer) EncodeTimestamp added in v1.7.4 

func (o *Buffer) EncodeTimestamp(ts time.Time)

EncodeTimestamp marshals a a time.Time as a google.protobuf.Timestamp, which is a pair of varints (secs,nanos) tagged 1 and 2

func (*Buffer) EncodeVarint 

func (p *Buffer) EncodeVarint(x uint64)

EncodeVarint writes a varint-encoded integer to the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) EncodeZigzag32 

func (p *Buffer) EncodeZigzag32(x uint64)

EncodeZigzag32 writes a zigzag-encoded 32-bit integer to the Buffer. This is the format used for the sint32 protocol buffer type.

func (*Buffer) EncodeZigzag64 

func (p *Buffer) EncodeZigzag64(x uint64)

EncodeZigzag64 writes a zigzag-encoded 64-bit integer to the Buffer. This is the format used for the sint64 protocol buffer type.

func (*Buffer) Find 

func (p *Buffer) Find(id uint, sorted bool) (position int, full []byte, val []byte, wt WireType, err error)

Find scans forward for next item which has id 'id'. Both the entire item and just the value bytes are returned. The entire item is useful because it is itself a valid protobuf message. If no match is found, ErrNotFound is returned. If sorted is true then this function assumes the message's fields are sorted by id, and encountering any id > 'id' short circuits the search

func (*Buffer) FindBytes added in v1.0.2 

func (p *Buffer) FindBytes(id uint, sorted bool) (position int, full []byte, val []byte, err error)

FindBytes is similar to Find but only matches and returns ids with wiretype "bytes". If the id is present but hasn't got the wiretype "bytes" then ErrNotFound is returned. It exists because calling Find() and checking for WireBytes is a common pattern in calling code.

func (*Buffer) Marshal 

func (o *Buffer) Marshal(pb Message) error

Marshal takes the protocol buffer and encodes it into the wire format, writing the result to the Buffer.

func (*Buffer) Next added in v1.0.2 

func (p *Buffer) Next() (id int, full []byte, val []byte, wt WireType, err error)

Next returns the next item in the message. Both the entire item and just the value bytes are returned. The entire item is useful because it is itself a valid protobuf message. If the end of the buffer is reached 0,nil,nil,0,nil is returned.

func (*Buffer) Reset 

func (p *Buffer) Reset()

Reset resets the Buffer, ready for marshaling a new protocol buffer.

func (*Buffer) Rewind 

func (p *Buffer) Rewind()

Rewind resets the read point to the start of the buffer.

func (*Buffer) SkipFixed 

func (p *Buffer) SkipFixed(n uint64) error

SkipFixed skips over n bytes. Useful for skipping over Fixed32 and Fixed64 with proper arguments, but also used to skip over arbitrary lengths.

func (*Buffer) SkipRawBytes 

func (p *Buffer) SkipRawBytes() error

SkipRawBytes skips over a count-delimited byte buffer from the Buffer. Functionally it is identical to calling DecodeRawBytes() and ignoring the value returned.

func (*Buffer) SkipVarint 

func (p *Buffer) SkipVarint() error

SkipVarint skips over a varint-encoded integer from the Buffer. Functionally it is similar to calling DecodeVarint and ignoring the value returned, except that it doesn't worry about 64-bit overflow of the varint value, and it runs much faster than DecodeVarint.

func (*Buffer) Unmarshal 

func (p *Buffer) Unmarshal(pb Message) error

Unmarshal parses the protocol buffer representation in the Buffer and places the decoded result in pb. If the struct underlying pb does not match the data in the buffer, the results can be unpredictable.

type IntEncoder 

type IntEncoder int

IntEncoder enumerates the different ways of encoding integers in Protobuf v3 

const (
	UnknownEncoder IntEncoder = iota // make the zero-value be different from any valid value so I can tell it is not set
	VarintEncoder
	Fixed32Encoder
	Fixed64Encoder
	Zigzag32Encoder
	Zigzag64Encoder
)

type Marshaler 

type Marshaler interface {
	MarshalProtobuf3() ([]byte, error)
	UnmarshalProtobuf3([]byte) error
}

Marshaler is the interface representing objects that can marshal and unmarshal themselves. (note this is a single interface because dealing with types which only implement half the operations creates too many edge cases, and so far I haven't had any cases where I didn't have both a custom marshal and a custom unmarshal function)

type Message 

type Message interface {
}

Message is implemented by generated protocol buffer messages.

type Properties 

type Properties struct {
	Name string // name of the field, for error messages
	Wire string

	Tag      uint32
	WireType WireType
	// contains filtered or unexported fields
}

Properties represents the protocol-specific behavior of a single struct field.

func (*Properties) Parse 

func (p *Properties) Parse(s string) (IntEncoder, bool, error)

Parse populates p by parsing a string in the protobuf struct field tag style.

func (*Properties) String 

func (p *Properties) String() string

String formats the properties in the protobuf struct field tag style.

func (*Properties) Subtype 

func (p *Properties) Subtype() reflect.Type

returns the inner type, or nil

type Reserved added in v1.7.0 

type Reserved [0]byte

Reserved is a special type used to indicate reserved protobuf IDs. Instead of the usual protobuf tag, the tag consists of a comma separated list of reserved protobuf IDs. Using these IDs elsewhere causes an error, and they are listed in the `reserved` section by asProtobuf. At the moment we only support reserveing IDs. If you need to reserve field names then you'll have to implement it.

type StructProperties 

type StructProperties struct {
	// contains filtered or unexported fields
}

StructProperties represents properties for all the fields of a struct.

func GetProperties 

func GetProperties(t reflect.Type) (*StructProperties, error)

GetProperties returns the list of properties for the type represented by t. t must represent a generated struct type of a protocol message.

func (*StructProperties) Len 

func (sp *StructProperties) Len() int

Implement the sorting interface so we can sort the fields in tag order, as recommended by the spec. See encode.go, (*Buffer).enc_struct.

func (*StructProperties) Less 

func (sp *StructProperties) Less(i, j int) bool

func (*StructProperties) Swap 

func (sp *StructProperties) Swap(i, j int)

type Types 

type Types []reflect.Type

func (Types) Len 

func (ts Types) Len() int

func (Types) Less 

func (ts Types) Less(i, j int) bool

func (Types) Swap 

func (ts Types) Swap(i, j int)

type WireType 

type WireType byte

func (WireType) String 

func (wt WireType) String() string

Source Files

View all Source files

Directories

Path Synopsis
internal
unit_tests/any
Package any is a generated protocol buffer package.
 Package any is a generated protocol buffer package.
unit_tests/duration
Package duration is a generated protocol buffer package.
 Package duration is a generated protocol buffer package.
unit_tests/proto
Package proto converts data structures to and from the wire format of protocol buffers.
 Package proto converts data structures to and from the wire format of protocol buffers.
unit_tests/proto3_proto
Package proto3_proto is a generated protocol buffer package.
 Package proto3_proto is a generated protocol buffer package.
unit_tests/timestamp
Package timestamp is a generated protocol buffer package.
 Package timestamp is a generated protocol buffer package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.