Documentation ¶
Overview ¶
MsgPack library for Go.
Implements:
http://wiki.msgpack.org/display/MSGPACK/Format+specification
It provides features similar to encoding packages in the standard library (ie json, xml, gob, etc).
Supports:
- Standard Marshal/Unmarshal interface.
- Standard field renaming via tags
- Encoding from any value (struct, slice, map, primitives, pointers, interface{}, etc)
- Decoding into pointer to any non-nil value (struct, slice, map, int, float32, bool, string, etc)
- Decoding into a nil interface{}
- Handles time.Time transparently
- Provides a Server and Client Codec so msgpack can be used as communication protocol for net/rpc. Also includes an option for msgpack-rpc: http://wiki.msgpack.org/display/MSGPACK/RPC+specification
Usage
dec = msgpack.NewDecoder(r, nil) err = dec.Decode(&v) enc = msgpack.NewEncoder(w) err = enc.Encode(v) //methods below are convenience methods over functions above. data, err = msgpack.Marshal(v) err = msgpack.Unmarshal(data, &v, nil) //RPC Server conn, err := listener.Accept() rpcCodec := msgpack.NewRPCServerCodec(conn, nil) rpc.ServeCodec(rpcCodec) //RPC Communication (client side) conn, err = net.Dial("tcp", "localhost:5555") rpcCodec := msgpack.NewRPCClientCodec(conn, nil) client := rpc.NewClientWithCodec(rpcCodec)
RPC ¶
An RPC Client and Server Codec is implemented, so that msgpack can be used with the standard net/rpc package. It supports both a basic net/rpc serialization, and the custom format defined at http://wiki.msgpack.org/display/MSGPACK/RPC+specification
Index ¶
- Constants
- Variables
- func Marshal(v interface{}) (b []byte, err error)
- func NewCustomRPCClientCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ClientCodec
- func NewCustomRPCServerCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ServerCodec
- func NewRPCClientCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ClientCodec
- func NewRPCServerCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ServerCodec
- func Unmarshal(data []byte, v interface{}, dam DecoderContainerResolver) error
- type Client
- func (c *Client) Close()
- func (c *Client) Closed() *ClientClosedError
- func (c *Client) Send(method string, params []interface{}) (interface{}, interface{})
- func (c *Client) SendAsync(method string, params []interface{}, replyChan chan ClientResponse) (msgid uint32, err *ClientClosedError)
- func (c *Client) ShutdownGracefully(timeout time.Duration, done chan int)
- func (c *Client) String() string
- type ClientClosedError
- type ClientResponse
- type ContainerType
- type Decoder
- type DecoderContainerResolver
- type Encoder
- type SimpleDecoderContainerResolver
Constants ¶
const ( ContainerRawBytes = ContainerType('b') ContainerList = ContainerType('a') ContainerMap = ContainerType('m') )
const DefaultFlightLimitSize = 100
If not set in a constructor, this will limit the number of "in-flight" requests that will be processed before blocking For single connections, this should be set high to avoid blocking. When pooling clients, a lower limit may be desireable to limit the number of requests that will fail when an error occurs.
Variables ¶
var DefaultDecoderContainerResolver = SimpleDecoderContainerResolver{ MapType: nil, SliceType: nil, BytesStringLiteral: true, BytesStringSliceElement: true, BytesStringMapValue: true, }
Default DecoderContainerResolver used when a nil parameter is passed to NewDecoder(). Sample Usage:
opts := msgpack.DefaultDecoderContainerResolver // makes a copy opts.BytesStringLiteral = false // change some options err := msgpack.NewDecoder(r, &opts).Decode(&v)
Functions ¶
func Marshal ¶
Marshal is a convenience function which encodes v to a stream of bytes. It delegates to Encoder.Encode.
func NewCustomRPCClientCodec ¶
func NewCustomRPCClientCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ClientCodec
NewCustomRPCClientCodec uses msgpack serialization for rpc communication from client side, but uses a custom protocol defined at http://wiki.msgpack.org/display/MSGPACK/RPC+specification
func NewCustomRPCServerCodec ¶
func NewCustomRPCServerCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ServerCodec
NewCustomRPCServerCodec uses msgpack serialization for rpc communication from server side, but uses a custom protocol defined at http://wiki.msgpack.org/display/MSGPACK/RPC+specification
func NewRPCClientCodec ¶
func NewRPCClientCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ClientCodec
NewRPCClientCodec uses basic msgpack serialization for rpc communication from client side.
Sample Usage:
conn, err = net.Dial("tcp", "localhost:5555") codec, err := msgpack.NewRPCClientCodec(conn, nil) client := rpc.NewClientWithCodec(codec) ... (see rpc package for how to use an rpc client)
func NewRPCServerCodec ¶
func NewRPCServerCodec(conn io.ReadWriteCloser, opts DecoderContainerResolver) rpc.ServerCodec
NewRPCServerCodec uses basic msgpack serialization for rpc communication from the server side.
func Unmarshal ¶
func Unmarshal(data []byte, v interface{}, dam DecoderContainerResolver) error
Unmarshal is a convenience function which decodes a stream of bytes into v. It delegates to Decoder.Decode.
Types ¶
type Client ¶
type Client struct { ClientClosedNotification func(*Client) // contains filtered or unexported fields }
The Client class reimplements the Msgpack-RPC protocol with pipelining support which is not supported by the Go net/rpc impelementation
In other words, many requests can be sent and replies may be sent before getting a reply and replies may come back out of order
func NewClientWithOptions ¶
func NewClientWithOptions(conn net.Conn, dopts DecoderContainerResolver) *Client
func NewClientWithOptionsAndFlightLimit ¶
func NewClientWithOptionsAndFlightLimit(conn net.Conn, dopts DecoderContainerResolver, inFlight int) *Client
func (*Client) Closed ¶
func (c *Client) Closed() *ClientClosedError
Use this to check if a client has closed and get the underlying error
func (*Client) Send ¶
Send a request and wait for a response. Under the hood, this uses SendAsync so the client can still be used by other goroutines without blocking @return the result data or error if there is an error
if the error is of type ClientClosedError, then the client has had a fatal error and can no longer be used.
func (*Client) SendAsync ¶
func (c *Client) SendAsync(method string, params []interface{}, replyChan chan ClientResponse) (msgid uint32, err *ClientClosedError)
Send a request but don't wait for the response. This call may block if the in-flight limit is reached
@param replyChan this channel may be new for every request, or a shared channel may be used. Be sure to
use a buffered channel or immediately wait (or have a goroutine already waiting) for the response because the message will be thrown away if the reply cannot be sent immediately to the channel
@return message id of the request. it is useful for demultiplexing requests if using a single reply channel. err will
always be nil unless the Client has closed due to an error. If you get a ClientClosedError
func (*Client) ShutdownGracefully ¶
This will close the client but attempt to finish all in-flight requests first The client will closed forefully after the timeout expires
type ClientClosedError ¶
type ClientClosedError struct {
Cause error
}
func (*ClientClosedError) Error ¶
func (cce *ClientClosedError) Error() string
func (*ClientClosedError) String ¶
func (cce *ClientClosedError) String() string
type ClientResponse ¶
type ClientResponse struct { Msgid uint32 Error interface{} Result interface{} }
type ContainerType ¶
type ContainerType byte
type Decoder ¶
type Decoder struct {
// contains filtered or unexported fields
}
A Decoder reads and decodes an object from an input stream in the msgpack format.
func NewDecoder ¶
func NewDecoder(r io.Reader, dam DecoderContainerResolver) (d *Decoder)
NewDecoder returns a Decoder for decoding a stream of bytes into an object. If nil DecoderContainerResolver is passed, we use DefaultDecoderContainerResolver
func (*Decoder) Decode ¶
Decode decodes the stream from reader and stores the result in the value pointed to by v.
If v is a pointer to a non-nil value, we will decode the stream into that value (if the value type and the stream match. For example: integer in stream must go into int type (int8...int64), etc
If you do not know what type of stream it is, pass in a pointer to a nil interface. We will decode and store a value in that nil interface.
time.Time is handled transparently, by (en)decoding (to)from a []int64{Seconds since Epoch, Nanoseconds offset}.
Sample usages:
// Decoding into a non-nil typed value var f float32 err = msgpack.NewDecoder(r, nil).Decode(&f) // Decoding into nil interface var v interface{} dec := msgpack.NewDecoder(r, nil) err = dec.Decode(&v) // To configure default options, see DefaultDecoderContainerResolver usage. // or write your own DecoderContainerResolver
type DecoderContainerResolver ¶
type DecoderContainerResolver interface { // DecoderContainer is used to get a proper reflect.Value when decoding // a msgpack map, array or raw bytes (for which the stream defines the length and // corresponding containerType) into a nil interface{}. // // This may be within the context of a container: ([]interface{} or map[XXX]interface{}), // or just a top-level literal. // // The parentcontainer and parentkey define the context // - If decoding into a map, they will be the map and the key in the map (a reflect.Value) // - If decoding into a slice, they will be the slice and the index into the slice (an int) // - Else they will be Invalid/nil // // Custom code can use this callback to determine how specifically to decode something. // A simple implementation exists which just uses some options to do it // (see SimpleDecoderContainerResolver). DecoderContainer(parentcontainer reflect.Value, parentkey interface{}, length int, ct ContainerType) (val reflect.Value) }
DecoderContainerResolver has the DecoderContainer method for getting a usable reflect.Value when decoding a container (map, array, raw bytes) from a stream into a nil interface{}.
type Encoder ¶
type Encoder struct {
// contains filtered or unexported fields
}
An Encoder writes an object to an output stream in the msgpack format.
func NewEncoder ¶
NewDecoder returns an Encoder for encoding an object.
func (*Encoder) Encode ¶
Encode writes an object into a stream in the MsgPack format.
time.Time is handled transparently, by (en)decoding (to)from a []int64{Seconds since Epoch, Nanoseconds offset}.
Struct values encode as maps. Each exported struct field is encoded unless:
- the field's tag is "-", or
- the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero.
Anonymous fields are encoded inline if no msgpack tag is present. Else they are encoded as regular fields.
The object's default key string is the struct field name but can be specified in the struct field's tag value. The "msgpack" key in struct field's tag value is the key name, followed by an optional comma and options.
To set an option on all fields (e.g. omitempty on all fields), you can create a field called _struct, and set flags on it.
Examples:
type MyStruct struct { _struct bool `msgpack:",omitempty"` //set omitempty for every field Field1 string `msgpack:"-"` //skip this field Field2 int `msgpack:"myName"` //Use key "myName" in encode stream Field3 int32 `msgpack:",omitempty"` //use key "Field3". Omit if empty. Field4 bool `msgpack:"f4,omitempty"` //use key "f4". Omit if empty. ... }
type SimpleDecoderContainerResolver ¶
type SimpleDecoderContainerResolver struct { // If decoding into a nil interface{} and we detect a map in the stream, // we create a map of the type specified. It defaults to creating a // map[interface{}]interface{} if not specified. MapType reflect.Type // If decoding into a nil interface{} and we detect a slice/array in the stream, // we create a slice of the type specified. It defaults to creating a // []interface{} if not specified. SliceType reflect.Type // convert to a string if raw bytes are detected while decoding // into a interface{}, BytesStringLiteral bool // convert to a string if raw bytes are detected while decoding // into a []interface{}, BytesStringSliceElement bool // convert to a string if raw bytes are detected while decoding // into a value in a map[XXX]interface{}, BytesStringMapValue bool }
SimpleDecoderContainerResolver is a simple DecoderContainerResolver which uses some simple options to determine how to decode into a nil interface{}. Most applications will work fine with just this.
func (SimpleDecoderContainerResolver) DecoderContainer ¶
func (d SimpleDecoderContainerResolver) DecoderContainer( parentcontainer reflect.Value, parentkey interface{}, length int, ct ContainerType) (rvn reflect.Value)
DecoderContainer supports common cases for decoding into a nil interface{} depending on the context.
When decoding into a nil interface{}, the following rules apply as we have to make assumptions about the specific types you want.
- Maps are decoded as map[interface{}]interface{} unless you provide a default map type when creating your decoder. option: MapType
- Lists are always decoded as []interface{} unless you provide a default slice type when creating your decoder. option: SliceType
- raw bytes are decoded into []byte or string depending on setting of: option: BytesStringMapValue (if within a map value, use this setting) option: BytesStringSliceElement (else if within a slice, use this setting) option: BytesStringLiteral (else use this setting)