cbor

package
v1.26.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 17, 2021 License: MIT Imports: 11 Imported by: 0

README

Reference:

CBOR Encoding is described in RFC7049

Comparison of JSON vs CBOR

Two main areas of reduction are:

  1. CPU usage to write a log msg
  2. Size (in bytes) of log messages.

CPU Usage savings are below:

name                                    JSON time/op    CBOR time/op   delta
Info-32                                   15.3ns ± 1%    11.7ns ± 3%  -23.78%  (p=0.000 n=9+10)      
ContextFields-32                          16.2ns ± 2%    12.3ns ± 3%  -23.97%  (p=0.000 n=9+9)       
ContextAppend-32                          6.70ns ± 0%    6.20ns ± 0%   -7.44%  (p=0.000 n=9+9)       
LogFields-32                              66.4ns ± 0%    24.6ns ± 2%  -62.89%  (p=0.000 n=10+9)      
LogArrayObject-32                          911ns ±11%     768ns ± 6%  -15.64%  (p=0.000 n=10+10)     
LogFieldType/Floats-32                    70.3ns ± 2%    29.5ns ± 1%  -57.98%  (p=0.000 n=10+10)     
LogFieldType/Err-32                       14.0ns ± 3%    12.1ns ± 8%  -13.20%  (p=0.000 n=8+10)      
LogFieldType/Dur-32                       17.2ns ± 2%    13.1ns ± 1%  -24.27%  (p=0.000 n=10+9)      
LogFieldType/Object-32                    54.3ns ±11%    52.3ns ± 7%     ~     (p=0.239 n=10+10)     
LogFieldType/Ints-32                      20.3ns ± 2%    15.1ns ± 2%  -25.50%  (p=0.000 n=9+10)      
LogFieldType/Interfaces-32                 642ns ±11%     621ns ± 9%     ~     (p=0.118 n=10+10)     
LogFieldType/Interface(Objects)-32         635ns ±13%     632ns ± 9%     ~     (p=0.592 n=10+10)     
LogFieldType/Times-32                      294ns ± 0%      27ns ± 1%  -90.71%  (p=0.000 n=10+9)      
LogFieldType/Durs-32                       121ns ± 0%      33ns ± 2%  -72.44%  (p=0.000 n=9+9)       
LogFieldType/Interface(Object)-32         56.6ns ± 8%    52.3ns ± 8%   -7.54%  (p=0.007 n=10+10)     
LogFieldType/Errs-32                      17.8ns ± 3%    16.1ns ± 2%   -9.71%  (p=0.000 n=10+9)      
LogFieldType/Time-32                      40.5ns ± 1%    12.7ns ± 6%  -68.66%  (p=0.000 n=8+9)       
LogFieldType/Bool-32                      12.0ns ± 5%    10.2ns ± 2%  -15.18%  (p=0.000 n=10+8)      
LogFieldType/Bools-32                     17.2ns ± 2%    12.6ns ± 4%  -26.63%  (p=0.000 n=10+10)     
LogFieldType/Int-32                       12.3ns ± 2%    11.2ns ± 4%   -9.27%  (p=0.000 n=9+10)      
LogFieldType/Float-32                     16.7ns ± 1%    12.6ns ± 2%  -24.42%  (p=0.000 n=7+9)       
LogFieldType/Str-32                       12.7ns ± 7%    11.3ns ± 7%  -10.88%  (p=0.000 n=10+9)      
LogFieldType/Strs-32                      20.3ns ± 3%    18.2ns ± 3%  -10.25%  (p=0.000 n=9+10)      
LogFieldType/Interface-32                  183ns ±12%     175ns ± 9%     ~     (p=0.078 n=10+10)     

Log message size savings is greatly dependent on the number and type of fields in the log message. Assuming this log message (with an Integer, timestamp and string, in addition to level).

{"level":"error","Fault":41650,"time":"2018-04-01T15:18:19-07:00","message":"Some Message"}

Two measurements were done for the log file sizes - one without any compression, second using compress/zlib.

Results for 10,000 log messages:

Log Format Plain File Size (in KB) Compressed File Size (in KB)
JSON 920 28
CBOR 550 28

The example used to calculate the above data is available in Examples.

Documentation

Overview

Package cbor provides primitives for storing different data in the CBOR (binary) format. CBOR is defined in RFC7049.

Index

Constants

This section is empty.

Variables

View Source
var IntegerTimeFieldFormat = time.RFC3339

IntegerTimeFieldFormat indicates the format of timestamp decoded from an integer (time in seconds).

View Source
var JSONMarshalFunc func(v interface{}) ([]byte, error)

JSONMarshalFunc is used to marshal interface to JSON encoded byte slice. Making it package level instead of embedded in Encoder brings some extra efforts at importing, but avoids value copy when the functions of Encoder being invoked. DO REMEMBER to set this variable at importing, or you might get a nil pointer dereference panic at runtime.

View Source
var NanoTimeFieldFormat = time.RFC3339Nano

NanoTimeFieldFormat indicates the format of timestamp decoded from a float value (time in seconds and nano seconds).

Functions

func AppendEmbeddedJSON

func AppendEmbeddedJSON(dst, s []byte) []byte

AppendEmbeddedJSON adds a tag and embeds input JSON as such.

func Cbor2JsonManyObjects

func Cbor2JsonManyObjects(src io.Reader, dst io.Writer) (err error)

Cbor2JsonManyObjects decodes all the CBOR Objects read from src reader. It keeps on decoding until reader returns EOF (error when reading). Decoded string is written to the dst. At the end of every CBOR Object newline is written to the output stream.

Returns error (if any) that was encountered during decode. The child functions will generate a panic when error is encountered and this function will recover non-runtime Errors and return the reason as error.

func DecodeIfBinaryToBytes

func DecodeIfBinaryToBytes(in []byte) []byte

DecodeIfBinaryToBytes checks if the input is a binary format, if so, it will decode all Objects and return the decoded string as byte array.

func DecodeIfBinaryToString

func DecodeIfBinaryToString(in []byte) string

DecodeIfBinaryToString converts a binary formatted log msg to a JSON formatted String Log message - suitable for printing to Console/Syslog.

func DecodeObjectToStr

func DecodeObjectToStr(in []byte) string

DecodeObjectToStr checks if the input is a binary format, if so, it will decode a single Object and return the decoded string.

Types

type Encoder added in v1.7.0

type Encoder struct{}

func (Encoder) AppendArrayDelim added in v1.7.0

func (Encoder) AppendArrayDelim(dst []byte) []byte

AppendArrayDelim adds markers to indicate end of a particular array element.

func (Encoder) AppendArrayEnd added in v1.7.0

func (Encoder) AppendArrayEnd(dst []byte) []byte

AppendArrayEnd adds markers to indicate the end of an array.

func (Encoder) AppendArrayStart added in v1.7.0

func (Encoder) AppendArrayStart(dst []byte) []byte

AppendArrayStart adds markers to indicate the start of an array.

func (Encoder) AppendBeginMarker added in v1.7.0

func (Encoder) AppendBeginMarker(dst []byte) []byte

AppendBeginMarker inserts a map start into the dst byte array.

func (Encoder) AppendBool added in v1.7.0

func (Encoder) AppendBool(dst []byte, val bool) []byte

AppendBool encodes and inserts a boolean value into the dst byte array.

func (Encoder) AppendBools added in v1.7.0

func (e Encoder) AppendBools(dst []byte, vals []bool) []byte

AppendBools encodes and inserts an array of boolean values into the dst byte array.

func (Encoder) AppendBytes added in v1.7.0

func (Encoder) AppendBytes(dst, s []byte) []byte

AppendBytes encodes and adds an array of bytes to the dst byte array.

func (Encoder) AppendDuration added in v1.7.0

func (e Encoder) AppendDuration(dst []byte, d time.Duration, unit time.Duration, useInt bool) []byte

AppendDuration encodes and adds a duration to the dst byte array. useInt field indicates whether to store the duration as seconds (integer) or as seconds+nanoseconds (float).

func (Encoder) AppendDurations added in v1.7.0

func (e Encoder) AppendDurations(dst []byte, vals []time.Duration, unit time.Duration, useInt bool) []byte

AppendDurations encodes and adds an array of durations to the dst byte array. useInt field indicates whether to store the duration as seconds (integer) or as seconds+nanoseconds (float).

func (Encoder) AppendEndMarker added in v1.7.0

func (Encoder) AppendEndMarker(dst []byte) []byte

AppendEndMarker inserts a map end into the dst byte array.

func (Encoder) AppendFloat32 added in v1.7.0

func (Encoder) AppendFloat32(dst []byte, val float32) []byte

AppendFloat32 encodes and inserts a single precision float value into the dst byte array.

func (Encoder) AppendFloat64 added in v1.7.0

func (Encoder) AppendFloat64(dst []byte, val float64) []byte

AppendFloat64 encodes and inserts a double precision float value into the dst byte array.

func (Encoder) AppendFloats32 added in v1.7.0

func (e Encoder) AppendFloats32(dst []byte, vals []float32) []byte

AppendFloats32 encodes and inserts an array of single precision float value into the dst byte array.

func (Encoder) AppendFloats64 added in v1.7.0

func (e Encoder) AppendFloats64(dst []byte, vals []float64) []byte

AppendFloats64 encodes and inserts an array of double precision float values into the dst byte array.

func (Encoder) AppendHex added in v1.7.0

func (e Encoder) AppendHex(dst []byte, val []byte) []byte

AppendHex adds a TAG and inserts a hex bytes as a string.

func (Encoder) AppendIPAddr added in v1.7.0

func (e Encoder) AppendIPAddr(dst []byte, ip net.IP) []byte

AppendIPAddr encodes and inserts an IP Address (IPv4 or IPv6).

func (Encoder) AppendIPPrefix added in v1.7.0

func (e Encoder) AppendIPPrefix(dst []byte, pfx net.IPNet) []byte

AppendIPPrefix encodes and inserts an IP Address Prefix (Address + Mask Length).

func (Encoder) AppendInt added in v1.7.0

func (Encoder) AppendInt(dst []byte, val int) []byte

AppendInt encodes and inserts an integer value into the dst byte array.

func (Encoder) AppendInt16 added in v1.7.0

func (e Encoder) AppendInt16(dst []byte, val int16) []byte

AppendInt16 encodes and inserts a int16 value into the dst byte array.

func (Encoder) AppendInt32 added in v1.7.0

func (e Encoder) AppendInt32(dst []byte, val int32) []byte

AppendInt32 encodes and inserts a int32 value into the dst byte array.

func (Encoder) AppendInt64 added in v1.7.0

func (Encoder) AppendInt64(dst []byte, val int64) []byte

AppendInt64 encodes and inserts a int64 value into the dst byte array.

func (Encoder) AppendInt8 added in v1.7.0

func (e Encoder) AppendInt8(dst []byte, val int8) []byte

AppendInt8 encodes and inserts an int8 value into the dst byte array.

func (Encoder) AppendInterface added in v1.7.0

func (e Encoder) AppendInterface(dst []byte, i interface{}) []byte

AppendInterface takes an arbitrary object and converts it to JSON and embeds it dst.

func (Encoder) AppendInts added in v1.7.0

func (e Encoder) AppendInts(dst []byte, vals []int) []byte

AppendInts encodes and inserts an array of integer values into the dst byte array.

func (Encoder) AppendInts16 added in v1.7.0

func (e Encoder) AppendInts16(dst []byte, vals []int16) []byte

AppendInts16 encodes and inserts an array of int16 values into the dst byte array.

func (Encoder) AppendInts32 added in v1.7.0

func (e Encoder) AppendInts32(dst []byte, vals []int32) []byte

AppendInts32 encodes and inserts an array of int32 values into the dst byte array.

func (Encoder) AppendInts64 added in v1.7.0

func (e Encoder) AppendInts64(dst []byte, vals []int64) []byte

AppendInts64 encodes and inserts an array of int64 values into the dst byte array.

func (Encoder) AppendInts8 added in v1.7.0

func (e Encoder) AppendInts8(dst []byte, vals []int8) []byte

AppendInts8 encodes and inserts an array of integer values into the dst byte array.

func (Encoder) AppendKey added in v1.7.0

func (e Encoder) AppendKey(dst []byte, key string) []byte

AppendKey adds a key (string) to the binary encoded log message

func (Encoder) AppendLineBreak added in v1.7.0

func (Encoder) AppendLineBreak(dst []byte) []byte

AppendLineBreak is a noop that keep API compat with json encoder.

func (Encoder) AppendMACAddr added in v1.7.0

func (e Encoder) AppendMACAddr(dst []byte, ha net.HardwareAddr) []byte

AppendMACAddr encodes and inserts an Hardware (MAC) address.

func (Encoder) AppendNil added in v1.7.0

func (Encoder) AppendNil(dst []byte) []byte

AppendNil inserts a 'Nil' object into the dst byte array.

func (Encoder) AppendObjectData added in v1.7.0

func (Encoder) AppendObjectData(dst []byte, o []byte) []byte

AppendObjectData takes an object in form of a byte array and appends to dst.

func (Encoder) AppendString added in v1.7.0

func (Encoder) AppendString(dst []byte, s string) []byte

AppendString encodes and adds a string to the dst byte array.

func (Encoder) AppendStrings added in v1.7.0

func (e Encoder) AppendStrings(dst []byte, vals []string) []byte

AppendStrings encodes and adds an array of strings to the dst byte array.

func (Encoder) AppendTime added in v1.7.0

func (e Encoder) AppendTime(dst []byte, t time.Time, unused string) []byte

AppendTime encodes and adds a timestamp to the dst byte array.

func (Encoder) AppendTimes added in v1.7.0

func (e Encoder) AppendTimes(dst []byte, vals []time.Time, unused string) []byte

AppendTimes encodes and adds an array of timestamps to the dst byte array.

func (Encoder) AppendUint added in v1.7.0

func (e Encoder) AppendUint(dst []byte, val uint) []byte

AppendUint encodes and inserts an unsigned integer value into the dst byte array.

func (Encoder) AppendUint16 added in v1.7.0

func (e Encoder) AppendUint16(dst []byte, val uint16) []byte

AppendUint16 encodes and inserts a uint16 value into the dst byte array.

func (Encoder) AppendUint32 added in v1.7.0

func (e Encoder) AppendUint32(dst []byte, val uint32) []byte

AppendUint32 encodes and inserts a uint32 value into the dst byte array.

func (Encoder) AppendUint64 added in v1.7.0

func (Encoder) AppendUint64(dst []byte, val uint64) []byte

AppendUint64 encodes and inserts a uint64 value into the dst byte array.

func (Encoder) AppendUint8 added in v1.7.0

func (e Encoder) AppendUint8(dst []byte, val uint8) []byte

AppendUint8 encodes and inserts a unsigned int8 value into the dst byte array.

func (Encoder) AppendUints added in v1.7.0

func (e Encoder) AppendUints(dst []byte, vals []uint) []byte

AppendUints encodes and inserts an array of unsigned integer values into the dst byte array.

func (Encoder) AppendUints16 added in v1.7.0

func (e Encoder) AppendUints16(dst []byte, vals []uint16) []byte

AppendUints16 encodes and inserts an array of uint16 values into the dst byte array.

func (Encoder) AppendUints32 added in v1.7.0

func (e Encoder) AppendUints32(dst []byte, vals []uint32) []byte

AppendUints32 encodes and inserts an array of uint32 values into the dst byte array.

func (Encoder) AppendUints64 added in v1.7.0

func (e Encoder) AppendUints64(dst []byte, vals []uint64) []byte

AppendUints64 encodes and inserts an array of uint64 values into the dst byte array.

func (Encoder) AppendUints8 added in v1.7.0

func (e Encoder) AppendUints8(dst []byte, vals []uint8) []byte

AppendUints8 encodes and inserts an array of uint8 values into the dst byte array.

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL