README
¶
msgpackzip
Usage
You can test compressing a message pack encoded input file by running:
go run ./cli/main.go <inputfile>
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrBinaryTooBig = errors.New("binary allocation is too big")
var ErrContainerTooBig = errors.New("container allocation is too big")
var ErrExtTooBig = errors.New("extenal data type too big")
var ErrIntTooBig = errors.New("Cannot handle ints largers than int64 max")
var ErrLenTooBig = errors.New("Lenghts bigger than 0x8000000 are too big")
var ErrMaxDepth = errors.New("input exceeded maximum allowed depth")
var ErrStringTooBig = errors.New("string allocation is too big")
Functions ¶
func Compress ¶
Compress the given msgpack encoding, compressing only static map keys, and not compressing any values.
func CompressWithWhitelist ¶
func CompressWithWhitelist(input []byte, wl ValueWhitelist) (output []byte, err error)
CompressWithWhitelist takes as input a msgpack encoded payload, and also a whitelist of values that it's OK to compress. It then compresses all map keys and values in the given whitelist, and returns a compression, or an error on error.
Types ¶
type BinaryMapKey ¶
type BinaryMapKey string
BinaryMapKey is a wrapper around a []byte vector of binary data so that it can be stored as an interface{} and differentiated from proper strings.
type Frequency ¶
type Frequency struct {
Key interface{}
Freq int
}
Frequency is a tuple, with a `Key interface{}` that can be an int64, a string, or a BinaryMapKey (which is a wrapper around a binary buffer). The `Freq` field is a count for how many times the `Key` shows up in the encoded msgpack object.
func ReportValuesFrequencies ¶
ReportValuesFrequencies takes as input a msgpack encoding, and reports which values are the most fequent in the encoding. It returns a list of Frequency objects, sorted from most frequent to least frequent.
type ValueWhitelist ¶
type ValueWhitelist struct {
// contains filtered or unexported fields
}
ValueWhitelist can be used to specify which values can be compressed. Values are either strings or binary []byte arrays.
func NewValueWhitelist ¶
func NewValueWhitelist() *ValueWhitelist
NewValueWhitelist makes an empty value white list, initialzed with empty lists.
func (*ValueWhitelist) AddBinary ¶
func (v *ValueWhitelist) AddBinary(b []byte)
AddBinary adds a binary buffer to the value whitelist
func (*ValueWhitelist) AddString ¶
func (v *ValueWhitelist) AddString(s string)
AddString adds a string to the value whitelist
Source Files
Directories