Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var DefaultConfig = Config{
StartSize: minDictionarySize,
RatioImprovements: 0.1,
SamplePath: "",
DictionaryPath: "./codec/zbor/",
}
DefaultConfig is the default configuration for the Mapper.
Functions ¶
This section is empty.
Types ¶
type Config ¶
type Config struct {
// The dictionary size in kB to start with when generating dictionaries.
// Gets multiplied by 2 at each loop.
StartSize int
// The tolerance for the improvement of compression ratio between each loop. Should be between 0 and 1.
// For example, a value of 0.1 means that as long as a dictionary is at least 10% more performant than the
// previously generated one, its size increase is tolerated and the generation loop continues. Only when a
// dictionary is generated which is not at least 10% more performant than the previous one does the loop
// stop, and the previous dictionary is selected as the most optimized one.
RatioImprovements float64
// The path in which to store samples that are generated temporarily to be used for training dictionaries.
SamplePath string
// The path in which to store compiled Go dictionaries. Should point to the package in which they should be used.
DictionaryPath string
}
type DictionaryKind ¶
type DictionaryKind string
DictionaryKind represents the resources that the dictionary is trained to compress efficiently.
const ( KindPayloads DictionaryKind = "payloads" KindEvents DictionaryKind = "events" KindTransactions DictionaryKind = "transactions" )
Supported dictionary kinds.
func (DictionaryKind) String ¶
func (k DictionaryKind) String() string
type Generator ¶
type Generator struct {
// contains filtered or unexported fields
}
Generator generates optimized Zstandard dictionaries and turns them into Go files to be used for compression.
func (*Generator) Dictionary ¶
func (g *Generator) Dictionary(kind DictionaryKind) error
Dictionary generates and compiles an optimized dictionary of the given kind.
type Option ¶
type Option func(*Config)
Option is an option that can be given to the generator to configure optional parameters on initialization.
func WithDictionaryPath ¶
WithDictionaryPath sets path in which to store compiled dictionaries.
func WithRatioImprovementTolerance ¶
WithRatioImprovementTolerance sets the total size in bytes of samples to use for benchmarking. Using high values will result in a more accurate calculation of the compression ratio at the expense of making benchmarks longer.
func WithSamplePath ¶
WithSamplePath sets path in which to temporarily store generated data samples.
func WithStartSize ¶
WithStartSize sets the dictionary size in Bytes to start with when generating dictionaries. This value cannot be below 512B, or it will trigger errors in the Zstandard training algorithm. See https://github.com/facebook/zstd/issues/2815
Source Files