Documentation ¶
Index ¶
- Constants
- Variables
- func Abs(x int32) int32
- func ComputeFirstOrderEntropy1024(blockLen int, histo []int) int
- func ComputeHistogram(block []byte, freqs []int, isOrder0, withTotal bool)
- func ComputeJobsPerTask(jobsPerTask []uint, jobs, tasks uint) ([]uint, error)
- func GetMagicType(src []byte) uint
- func IsDataCompressed(magic uint) bool
- func IsDataExecutable(magic uint) bool
- func IsDataMultimedia(magic uint) bool
- func IsPowerOf2(x int32) bool
- func Log2(x uint32) (uint32, error)
- func Log2NoCheck(x uint32) uint32
- func Log2ScaledBy1024(x uint32) (uint32, error)
- func Lsb(x int32) int32
- func Max(x, y int32) int32
- func Min(x, y int32) int32
- func Msb(x int32) int32
- func PositiveOrNull(x int32) int32
- func ResetLsb(x int32) int32
- func RoundUpPowerOfTwo(x int32) int32
- func Squash(d int) int
- type ByteTransform
- type DataType
- type EntropyDecoder
- type EntropyEncoder
- type Event
- type InputBitStream
- type IntTransform
- type Listener
- type Magic
- type OutputBitStream
- type Predictor
Constants ¶
const ( EVT_COMPRESSION_START = 0 // Compression starts EVT_DECOMPRESSION_START = 1 // Decompression starts EVT_BEFORE_TRANSFORM = 2 // Transform forward/inverse starts EVT_AFTER_TRANSFORM = 3 // Transform forward/inverse ends EVT_BEFORE_ENTROPY = 4 // Entropy encoding/decoding starts EVT_AFTER_ENTROPY = 5 // Entropy encoding/decoding ends EVT_COMPRESSION_END = 6 // Compression ends EVT_DECOMPRESSION_END = 7 // Decompression ends EVT_AFTER_HEADER_DECODING = 8 // Compression header decoding ends )
const ( ERR_MISSING_PARAM = 1 ERR_BLOCK_SIZE = 2 ERR_INVALID_CODEC = 3 ERR_CREATE_COMPRESSOR = 4 ERR_CREATE_DECOMPRESSOR = 5 ERR_OUTPUT_IS_DIR = 6 ERR_OVERWRITE_FILE = 7 ERR_CREATE_FILE = 8 ERR_CREATE_BITSTREAM = 9 ERR_OPEN_FILE = 10 ERR_READ_FILE = 11 ERR_WRITE_FILE = 12 ERR_PROCESS_BLOCK = 13 ERR_CREATE_CODEC = 14 ERR_INVALID_FILE = 15 ERR_STREAM_VERSION = 16 ERR_CREATE_STREAM = 17 ERR_INVALID_PARAM = 18 ERR_CRC_CHECK = 19 ERR_UNKNOWN = 127 )
const ( NO_MAGIC = 0 JPG_MAGIC = 0xFFD8FFE0 GIF_MAGIC = 0x47494638 PDF_MAGIC = 0x25504446 ZIP_MAGIC = 0x504B0304 // Works for jar & office docs LZMA_MAGIC = 0x377ABCAF // Works for 7z 37 7A BC AF 27 1C PNG_MAGIC = 0x89504E47 ELF_MAGIC = 0x7F454C46 MAC_MAGIC32 = 0xFEEDFACE MAC_CIGAM32 = 0xCEFAEDFE MAC_MAGIC64 = 0xFEEDFACF MAC_CIGAM64 = 0xCFFAEDFE ZSTD_MAGIC = 0x28B52FFD BROTLI_MAGIC = 0x81CFB2CE RIFF_MAGIC = 0x52494646 // WAV, AVI, WEBP CAB_MAGIC = 0x4D534346 FLAC_MAGIC = 0x664C6143 XZ_MAGIC = 0xFD377A58 // FD 37 7A 58 5A 00 KNZ_MAGIC = 0x4B414E5A BZIP2_MAGIC = 0x425A68 MP3_ID3_MAGIC = 0x494433 GZIP_MAGIC = 0x1F8B BMP_MAGIC = 0x424D WIN_MAGIC = 0x4D5A PBM_MAGIC = 0x5034 // bin only PGM_MAGIC = 0x5035 // bin only PPM_MAGIC = 0x5036 // bin only )
Variables ¶
var ( KEYS32 = [17]uint{ GIF_MAGIC, PDF_MAGIC, ZIP_MAGIC, LZMA_MAGIC, PNG_MAGIC, ELF_MAGIC, MAC_MAGIC32, MAC_CIGAM32, MAC_MAGIC64, MAC_CIGAM64, ZSTD_MAGIC, BROTLI_MAGIC, CAB_MAGIC, RIFF_MAGIC, FLAC_MAGIC, XZ_MAGIC, KNZ_MAGIC, } KEYS16 = [3]uint{ GZIP_MAGIC, BMP_MAGIC, WIN_MAGIC, } )
var LOG2 = [...]uint32{}/* 256 elements not displayed */
LOG2 is an array with 256 elements: int(Math.log2(x-1))
var LOG2_4096 = [...]uint32{}/* 257 elements not displayed */
LOG2_4096 is an array with 256 elements: 4096*Math.log2(x)
var SQUASH [4096]int
SQUASH contains p = 1/(1 + exp(-d)), d scaled by 8 bits, p scaled by 12 bits
var STRETCH [4096]int
STRETCH is the inverse of squash. d = ln(p/(1-p)), d scaled by 8 bits, p by 12 bits. d has range -2047 to 2047 representing -8 to 8. p in [0..4095].
Functions ¶
func ComputeFirstOrderEntropy1024 ¶
ComputeFirstOrderEntropy1024 computes the order 0 entropy of the block and scales the result by 1024 (result in the [0..1024] range) Fills in the histogram with order 0 frequencies. Incoming array size must be at least 256
func ComputeHistogram ¶
ComputeHistogram computes the order 0 or order 1 histogram for the input block and returns it in the 'freqs' slice. If withTotal is true, the last spot in each frequencies order 0 array is for the total (each order 0 frequency slice must be of length 257 in this case).
func ComputeJobsPerTask ¶
ComputeJobsPerTask computes the number of jobs associated with each task given a number of jobs available and a number of tasks to perform. The provided 'jobsPerTask' slice is returned as result.
func GetMagicType ¶
GetMagicType checks the first bytes of the slice against a list of common magic values 4 bytes must be available in 'src'
func IsDataCompressed ¶
IsDataCompressed return true if the provided magic parameter corresponds to a known compressed data type.
func IsDataExecutable ¶
IsDataExecutable return true if the provided magic parameter corresponds to a known executable data type.
func IsDataMultimedia ¶
IsDataMultimedia return true if the provided magic parameter corresponds to a known multimedia data type.
func IsPowerOf2 ¶
IsPowerOf2 returns true if the input value is a power of two
func Log2NoCheck ¶
Log2NoCheck does the same as Log2() minus a null check on input value
func Log2ScaledBy1024 ¶
Log2ScaledBy1024 returns 1024 * log2(x). Max error is around 0.1%
func PositiveOrNull ¶
PositiveOrNull returns the input value if positive else 0
func RoundUpPowerOfTwo ¶
RoundUpPowerOfTwo returns the smnallest power of two greater than or equal to the input value
Types ¶
type ByteTransform ¶
type ByteTransform interface { // Forward applies the function to the src and writes the result // to the destination. Returns number of bytes read, number of bytes // written and possibly an error. Forward(src, dst []byte) (uint, uint, error) // Inverse applies the reverse function to the src and writes the result // to the destination. Returns number of bytes read, number of bytes // written and possibly an error. Inverse(src, dst []byte) (uint, uint, error) // MaxEncodedLen returns the max size required for the encoding output buffer MaxEncodedLen(srcLen int) int }
ByteTransform is a function that transforms the input byte slice and writes the result in the output byte slice. The result may have a different size.
type EntropyDecoder ¶
type EntropyDecoder interface { // Read decodes data from the bitstream and return it in the provided buffer. // Return the number of bytes read from the bitstream Read(block []byte) (int, error) // BitStream returns the underlying bitstream BitStream() InputBitStream // Dispose must be called before getting rid of the entropy decoder // Trying to decode after a call to dispose gives undefined behavior Dispose() }
EntropyDecoder entropy decodes data from a bitstream
type EntropyEncoder ¶
type EntropyEncoder interface { // Write encodes the data provided into the bitstream. Return the number of bytes // written to the bitstream Write(block []byte) (int, error) // BitStream returns the underlying bitstream BitStream() OutputBitStream // Dispose must be called before getting rid of the entropy encoder // Trying to encode after a call to dispose gives undefined behavior Dispose() }
EntropyEncoder entropy encodes data to a bitstream
type Event ¶
type Event struct {
// contains filtered or unexported fields
}
Event a compression/decompression event
func NewEventFromString ¶
NewEventFromString creates a new Event instance that wraps a message
type InputBitStream ¶
type InputBitStream interface { // ReadBit returns the next bit in the bitstream. Panics if closed or EOS is reached. ReadBit() int // ReadBits reads 'length' (in [1..64]) bits from the bitstream . // Returns the bits read as an uint64. // Panics if closed or EOS is reached. ReadBits(length uint) uint64 // ReadArray reads 'length' bits from the bitstream and put them in the byte slice. // Returns the number of bits read. // Panics if closed or EOS is reached. ReadArray(bits []byte, length uint) uint // Close makes the bitstream unavailable for further reads. Close() (bool, error) // Read returns the number of bits read Read() uint64 // HasMoreToRead returns false when the bitstream is closed or the EOS has been reached HasMoreToRead() (bool, error) }
InputBitStream is a bitstream reader
type IntTransform ¶
type IntTransform interface { // Forward applies the function to the src and writes the result // to the destination. Returns number of bytes read, number of bytes // written and possibly an error. Forward(src, dst []int) (uint, uint, error) // Inverse applies the reverse function to the src and writes the result // to the destination. Returns number of bytes read, number of bytes // written and possibly an error. Inverse(src, dst []int) (uint, uint, error) // MaxEncodedLen returns the max size required for the encoding output buffer // If the max size of the output buffer is not known, return -1 MaxEncodedLen(srcLen int) int }
IntTransform is a function that transforms the input int slice and writes the result in the output int slice. The result may have a different size.
type Listener ¶
type Listener interface {
ProcessEvent(evt *Event)
}
Listener is an interface implemented by event processors
type OutputBitStream ¶
type OutputBitStream interface { // WriteBit writes the least significant bit of the input integer // Panics if closed or an IO error is received. WriteBit(bit int) // WriteBits writes the least significant bits of 'bits' to the bitstream. // Length is the number of bits to write (in [1..64]). // Returns the number of bits written. // Panics if closed or an IO error is received. WriteBits(bits uint64, length uint) uint // WriteArray writes bits out of the byte slice. Length is the number of bits. // Returns the number of bits written. // Panics if closed or an IO error is received. WriteArray(bits []byte, length uint) uint // Close makes the bitstream unavailable for further writes. Close() (bool, error) // Written returns the number of bits written Written() uint64 }
OutputBitStream is a bitstream writer
type Predictor ¶
type Predictor interface { // Update updates the internal probability model based on the observed bit Update(bit byte) // Get returns the value representing the probability of the next bit being 1 // in the [0..4095] range. // E.G. 410 represents roughly a probability of 10% for 1 Get() int }
Predictor predicts the probability of the next bit being 1.