Documentation
¶
Overview ¶
Package compressio provides parallel compression and decompression, as well as optional SHA-256 hashing.
The stream format is defined as follows.
/------------------------------------------------------\ | chunk size (4-bytes) | +------------------------------------------------------+ | (optional) hash (32-bytes) | +------------------------------------------------------+ | compressed data size (4-bytes) | +------------------------------------------------------+ | compressed data | +------------------------------------------------------+ | (optional) hash (32-bytes) | +------------------------------------------------------+ | compressed data size (4-bytes) | +------------------------------------------------------+ | ...... | \------------------------------------------------------/
where each subsequent hash is calculated from the following items in order
compressed data compressed data size previous hash
so the stream integrity cannot be compromised by switching and mixing compressed chunks.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrHashMismatch = errors.New("hash mismatch")
ErrHashMismatch is returned if the hash does not match.
Functions ¶
func NewReader ¶
NewReader returns a new compressed reader. If key is non-nil, the data stream is assumed to contain expected hash values, which will be compared against hash values computed from the compressed bytes. See package comments for details.
func NewWriter ¶
NewWriter returns a new compressed writer. If key is non-nil, hash values are generated and written out for compressed bytes. See package comments for details.
The recommended chunkSize is on the order of 1M. Extra memory may be buffered (in the form of read-ahead, or buffered writes), and is limited to O(chunkSize * [1+GOMAXPROCS]).
Types ¶
This section is empty.
Source Files