README
¶
pzlib
Go parallel zlib compression/decompression. This is a fully zlib compatible drop in replacement for "compress/zlib".
The implementation is inspired by klauspost/pgzip, and completed as the final project of CMU 15-618: Parallel Computer Architecture and Programming.
According to our experiment, pzlib can achieve a 38× speedup compared to "compress/zlib" on 36-core AWS machines compressing files of 1GB, with moderate file size overhead after compression. Detailed benchmark results can be found in this report.
Documentation
¶
Index ¶
Constants ¶
const ( NoCompression = flate.NoCompression BestSpeed = flate.BestSpeed BestCompression = flate.BestCompression DefaultCompression = flate.DefaultCompression HuffmanOnly = flate.HuffmanOnly )
Variables ¶
var ( // ErrChecksum is returned when reading ZLIB data that has an invalid checksum. ErrChecksum = errors.New("zlib: invalid checksum") // ErrDictionary is returned when reading ZLIB data that has an invalid dictionary. ErrDictionary = errors.New("zlib: invalid dictionary") // ErrHeader is returned when reading ZLIB data that has an invalid header. ErrHeader = errors.New("zlib: invalid header") )
Functions ¶
func NewReader ¶
func NewReader(r io.Reader) (io.ReadCloser, error)
NewReader creates a new ReadCloser. Reads from the returned ReadCloser read and decompress data from r. If r does not implement io.ByteReader, the decompressor may read more data than necessary from r. It is the caller's responsibility to call Close on the ReadCloser when done.
The ReadCloser returned by NewReader also implements Resetter.
func NewReaderDict ¶
NewReaderDict is like NewReader but uses a preset dictionary. NewReaderDict ignores the dictionary if the compressed data does not refer to it. If the compressed data refers to a different dictionary, NewReaderDict returns ErrDictionary.
The ReadCloser returned by NewReaderDict also implements Resetter.
Types ¶
type Resetter ¶
type Resetter interface {
// Reset discards any buffered data and resets the Resetter as if it was
// newly initialized with the given reader.
Reset(r io.Reader, dict []byte) error
}
Resetter resets a ReadCloser returned by NewReader or NewReaderDict to switch to a new underlying Reader. This permits reusing a ReadCloser instead of allocating a new one.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
func NewWriterLevelDict ¶
func (*Writer) Close ¶
Close closes the Writer, flushing any unwritten data to the underlying io.Writer, but does not close the underlying io.Writer.
Source Files