README
¶
#sz
Though this was fun to write, https://github.com/mreiferson/go-snappystream appears more complete and better tested. Go forth and compress!
sz provides a Reader and Writer for using the framing format
for Google's Snappy compression algorithm from Go code. If you
just want to encode and decode bare blocks, you don't need this; use the
snappy-go library.
sz includes a modified version of snappy-go with snappy-go's encoding logic ported to C. (This isn't the more optimized C++ Snappy code.) It's disabled by default; change the import statements in sz.go to use it. If used, it's compiled into your binary: target systems do not need the snappy library, though building the package requires a C compiler targeting the right platform. The port ran about 50% faster for my compressible test content. It passes the original test suite, but that's all the assurance we have, so be aware. .
In general, sz is not a mature, tested implementation. Some things that would be great:
- Tests. Simple "you can round-trip a file" test, odd read/write/block size patterns.
- A "raw" Writer that makes each Write call output a compressed block, rather than buffering up 64KB unless flushed/closed earlier.
- Smart skipping of long incompressible runs: after several blocks fail to compress, stop trying for a while, or only test-compress small samples.
Randall Farmer, 2014. sz is under an MIT license whose full text is in
LICENSE. sz/snappy is under a BSD-style license whose full text is in
snappy/LICENSE. There is no endorsement by the Snappy-Go project.
Documentation
¶
Overview ¶
Package sz is an immature implementation of a Reader/Writer for the Snappy framing format. Use at your own risk; bug reports and test cases are welcome.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrCRCMismatch = errors.New("sz: CRC mismatch")
var ErrFramingCorrupt = errors.New("sz: snappy framing corrupt")
var ErrStrictMemLimitExceeded = errors.New("sz: oversized block in strict-memory-usage mode")
var ErrUnexpectedChunk = errors.New("sz: unexpected chunk type")
Functions ¶
This section is empty.
Types ¶
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader reads content in the Snappy framing format.
func NewReaderStrictMem ¶
NewReaderStrictMem returns a Reader that rejects any chunk that's larger than a Snappy compressor would produce even for worst-case incompressible content. This is against the framing format spec (which permits 16MB chunks), but prevents a specially constructed stream from using a lot of memory, and may be useful when consuming untrusted content.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
sz
cmd/sz is a toy program for playing with github.com/twotwotwo/sz.
|
cmd/sz is a toy program for playing with github.com/twotwotwo/sz. |
|
Package snappy implements the snappy block-based compression format.
|
Package snappy implements the snappy block-based compression format. |