README
¶
go-snappystream
a Go package for framed snappy streams.
This package wraps snappy-go and supplies a Reader and Writer
for the snappy framed stream format.
Documentation
¶
Overview ¶
snappystream wraps snappy-go and supplies a Reader and Writer for the snappy framed stream format:
https://snappy.googlecode.com/svn/trunk/framing_format.txt
Index ¶
Constants ¶
const ContentEncoding = "x-snappy-framed"
ContentEncoding is the appropriate HTTP Content-Encoding header value for requests containing a snappy framed entity body.
const Ext = ".sz"
Ext is the file extension for files whose content is a snappy framed stream.
const MaxBlockSize = 65536
MaxBlockSize is the maximum number of decoded bytes allowed to be represented in a snappy framed block (sections 4.2 and 4.3).
const MediaType = "application/x-snappy-framed"
MediaType is the MIME type used to represent snappy framed content.
const SkipVerifyChecksum = false
const VerifyChecksum = true
Variables ¶
This section is empty.
Functions ¶
func NewReader ¶
NewReader returns an io.Reader interface to the snappy framed stream format.
It transparently handles reading the stream identifier (but does not proxy this to the caller), decompresses blocks, and (optionally) validates checksums.
Internally, three buffers are maintained. The first two are for reading off the wrapped io.Reader and for holding the decompressed block (both are grown automatically and re-used and will never exceed the largest block size, 65536). The last buffer contains the *unread* decompressed bytes (and can grow indefinitely).
The second param determines whether or not the reader will verify block checksums and can be enabled/disabled with the constants VerifyChecksum and SkipVerifyChecksum
For each Read, the returned length will be up to the lesser of len(b) or 65536 decompressed bytes, regardless of the length of *compressed* bytes read from the wrapped io.Reader.
func NewWriter ¶
NewWriter returns an io.Writer that writes its input to an underlying io.Writer encoded as a snappy framed stream. A stream identifier block is written to w preceding the first data block. The returned writer will never emit a block with length in bytes greater than MaxBlockSize+4 nor one containing more than MaxBlockSize bytes of (uncompressed) data.
For each Write, the returned length will only ever be len(p) or 0, regardless of the length of *compressed* bytes written to the wrapped io.Writer. If the returned length is 0 then error will be non-nil. If len(p) exceeds 65536, the slice will be automatically chunked into smaller blocks which are all emitted before the call returns.
Types ¶
type BufferedWriter ¶
type BufferedWriter struct {
// contains filtered or unexported fields
}
BufferedWriter is an io.WriteCloser with behavior similar to writers returned by NewWriter but it buffers written data, maximizing block size (to improve the output compression ratio) at the cost of speed. Benefits over NewWriter are most noticible when individual writes are small and when streams are long.
Failure to call a BufferedWriter's Close or Flush methods after it is done being written to will likely result in missing data frames which will be undetectable in the decoding process.
NOTE: BufferedWriter cannot be instantiated via struct literal and must use NewBufferedWriter (i.e. its zero value is not usable).
func NewBufferedWriter ¶
func NewBufferedWriter(w io.Writer) *BufferedWriter
NewBufferedWriter allocates and returns a BufferedWriter with an internal buffer of MaxBlockSize bytes. If an error occurs writing a block to w, all future writes will fail with the same error. After all data has been written, the client should call the Flush method to guarantee all data has been forwarded to the underlying io.Writer.
func (*BufferedWriter) Close ¶
func (w *BufferedWriter) Close() error
Close flushes w's internal buffer and tears down internal data structures. After a successful call to Close method calls on w return an error. Close makes no attempt to close the underlying writer.
func (*BufferedWriter) Flush ¶
func (w *BufferedWriter) Flush() error
Flush encodes and writes a block with the contents of w's internal buffer to the underlying writer even if the buffer does not contain a full block of data (MaxBlockSize bytes).
func (*BufferedWriter) ReadFrom ¶ added in v0.2.2
func (w *BufferedWriter) ReadFrom(r io.Reader) (int64, error)
ReadFrom implements the io.ReaderFrom interface used by io.Copy. It encodes data read from r as a snappy framed stream that is written to the underlying writer. ReadFrom returns the number number of bytes read, along with any error encountered (other than io.EOF).
Source Files
Directories