bitio: github.com/icza/bitio Index | Files

package bitio

import "github.com/icza/bitio"

Package bitio provides an optimized bit-level Reader and Writer.

You can use Reader.ReadBits() to read arbitrary number of bits from an io.Reader and return it as an uint64, and Writer.WriteBits() to write arbitrary number of bits of an uint64 value to an io.Writer.

Both Reader and Writer also provide optimized methods for reading / writing 1 bit of information in the form of a bool value: Reader.ReadBool() and Writer.WriteBool(). These make this package ideal for compression algorithms that use Huffman coding for example, where decision whether to step left or right in the Huffman tree is the most frequent operation.

Reader and Writer give a bit-level view of the underlying io.Reader and io.Writer, but they also provide a byte-level view (io.Reader and io.Writer) at the same time. This means you can also use the Reader.Read() and Writer.Write() methods to read and write slices of bytes. These will give you best performance if the underlying io.Reader and io.Writer are aligned to a byte boundary (else all the individual bytes are assembled from / spread to multiple bytes). You can ensure byte boundary alignment by calling the Align() method of Reader and Writer. As an extra, io.ByteReader and io.ByteWriter are also implemented.

Bit order

The more general highest-bits-first order is used. So for example if the input provides the bytes 0x8f and 0x55:

HEXA    8    f     5    5
BINARY  1100 1111  0101 0101
        aaaa bbbc  ccdd dddd

Then ReadBits will return the following values:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a, err := r.ReadBits(4) //   1100 = 0x08
b, err := r.ReadBits(3) //    111 = 0x07
c, err := r.ReadBits(3) //    101 = 0x05
d, err := r.ReadBits(6) // 010101 = 0x15

Writing the above values would result in the same sequence of bytes:

b := &bytes.Buffer{}
w := NewWriter(b)
err := w.WriteBits(0x08, 4)
err = w.WriteBits(0x07, 3)
err = w.WriteBits(0x05, 3)
err = w.WriteBits(0x15, 6)
err = w.Close()
// b will hold the bytes: 0x8f and 0x55

Error handling

All ReadXXX() and WriteXXX() methods return an error which you are expected to handle. For convenience, there are also matching TryReadXXX() and TryWriteXXX() methods which do not return an error. Instead they store the (first) error in the Reader.TryError / Writer.TryError field which you can inspect later. These TryXXX() methods are a no-op if a TryError has been encountered before, so it's safe to call multiple TryXXX() methods and defer the error checking.

For example:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := r.TryReadBits(4) //   1100 = 0x08
b := r.TryReadBits(3) //    111 = 0x07
c := r.TryReadBits(3) //    101 = 0x05
d := r.TryReadBits(6) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

This allows you to easily convert the result of individual ReadBits(), like this:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := byte(r.TryReadBits(4))   //   1100 = 0x08
b := int32(r.TryReadBits(3))  //    111 = 0x07
c := int64(r.TryReadBits(3))  //    101 = 0x05
d := uint16(r.TryReadBits(6)) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

And similarly:

b := &bytes.Buffer{}
w := NewWriter(b)
w.TryWriteBits(0x08, 4)
w.TryWriteBits(0x07, 3)
w.TryWriteBits(0x05, 3)
w.TryWriteBits(0x15, 6)
if w.TryError != nil {
    // Handle error
}
err = w.Close()
// b will hold the bytes: 0x8f and 0x55

Index

Package Files

doc.go reader.go writer.go

type Reader Uses

type Reader struct {

    // TryError holds the first error occurred in TryXXX() methods.
    TryError error
    // contains filtered or unexported fields
}

Reader is the bit reader implementation.

For convenience, it also implements io.Reader and io.ByteReader.

func NewReader Uses

func NewReader(in io.Reader) *Reader

NewReader returns a new Reader using the specified io.Reader as the input (source).

func (*Reader) Align Uses

func (r *Reader) Align() (skipped uint8)

Align aligns the bit stream to a byte boundary, so next read will read/use data from the next byte. Returns the number of unread / skipped bits.

func (*Reader) Read Uses

func (r *Reader) Read(p []byte) (n int, err error)

Read reads up to len(p) bytes (8 * len(p) bits) from the underlying reader.

Read implements io.Reader, and gives a byte-level view of the bit stream. This will give best performance if the underlying io.Reader is aligned to a byte boundary (else all the individual bytes are assembled from multiple bytes). Byte boundary can be ensured by calling Align().

func (*Reader) ReadBits Uses

func (r *Reader) ReadBits(n uint8) (u uint64, err error)

ReadBits reads n bits and returns them as the lowest n bits of u.

func (*Reader) ReadBool Uses

func (r *Reader) ReadBool() (b bool, err error)

ReadBool reads the next bit, and returns true if it is 1.

func (*Reader) ReadByte Uses

func (r *Reader) ReadByte() (b byte, err error)

ReadByte reads the next 8 bits and returns them as a byte.

ReadByte implements io.ByteReader.

func (*Reader) TryRead Uses

func (r *Reader) TryRead(p []byte) (n int)

TryRead tries to read up to len(p) bytes (8 * len(p) bits) from the underlying reader.

If there was a previous TryError, it does nothing. Else it calls Read(), returns the data it provides and stores the error in the TryError field.

func (*Reader) TryReadBits Uses

func (r *Reader) TryReadBits(n uint8) (u uint64)

TryReadBits tries to read n bits.

If there was a previous TryError, it does nothing. Else it calls ReadBits(), returns the data it provides and stores the error in the TryError field.

func (*Reader) TryReadBool Uses

func (r *Reader) TryReadBool() (b bool)

TryReadBool tries to read the next bit, and return true if it is 1.

If there was a previous TryError, it does nothing. Else it calls ReadBool(), returns the data it provides and stores the error in the TryError field.

func (*Reader) TryReadByte Uses

func (r *Reader) TryReadByte() (b byte)

TryReadByte tries to read the next 8 bits and return them as a byte.

If there was a previous TryError, it does nothing. Else it calls ReadByte(), returns the data it provides and stores the error in the TryError field.

type Writer Uses

type Writer struct {

    // TryError holds the first error occurred in TryXXX() methods.
    TryError error
    // contains filtered or unexported fields
}

Writer is the bit writer implementation.

For convenience, it also implements io.WriterCloser and io.ByteWriter.

func NewWriter Uses

func NewWriter(out io.Writer) *Writer

NewWriter returns a new Writer using the specified io.Writer as the output.

Must be closed in order to flush cached data. If you can't or don't want to close it, flushing data can also be forced by calling Align().

func (*Writer) Align Uses

func (w *Writer) Align() (skipped uint8, err error)

Align aligns the bit stream to a byte boundary, so next write will start/go into a new byte. If there are cached bits, they are first written to the output. Returns the number of skipped (unset but still written) bits.

func (*Writer) Close Uses

func (w *Writer) Close() (err error)

Close closes the bit writer, writes out cached bits. It does not close the underlying io.Writer.

Close implements io.Closer.

func (*Writer) TryAlign Uses

func (w *Writer) TryAlign() (skipped uint8)

TryAlign tries to align the bit stream to a byte boundary.

If there was a previous TryError, it does nothing. Else it calls Align(), returns the data it provides and stores the error in the TryError field.

func (*Writer) TryWrite Uses

func (w *Writer) TryWrite(p []byte) (n int)

TryWrite tries to write len(p) bytes (8 * len(p) bits) to the underlying writer.

If there was a previous TryError, it does nothing. Else it calls Write(), returns the data it provides and stores the error in the TryError field.

func (*Writer) TryWriteBits Uses

func (w *Writer) TryWriteBits(r uint64, n uint8)

TryWriteBits tries to write out the n lowest bits of r.

If there was a previous TryError, it does nothing. Else it calls WriteBits(), and stores the error in the TryError field.

func (*Writer) TryWriteBitsUnsafe Uses

func (w *Writer) TryWriteBitsUnsafe(r uint64, n uint8)

TryWriteBitsUnsafe tries to write out the n lowest bits of r.

If there was a previous TryError, it does nothing. Else it calls WriteBitsUnsafe(), and stores the error in the TryError field.

func (*Writer) TryWriteBool Uses

func (w *Writer) TryWriteBool(b bool)

TryWriteBool tries to write one bit: 1 if param is true, 0 otherwise.

If there was a previous TryError, it does nothing. Else it calls WriteBool(), and stores the error in the TryError field.

func (*Writer) TryWriteByte Uses

func (w *Writer) TryWriteByte(b byte)

TryWriteByte tries to write 8 bits.

If there was a previous TryError, it does nothing. Else it calls WriteByte(), and stores the error in the TryError field.

func (*Writer) Write Uses

func (w *Writer) Write(p []byte) (n int, err error)

Write writes len(p) bytes (8 * len(p) bits) to the underlying writer.

Write implements io.Writer, and gives a byte-level interface to the bit stream. This will give best performance if the underlying io.Writer is aligned to a byte boundary (else all the individual bytes are spread to multiple bytes). Byte boundary can be ensured by calling Align().

func (*Writer) WriteBits Uses

func (w *Writer) WriteBits(r uint64, n uint8) (err error)

WriteBits writes out the n lowest bits of r. Bits of r in positions higher than n are ignored.

For example:

err := w.WriteBits(0x1234, 8)

is equivalent to:

err := w.WriteBits(0x34, 8)

func (*Writer) WriteBitsUnsafe Uses

func (w *Writer) WriteBitsUnsafe(r uint64, n uint8) (err error)

WriteBitsUnsafe writes out the n lowest bits of r.

r must not have bits set at n or higher positions (zero indexed). If r might not satisfy this, a mask must be explicitly applied before passing it to WriteBitsUnsafe(), or WriteBits() should be used instead.

WriteBitsUnsafe() offers slightly better performance than WriteBits() because the input r is not masked. Calling WriteBitsUnsafe() with an r that does not satisfy this is undefined behavior (might corrupt previously written bits).

E.g. if you want to write 8 bits:

err := w.WriteBitsUnsafe(0x34, 8)        // This is OK,
                                         // 0x34 has no bits set higher than the 8th
err := w.WriteBitsUnsafe(0x1234&0xff, 8) // &0xff masks out bits higher than the 8th

Or:

err := w.WriteBits(0x1234, 8)            // bits higher than the 8th are ignored here

func (*Writer) WriteBool Uses

func (w *Writer) WriteBool(b bool) (err error)

WriteBool writes one bit: 1 if param is true, 0 otherwise.

func (*Writer) WriteByte Uses

func (w *Writer) WriteByte(b byte) (err error)

WriteByte writes 8 bits.

WriteByte implements io.ByteWriter.

Package bitio imports 2 packages (graph) and is imported by 10 packages. Updated 2019-12-10. Refresh now. Tools for package owners.