luci: go.chromium.org/luci/common/iotools Index | Files

package iotools

import "go.chromium.org/luci/common/iotools"

Package iotools contains a collection of I/O-related utility structs and methods.

Index

Package Files

bufferingreaderat.go byteslicereader.go chainreader.go countingWriter.go countingreader.go deadlinereader.go doc.go panicwriter.go

Variables

var ErrPanicWriter = errors.New("ErrPanicWriter")

ErrPanicWriter is panic'd from the Writer provided to the callback in WriteTracker in the event of an io error.

var ErrTimeout = errors.New("iotools: timeout")

ErrTimeout is an error returned if the DeadlineReader times out.

func NewBufferingReaderAt Uses

func NewBufferingReaderAt(r io.ReaderAt, blockSize int, lruSize int) io.ReaderAt

NewBufferingReaderAt returns an io.ReaderAt that reads data in blocks of configurable size and keeps LRU of recently read blocks.

It is great for cases when data is read sequentially from an io.ReaderAt, (e.g. when extracting files using zip.Reader), since by setting large block size we can effectively do lookahead reads.

For example, zip.Reader reads data in 4096 byte chunks by default. By setting block size to 512Kb and LRU size to 1 we reduce the number of read operations significantly (128x), in exchange for the modest amount of RAM.

The reader is safe to user concurrently (just like any ReaderAt), but beware that the LRU is shared and all reads from the underlying reader happen under the lock, so multiple goroutines may end up slowing down each other.

func WriteTracker Uses

func WriteTracker(w io.Writer, cb func(io.Writer) error) (n int, err error)

WriteTracker helps to write complex writer routines correctly.

This wraps a Writer with an implementation where any Write method will panic with the ErrPanicWriter error, catch that panic, and return the original io error as well as the number of written bytes.

This means that the callback can use its Writer without tracking the number of bytes written, nor any io errors (i.e. it can ignore the return values from write operations entirely).

If no io errors are encountered, this will return the callback's error and the number of written bytes.

type ByteSliceReader Uses

type ByteSliceReader []byte

ByteSliceReader is an io.Reader and io.ByteReader implementation that reads and mutates an underlying byte slice.

func (*ByteSliceReader) Read Uses

func (r *ByteSliceReader) Read(buf []byte) (int, error)

Read implements io.Reader.

func (*ByteSliceReader) ReadByte Uses

func (r *ByteSliceReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

type ChainReader Uses

type ChainReader []io.Reader

ChainReader is an io.Reader that consumes data sequentially from independent arrays of data to appear as if they were one single concatenated data source.

The underlying io.Reader will be mutated during operation.

func (*ChainReader) Read Uses

func (cr *ChainReader) Read(p []byte) (int, error)

Read implements io.Reader.

func (ChainReader) ReadByte Uses

func (cr ChainReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

func (ChainReader) Remaining Uses

func (cr ChainReader) Remaining() int64

Remaining calculates the amount of data left in the ChainReader. It will panic if an error condition in RemainingErr is encountered.

func (ChainReader) RemainingErr Uses

func (cr ChainReader) RemainingErr() (int64, error)

RemainingErr returns the amount of data left in the ChainReader. An error is returned if any reader in the chain is not either nil or a bytes.Reader.

Note that this method iterates over all readers in the chain each time that it's called.

type CountingReader Uses

type CountingReader struct {
    io.Reader // The underlying io.Reader.

    Count int64
}

CountingReader is an io.Reader that counts the number of bytes that are read.

func (*CountingReader) Read Uses

func (c *CountingReader) Read(buf []byte) (int, error)

Read implements io.Reader.

func (*CountingReader) ReadByte Uses

func (c *CountingReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

type CountingWriter Uses

type CountingWriter struct {
    io.Writer // The underlying io.Writer.

    // Count is the number of bytes that have been written.
    Count int64
    // contains filtered or unexported fields
}

CountingWriter is an io.Writer that counts the number of bytes that are written.

func (*CountingWriter) Write Uses

func (c *CountingWriter) Write(buf []byte) (int, error)

Write implements io.Writer.

func (*CountingWriter) WriteByte Uses

func (c *CountingWriter) WriteByte(b byte) error

WriteByte implements io.ByteWriter.

type DeadlineReader Uses

type DeadlineReader struct {
    net.Conn

    // Deadline is the read deadline to apply prior to each 'Read()'. It can also
    // be set via SetReadTimeout.
    Deadline time.Duration
}

DeadlineReader is a wrapper around a net.Conn that applies an idle timeout deadline to the Conn's Read() method.

func (*DeadlineReader) Read Uses

func (r *DeadlineReader) Read(b []byte) (int, error)

Read implements io.Reader.

func (*DeadlineReader) SetReadTimeout Uses

func (r *DeadlineReader) SetReadTimeout(d time.Duration) error

SetReadTimeout implements ReadDeadlineSetter.

type ReadTimeoutSetter Uses

type ReadTimeoutSetter interface {
    // SetReadTimeout sets the read deadline for subsqeuent reads on this
    // Reader.
    SetReadTimeout(time.Duration) error
}

ReadTimeoutSetter is an interface for an object that can have its read timeout set.

Package iotools imports 7 packages (graph) and is imported by 21 packages. Updated 2018-12-17. Refresh now. Tools for package owners.