luci: go.chromium.org/luci/common/iotools Index | Files

package iotools

import "go.chromium.org/luci/common/iotools"

Package iotools contains a collection of I/O-related utility structs and methods.

Index

Package Files

bufferingreaderat.go byteslicereader.go chainreader.go countingWriter.go countingreader.go doc.go panicwriter.go responsewriter.go

Variables

var ErrPanicWriter = errors.New("ErrPanicWriter")

ErrPanicWriter is panic'd from the Writer provided to the callback in WriteTracker in the event of an io error.

func NewBufferingReaderAt Uses

func NewBufferingReaderAt(r io.ReaderAt, blockSize int, lruSize int) io.ReaderAt

NewBufferingReaderAt returns an io.ReaderAt that reads data in blocks of configurable size and keeps LRU of recently read blocks.

It is great for cases when data is read sequentially from an io.ReaderAt, (e.g. when extracting files using zip.Reader), since by setting large block size we can effectively do lookahead reads.

For example, zip.Reader reads data in 4096 byte chunks by default. By setting block size to 512Kb and LRU size to 1 we reduce the number of read operations significantly (128x), in exchange for the modest amount of RAM.

The reader is safe to user concurrently (just like any ReaderAt), but beware that the LRU is shared and all reads from the underlying reader happen under the lock, so multiple goroutines may end up slowing down each other.

func WriteTracker Uses

func WriteTracker(w io.Writer, cb func(io.Writer) error) (n int, err error)

WriteTracker helps to write complex writer routines correctly.

This wraps a Writer with an implementation where any Write method will panic with the ErrPanicWriter error, catch that panic, and return the original io error as well as the number of written bytes.

This means that the callback can use its Writer without tracking the number of bytes written, nor any io errors (i.e. it can ignore the return values from write operations entirely).

If no io errors are encountered, this will return the callback's error and the number of written bytes.

type ByteSliceReader Uses

type ByteSliceReader []byte

ByteSliceReader is an io.Reader and io.ByteReader implementation that reads and mutates an underlying byte slice.

func (*ByteSliceReader) Read Uses

func (r *ByteSliceReader) Read(buf []byte) (int, error)

Read implements io.Reader.

func (*ByteSliceReader) ReadByte Uses

func (r *ByteSliceReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

type ChainReader Uses

type ChainReader []io.Reader

ChainReader is an io.Reader that consumes data sequentially from independent arrays of data to appear as if they were one single concatenated data source.

The underlying io.Reader will be mutated during operation.

func (*ChainReader) Read Uses

func (cr *ChainReader) Read(p []byte) (int, error)

Read implements io.Reader.

func (ChainReader) ReadByte Uses

func (cr ChainReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

func (ChainReader) Remaining Uses

func (cr ChainReader) Remaining() int64

Remaining calculates the amount of data left in the ChainReader. It will panic if an error condition in RemainingErr is encountered.

func (ChainReader) RemainingErr Uses

func (cr ChainReader) RemainingErr() (int64, error)

RemainingErr returns the amount of data left in the ChainReader. An error is returned if any reader in the chain is not either nil or a bytes.Reader.

Note that this method iterates over all readers in the chain each time that it's called.

type CountingReader Uses

type CountingReader struct {
    io.Reader // The underlying io.Reader.

    Count int64
}

CountingReader is an io.Reader that counts the number of bytes that are read.

func (*CountingReader) Read Uses

func (c *CountingReader) Read(buf []byte) (int, error)

Read implements io.Reader.

func (*CountingReader) ReadByte Uses

func (c *CountingReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

type CountingWriter Uses

type CountingWriter struct {
    io.Writer // The underlying io.Writer.

    // Count is the number of bytes that have been written.
    Count int64
    // contains filtered or unexported fields
}

CountingWriter is an io.Writer that counts the number of bytes that are written.

func (*CountingWriter) Write Uses

func (c *CountingWriter) Write(buf []byte) (int, error)

Write implements io.Writer.

func (*CountingWriter) WriteByte Uses

func (c *CountingWriter) WriteByte(b byte) error

WriteByte implements io.ByteWriter.

type ResponseWriter Uses

type ResponseWriter struct {
    // contains filtered or unexported fields
}

ResponseWriter wraps a given http.ResponseWriter, records its status code and response size.

Assumes all writes are externally synchronized.

func NewResponseWriter Uses

func NewResponseWriter(rw http.ResponseWriter) *ResponseWriter

NewResponseWriter constructs a ResponseWriter that wraps given 'rw' and tracks how much data was written to it and what status code was set.

func (*ResponseWriter) Flush Uses

func (rw *ResponseWriter) Flush()

Flush sends any buffered data to the client.

func (*ResponseWriter) Header Uses

func (rw *ResponseWriter) Header() http.Header

Header returns the header map that will be sent by WriteHeader.

func (*ResponseWriter) ResponseSize Uses

func (rw *ResponseWriter) ResponseSize() int64

ResponseSize is size of the response body written so far.

func (*ResponseWriter) Status Uses

func (rw *ResponseWriter) Status() int

Status is the HTTP status code set in the response.

func (*ResponseWriter) Write Uses

func (rw *ResponseWriter) Write(buf []byte) (int, error)

Write writes the data to the connection as part of an HTTP reply.

func (*ResponseWriter) WriteHeader Uses

func (rw *ResponseWriter) WriteHeader(code int)

WriteHeader sends an HTTP response header with the provided status code.

Package iotools imports 6 packages (graph) and is imported by 17 packages. Updated 2019-12-05. Refresh now. Tools for package owners.