iotools

package
v0.0.0-...-a0a3655 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 28, 2019 License: Apache-2.0 Imports: 8 Imported by: 0

Documentation

Overview

Package iotools contains a collection of I/O-related utility structs and methods.

Index

Constants

This section is empty.

Variables

View Source
var ErrPanicWriter = errors.New("ErrPanicWriter")

ErrPanicWriter is panic'd from the Writer provided to the callback in WriteTracker in the event of an io error.

View Source
var ErrTimeout = errors.New("iotools: timeout")

ErrTimeout is an error returned if the DeadlineReader times out.

Functions

func NewBufferingReaderAt

func NewBufferingReaderAt(r io.ReaderAt, blockSize int, lruSize int) io.ReaderAt

NewBufferingReaderAt returns an io.ReaderAt that reads data in blocks of configurable size and keeps LRU of recently read blocks.

It is great for cases when data is read sequentially from an io.ReaderAt, (e.g. when extracting files using zip.Reader), since by setting large block size we can effectively do lookahead reads.

For example, zip.Reader reads data in 4096 byte chunks by default. By setting block size to 512Kb and LRU size to 1 we reduce the number of read operations significantly (128x), in exchange for the modest amount of RAM.

The reader is safe to user concurrently (just like any ReaderAt), but beware that the LRU is shared and all reads from the underlying reader happen under the lock, so multiple goroutines may end up slowing down each other.

func WriteTracker

func WriteTracker(w io.Writer, cb func(io.Writer) error) (n int, err error)

WriteTracker helps to write complex writer routines correctly.

This wraps a Writer with an implementation where any Write method will panic with the ErrPanicWriter error, catch that panic, and return the original io error as well as the number of written bytes.

This means that the callback can use its Writer without tracking the number of bytes written, nor any io errors (i.e. it can ignore the return values from write operations entirely).

If no io errors are encountered, this will return the callback's error and the number of written bytes.

Types

type ByteSliceReader

type ByteSliceReader []byte

ByteSliceReader is an io.Reader and io.ByteReader implementation that reads and mutates an underlying byte slice.

func (*ByteSliceReader) Read

func (r *ByteSliceReader) Read(buf []byte) (int, error)

Read implements io.Reader.

func (*ByteSliceReader) ReadByte

func (r *ByteSliceReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

type ChainReader

type ChainReader []io.Reader

ChainReader is an io.Reader that consumes data sequentially from independent arrays of data to appear as if they were one single concatenated data source.

The underlying io.Reader will be mutated during operation.

func (*ChainReader) Read

func (cr *ChainReader) Read(p []byte) (int, error)

Read implements io.Reader.

func (ChainReader) ReadByte

func (cr ChainReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

func (ChainReader) Remaining

func (cr ChainReader) Remaining() int64

Remaining calculates the amount of data left in the ChainReader. It will panic if an error condition in RemainingErr is encountered.

func (ChainReader) RemainingErr

func (cr ChainReader) RemainingErr() (int64, error)

RemainingErr returns the amount of data left in the ChainReader. An error is returned if any reader in the chain is not either nil or a bytes.Reader.

Note that this method iterates over all readers in the chain each time that it's called.

type CountingReader

type CountingReader struct {
	io.Reader // The underlying io.Reader.

	Count int64
}

CountingReader is an io.Reader that counts the number of bytes that are read.

func (*CountingReader) Read

func (c *CountingReader) Read(buf []byte) (int, error)

Read implements io.Reader.

func (*CountingReader) ReadByte

func (c *CountingReader) ReadByte() (byte, error)

ReadByte implements io.ByteReader.

type CountingWriter

type CountingWriter struct {
	io.Writer // The underlying io.Writer.

	// Count is the number of bytes that have been written.
	Count int64
	// contains filtered or unexported fields
}

CountingWriter is an io.Writer that counts the number of bytes that are written.

func (*CountingWriter) Write

func (c *CountingWriter) Write(buf []byte) (int, error)

Write implements io.Writer.

func (*CountingWriter) WriteByte

func (c *CountingWriter) WriteByte(b byte) error

WriteByte implements io.ByteWriter.

type DeadlineReader

type DeadlineReader struct {
	net.Conn

	// Deadline is the read deadline to apply prior to each 'Read()'. It can also
	// be set via SetReadTimeout.
	Deadline time.Duration
}

DeadlineReader is a wrapper around a net.Conn that applies an idle timeout deadline to the Conn's Read() method.

func (*DeadlineReader) Read

func (r *DeadlineReader) Read(b []byte) (int, error)

Read implements io.Reader.

func (*DeadlineReader) SetReadTimeout

func (r *DeadlineReader) SetReadTimeout(d time.Duration) error

SetReadTimeout implements ReadDeadlineSetter.

type ReadTimeoutSetter

type ReadTimeoutSetter interface {
	// SetReadTimeout sets the read deadline for subsqeuent reads on this
	// Reader.
	SetReadTimeout(time.Duration) error
}

ReadTimeoutSetter is an interface for an object that can have its read timeout set.

type ResponseWriter

type ResponseWriter struct {
	// contains filtered or unexported fields
}

ResponseWriter wraps a given http.ResponseWriter, records its status code and response size.

Assumes all writes are externally synchronized.

func NewResponseWriter

func NewResponseWriter(rw http.ResponseWriter) *ResponseWriter

NewResponseWriter constructs a ResponseWriter that wraps given 'rw' and tracks how much data was written to it and what status code was set.

func (*ResponseWriter) Flush

func (rw *ResponseWriter) Flush()

Flush sends any buffered data to the client.

func (*ResponseWriter) Header

func (rw *ResponseWriter) Header() http.Header

Header returns the header map that will be sent by WriteHeader.

func (*ResponseWriter) ResponseSize

func (rw *ResponseWriter) ResponseSize() int64

ResponseSize is size of the response body written so far.

func (*ResponseWriter) Status

func (rw *ResponseWriter) Status() int

Status is the HTTP status code set in the response.

func (*ResponseWriter) Write

func (rw *ResponseWriter) Write(buf []byte) (int, error)

Write writes the data to the connection as part of an HTTP reply.

func (*ResponseWriter) WriteHeader

func (rw *ResponseWriter) WriteHeader(code int)

WriteHeader sends an HTTP response header with the provided status code.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL