README
¶
bounceback
An io.ReadSeeker and io.WriteSeeker that returns to the right place before
reading or writing. Useful when the same file resource is being reused for reads
or writes throughout that file.
list_files
A recursive path walker that supports filters.
seekable_buffer
A memory structure that satisfies io.ReadWriteSeeker.
copy_bytes_between_positions
Given an io.ReadWriteSeeker, copy N bytes from one position to an earlier
position.
read_counter, write_counter
Wrap io.Reader and io.Writer structs in order to report how many bytes were
transferred.
readseekwritecloser
Provides the ReadWriteSeekCloser interface that combines a RWS and a Closer. Also provides a no-op wrapper to augment a plain RWS with a closer.
boundedreadwriteseek
Wraps a ReadWriteSeeker such that no seeks can be at an offset less than a specific-offset.
calculateseek
Provides a reusable function with which to calculate seek offsets.
progress_wrapper
Provides io.Reader and io.Writer wrappers that also trigger callbacks after
each call. The reader wrapper also invokes the callback upon EOF.
does_exist
Check whether a file/directory exists using a file-path.
graceful_copy
Do a copy but correctly handle short-writes and reads that might return a non- zero read count and EOF.
readseeker_to_readerat
A wrapper that allows an io.ReadSeeker to be used as a io.ReaderAt.
simplefileinfo
An implementation of os.FileInfo to support testing.
Documentation
¶
Index ¶
- Variables
- func CalculateSeek(currentOffset int64, delta int64, whence int, fileSize int64) (finalOffset int64, err error)
- func CopyBytesBetweenPositions(rws io.ReadWriteSeeker, fromPosition, toPosition int64, count int) (n int, err error)
- func DoesExist(filepath string) bool
- func GetOffset(s io.Seeker) int64
- func GracefulCopy(w io.Writer, r io.Reader, buffer []byte) (copyCount int, err error)
- func ListFiles(rootPath string, cb FileListFilterPredicate) (filesC chan VisitedFile, count int, errC chan error)
- func NewReadProgressWrapper(r io.Reader, progressCb ProgressFunc) io.Reader
- func NewWriteProgressWrapper(w io.Writer, progressCb ProgressFunc) io.Writer
- type BouncebackReader
- func (bb *BouncebackReader) Position() int64
- func (br *BouncebackReader) Read(p []byte) (n int, err error)
- func (br *BouncebackReader) Seek(offset int64, whence int) (newPosition int64, err error)
- func (bb *BouncebackReader) StatsReads() int
- func (bb *BouncebackReader) StatsSeeks() int
- func (bb *BouncebackReader) StatsSyncs() int
- func (bb *BouncebackReader) StatsWrites() int
- type BouncebackStats
- type BouncebackWriter
- func (bb *BouncebackWriter) Position() int64
- func (bw *BouncebackWriter) Seek(offset int64, whence int) (newPosition int64, err error)
- func (bb *BouncebackWriter) StatsReads() int
- func (bb *BouncebackWriter) StatsSeeks() int
- func (bb *BouncebackWriter) StatsSyncs() int
- func (bb *BouncebackWriter) StatsWrites() int
- func (bw *BouncebackWriter) Write(p []byte) (n int, err error)
- type BoundedReadWriteSeekCloser
- func (rwsc *BoundedReadWriteSeekCloser) Close() (err error)
- func (rwsc *BoundedReadWriteSeekCloser) Read(buffer []byte) (readCount int, err error)
- func (rwsc *BoundedReadWriteSeekCloser) Seek(offset int64, whence int) (newOffset int64, err error)
- func (rwsc *BoundedReadWriteSeekCloser) Write(buffer []byte) (writtenCount int, err error)
- type BoundedReadWriteSeeker
- func (brws *BoundedReadWriteSeeker) MinimumOffset() int64
- func (brws *BoundedReadWriteSeeker) Read(buffer []byte) (readCount int, err error)
- func (brws *BoundedReadWriteSeeker) Seek(offset int64, whence int) (updatedOffset int64, err error)
- func (brws *BoundedReadWriteSeeker) Write(buffer []byte) (writtenCount int, err error)
- type FileListFilterPredicate
- type ProgressFunc
- type ReadCounter
- type ReadProgressWrapper
- type ReadSeekerToReaderAt
- type ReadWriteSeekCloser
- type SeekType
- type SeekableBuffer
- func (sb *SeekableBuffer) Bytes() []byte
- func (sb *SeekableBuffer) Len() int
- func (sb *SeekableBuffer) Read(p []byte) (n int, err error)
- func (sb *SeekableBuffer) Seek(offset int64, whence int) (n int64, err error)
- func (sb *SeekableBuffer) Truncate(size int64) (err error)
- func (sb *SeekableBuffer) Write(p []byte) (n int, err error)
- type SimpleFileInfo
- type VisitedFile
- type WriteCounter
- type WriteProgressWrapper
Constants ¶
This section is empty.
Variables ¶
var ( // ErrSeekBeyondBound is returned when a seek is requested beyond the // statically-given file-size. No writes or seeks beyond boundaries are // supported with a statically-given file size. ErrSeekBeyondBound = errors.New("seek beyond boundary") )
Functions ¶
func CalculateSeek ¶
func CalculateSeek(currentOffset int64, delta int64, whence int, fileSize int64) (finalOffset int64, err error)
CalculateSeek calculates an offset in a file-stream given the parameters.
func CopyBytesBetweenPositions ¶
func CopyBytesBetweenPositions(rws io.ReadWriteSeeker, fromPosition, toPosition int64, count int) (n int, err error)
CopyBytesBetweenPositions will copy bytes from one position in the given RWS to an earlier position in the same RWS.
func DoesExist ¶
DoesExist returns true if we can open the given file/path without error. We can't simply use `os.IsNotExist()` because we'll get a different error when the parent directory doesn't exist, and really the only important thing is if it exists *and* it's readable.
func GetOffset ¶
GetOffset returns the current offset of the Seeker and just panics if unable to find it.
func GracefulCopy ¶
GracefulCopy willcopy while enduring lesser normal issues.
- We'll ignore EOF if the read byte-count is more than zero. Only an EOF when zero bytes were read will terminate the loop.
- Ignore short-writes. If less bytes were written than the bytes that were given, we'll keep trying until done.
func ListFiles ¶
func ListFiles(rootPath string, cb FileListFilterPredicate) (filesC chan VisitedFile, count int, errC chan error)
ListFiles feeds a continuous list of files from a recursive folder scan. An optional predicate can be provided in order to filter. When done, the `filesC` channel is closed. If there's an error, the `errC` channel will receive it.
func NewReadProgressWrapper ¶
func NewReadProgressWrapper(r io.Reader, progressCb ProgressFunc) io.Reader
NewReadProgressWrapper returns a new RPW instance.
func NewWriteProgressWrapper ¶
func NewWriteProgressWrapper(w io.Writer, progressCb ProgressFunc) io.Writer
NewWriteProgressWrapper returns a new WPW instance.
Types ¶
type BouncebackReader ¶
type BouncebackReader struct {
// contains filtered or unexported fields
}
BouncebackReader wraps a ReadSeeker, keeps track of our position, and seeks back to it before writing. This allows an underlying ReadWriteSeeker with an unstable position can still be used for a prolonged series of writes.
func NewBouncebackReader ¶
func NewBouncebackReader(rs io.ReadSeeker) (br *BouncebackReader, err error)
NewBouncebackReader returns a `*BouncebackReader` struct.
func (*BouncebackReader) Position ¶
func (bb *BouncebackReader) Position() int64
Position returns the position that we're supposed to be at.
func (*BouncebackReader) Read ¶
func (br *BouncebackReader) Read(p []byte) (n int, err error)
Seek does a standard read.
func (*BouncebackReader) Seek ¶
func (br *BouncebackReader) Seek(offset int64, whence int) (newPosition int64, err error)
Seek does a seek to an arbitrary place in the `io.ReadSeeker`.
func (*BouncebackReader) StatsReads ¶
func (bb *BouncebackReader) StatsReads() int
StatsReads returns the number of reads that have been attempted.
func (*BouncebackReader) StatsSeeks ¶
func (bb *BouncebackReader) StatsSeeks() int
StatsSeeks returns the number of seeks.
func (*BouncebackReader) StatsSyncs ¶
func (bb *BouncebackReader) StatsSyncs() int
StatsSyncs returns the number of corrective seeks ("bounce-backs").
func (*BouncebackReader) StatsWrites ¶
func (bb *BouncebackReader) StatsWrites() int
StatsWrites returns the number of write operations.
type BouncebackStats ¶
type BouncebackStats struct {
// contains filtered or unexported fields
}
BouncebackStats describes operation counts.
func (BouncebackStats) String ¶
func (bbs BouncebackStats) String() string
type BouncebackWriter ¶
type BouncebackWriter struct {
// contains filtered or unexported fields
}
BouncebackWriter wraps a WriteSeeker, keeps track of our position, and seeks back to it before writing. This allows an underlying ReadWriteSeeker with an unstable position can still be used for a prolonged series of writes.
func NewBouncebackWriter ¶
func NewBouncebackWriter(ws io.WriteSeeker) (bw *BouncebackWriter, err error)
NewBouncebackWriter returns a new `BouncebackWriter` struct.
func (*BouncebackWriter) Position ¶
func (bb *BouncebackWriter) Position() int64
Position returns the position that we're supposed to be at.
func (*BouncebackWriter) Seek ¶
func (bw *BouncebackWriter) Seek(offset int64, whence int) (newPosition int64, err error)
Seek puts us at a specific position in the internal writer for the next write/seek.
func (*BouncebackWriter) StatsReads ¶
func (bb *BouncebackWriter) StatsReads() int
StatsReads returns the number of reads that have been attempted.
func (*BouncebackWriter) StatsSeeks ¶
func (bb *BouncebackWriter) StatsSeeks() int
StatsSeeks returns the number of seeks.
func (*BouncebackWriter) StatsSyncs ¶
func (bb *BouncebackWriter) StatsSyncs() int
StatsSyncs returns the number of corrective seeks ("bounce-backs").
func (*BouncebackWriter) StatsWrites ¶
func (bb *BouncebackWriter) StatsWrites() int
StatsWrites returns the number of write operations.
type BoundedReadWriteSeekCloser ¶
type BoundedReadWriteSeekCloser struct {
io.Closer
*BoundedReadWriteSeeker
}
BoundedReadWriteSeekCloser wraps a RWS that is also a closer with boundaries. This proxies the RWS methods to the inner BRWS inside.
func NewBoundedReadWriteSeekCloser ¶
func NewBoundedReadWriteSeekCloser(rwsc ReadWriteSeekCloser, minimumOffset int64, staticFileSize int64) (brwsc *BoundedReadWriteSeekCloser, err error)
NewBoundedReadWriteSeekCloser returns a new BoundedReadWriteSeekCloser.
func (*BoundedReadWriteSeekCloser) Close ¶
func (rwsc *BoundedReadWriteSeekCloser) Close() (err error)
Close forwards calls to the inner RWS.
func (*BoundedReadWriteSeekCloser) Read ¶
func (rwsc *BoundedReadWriteSeekCloser) Read(buffer []byte) (readCount int, err error)
Read forwards calls to the inner RWS.
type BoundedReadWriteSeeker ¶
type BoundedReadWriteSeeker struct {
io.ReadWriteSeeker
// contains filtered or unexported fields
}
BoundedReadWriteSeeker is a thin filter that ensures that no seeks can be done to offsets smaller than the one we were given. This supports libraries that might be expecting to read from the front of the stream being used on data that is in the middle of a stream instead.
func NewBoundedReadWriteSeeker ¶
func NewBoundedReadWriteSeeker(rws io.ReadWriteSeeker, minimumOffset int64, staticFileSize int64) (brws *BoundedReadWriteSeeker, err error)
NewBoundedReadWriteSeeker returns a new BoundedReadWriteSeeker instance.
func (*BoundedReadWriteSeeker) MinimumOffset ¶
func (brws *BoundedReadWriteSeeker) MinimumOffset() int64
MinimumOffset returns the configured minimum-offset.
func (*BoundedReadWriteSeeker) Read ¶
func (brws *BoundedReadWriteSeeker) Read(buffer []byte) (readCount int, err error)
Read forwards writes to the inner RWS.
type FileListFilterPredicate ¶
FileListFilterPredicate is the callback predicate used for filtering.
type ProgressFunc ¶
ProgressFunc receives progress updates.
type ReadCounter ¶
type ReadCounter struct {
// contains filtered or unexported fields
}
ReadCounter proxies read requests and maintains a counter of bytes read.
func NewReadCounter ¶
func NewReadCounter(r io.Reader) *ReadCounter
NewReadCounter returns a new `ReadCounter` struct wrapping a `Reader`.
func (*ReadCounter) Count ¶
func (rc *ReadCounter) Count() int
Count returns the total number of bytes read.
type ReadProgressWrapper ¶
type ReadProgressWrapper struct {
// contains filtered or unexported fields
}
ReadProgressWrapper wraps a reader and calls a callback after each read with count and duration info.
type ReadSeekerToReaderAt ¶
type ReadSeekerToReaderAt struct {
// contains filtered or unexported fields
}
ReadSeekerToReaderAt is a wrapper that allows a ReadSeeker to masquerade as a ReaderAt.
func NewReadSeekerToReaderAt ¶
func NewReadSeekerToReaderAt(rs io.ReadSeeker) *ReadSeekerToReaderAt
NewReadSeekerToReaderAt returns a new ReadSeekerToReaderAt instance.
func (*ReadSeekerToReaderAt) ReadAt ¶
func (rstra *ReadSeekerToReaderAt) ReadAt(p []byte, offset int64) (n int, err error)
ReadAt is a wrapper that satisfies the ReaderAt interface.
Note that a requirement of ReadAt is that it doesn't have an effect on the offset in the underlying resource as well as that concurrent calls can be made to it. Since we're capturing the current offset in the underlying resource and then seeking back to it before returning, it is the responsibility of the caller to serialize (i.e. use a mutex with) these requests in order to eliminate race-conditions in the parallel-usage scenario.
Note also that, since ReadAt() is going to be called on a particular instance, that instance is going to internalize a file resource, that file- resource is provided by the OS, and [most] OSs are only gonna support one file-position per resource, locking is already going to be a necessary internal semantic of a ReaderAt implementation.
type ReadWriteSeekCloser ¶
type ReadWriteSeekCloser interface {
io.ReadWriteSeeker
io.Closer
}
ReadWriteSeekCloser satisfies `io.ReadWriteSeeker` and `io.Closer` interfaces.
func ReadWriteSeekNoopCloser ¶
func ReadWriteSeekNoopCloser(rws io.ReadWriteSeeker) ReadWriteSeekCloser
ReadWriteSeekNoopCloser wraps a `io.ReadWriteSeeker` with a no-op Close() call.
type SeekType ¶
type SeekType int
SeekType is a convenience type to associate the different seek-types with printable descriptions.
type SeekableBuffer ¶
type SeekableBuffer struct {
// contains filtered or unexported fields
}
SeekableBuffer is a simple memory structure that satisfies `io.ReadWriteSeeker`.
func NewSeekableBuffer ¶
func NewSeekableBuffer() *SeekableBuffer
NewSeekableBuffer is a factory that returns a `*SeekableBuffer`.
func NewSeekableBufferWithBytes ¶
func NewSeekableBufferWithBytes(originalData []byte) *SeekableBuffer
NewSeekableBufferWithBytes is a factory that returns a `*SeekableBuffer`.
func (*SeekableBuffer) Bytes ¶
func (sb *SeekableBuffer) Bytes() []byte
Bytes returns the underlying slice.
func (*SeekableBuffer) Len ¶
func (sb *SeekableBuffer) Len() int
Len returns the number of bytes currently stored.
func (*SeekableBuffer) Read ¶
func (sb *SeekableBuffer) Read(p []byte) (n int, err error)
Read does a standard read against the internal slice.
func (*SeekableBuffer) Seek ¶
func (sb *SeekableBuffer) Seek(offset int64, whence int) (n int64, err error)
Seek does a standard seek on the internal slice.
func (*SeekableBuffer) Truncate ¶
func (sb *SeekableBuffer) Truncate(size int64) (err error)
Truncate either chops or extends the internal buffer.
type SimpleFileInfo ¶
type SimpleFileInfo struct {
// contains filtered or unexported fields
}
SimpleFileInfo is a simple `os.FileInfo` implementation useful for testing with the bare minimum.
func NewSimpleFileInfoWithDirectory ¶
func NewSimpleFileInfoWithDirectory(filename string, modTime time.Time) *SimpleFileInfo
NewSimpleFileInfoWithDirectory returns a new directory-specific SimpleFileInfo.
func NewSimpleFileInfoWithFile ¶
func NewSimpleFileInfoWithFile(filename string, size int64, mode os.FileMode, modTime time.Time) *SimpleFileInfo
NewSimpleFileInfoWithFile returns a new file-specific SimpleFileInfo.
func (*SimpleFileInfo) IsDir ¶
func (sfi *SimpleFileInfo) IsDir() bool
IsDir returns true if a directory.
func (*SimpleFileInfo) ModTime ¶
func (sfi *SimpleFileInfo) ModTime() time.Time
ModTime returns the modification time.
func (*SimpleFileInfo) Mode ¶
func (sfi *SimpleFileInfo) Mode() os.FileMode
Mode returns the file mode bits.
func (*SimpleFileInfo) Name ¶
func (sfi *SimpleFileInfo) Name() string
Name returns the base name of the file.
func (*SimpleFileInfo) Size ¶
func (sfi *SimpleFileInfo) Size() int64
Size returns the length in bytes for regular files; system-dependent for others.
func (*SimpleFileInfo) Sys ¶
func (sfi *SimpleFileInfo) Sys() interface{}
Sys returns internal state.
type VisitedFile ¶
VisitedFile is one visited file.
type WriteCounter ¶
type WriteCounter struct {
// contains filtered or unexported fields
}
WriteCounter proxies write requests and maintains a counter of bytes written.
func NewWriteCounter ¶
func NewWriteCounter(w io.Writer) *WriteCounter
NewWriteCounter returns a new `WriteCounter` struct wrapping a `Writer`.
func (*WriteCounter) Count ¶
func (wc *WriteCounter) Count() int
Count returns the total number of bytes read.
type WriteProgressWrapper ¶
type WriteProgressWrapper struct {
// contains filtered or unexported fields
}
WriteProgressWrapper wraps a reader and calls a callback after each read with count and duration info.
Source Files
¶
- bounceback.go
- boundedreadwriteseekcloser.go
- boundedreadwriteseeker.go
- calculate_seek.go
- common.go
- copy_bytes_between_positions.go
- does_exist.go
- graceful_copy.go
- list_files.go
- progress_wrapper.go
- read_counter.go
- readseeker_to_readerat.go
- readwriteseekcloser.go
- seekable_buffer.go
- simplefileinfo.go
- utility.go
- write_counter.go