README
¶
Dynamic and thread safe byte buffer
BuffReader reads all available data from the input io.Reader and writes it in to the internal buffer asynchronously. BuffReader implements io.Reader interface for reading data from the internal buffer.
- Dynamic buffer size
- Thread safe buffer reading
- Cancellabe buffering routine
- Buffering completion notification
When to use buffered reader?
- Slow data producer (e.g. network): you can start receiving data before you do something with it
- Slow data consumer (e.g. heavy data processing): producer is not going be blocked
- Both consumer and producer slow: data transfer and processing will be faster because consumer and producer will not wait each other
Installation
Simple install the package to your $GOPATH with the go tool from shell:
$ go get gitlab.com/jonas.jasas/buffreader
Make sure Git is installed on your machine and in your system's PATH.
Exaples
Basic
Example show how to create buffered reader br from myReader (io.Reader)
// Creating buffered reader br
br := buffreader.New(myReader)
br.Buff() // Start buffering
// At this point all data from myReader is getting red asynchronously in to the internal buffer.
// Slow operations can be done here not warring about blocking myReader.
// Read data from the buffered reader br
myConsumer(br)
Buffering completion notification
Example shows how to take an action when buffering is finished or error occurred while buffering.
// Creating buffered reader br and channel resChan that will notify when buffering is done
br := buffreader.New(myReader)
go func() {
res := <-br.Buff() // Blocking until buffering is finished
if res.Err == nil {
// Do something knowing that myReader is no longer needed (e.g. close network connection)
} else {
// In case of error buffer will be freed immediately
log.Printf("Buffering failed with the error: %v", res.Err)
}
}()
// Read() method will return same error as in res.Err
myConsumer(br)
Full examples can be found in examples directory.
Buffer size
As you probably already noticed there is no buffer size limit parameter. All memory that is available to the process will be used for buffering. Read method and resChan returns error buffer.ErrTooLarge if it is not possible to allocate memory for the buffer. Please make sure that in the worse case scenario all data is going to fit in memory. Buffer limiting feature will be implemented in feature versions.
Documentation
¶
Overview ¶
BuffReader is a Go package that implements thread safe and dynamically resized buffer for io.Reader. BuffReader increases data transfer performance between reader and writer. It makes sure that reader (e.g. network connection) never waits slow writer (e.g. slow data processing) by buffering received data in to memory.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrClosedBuffReader = errors.New("BuffReader is closed")
Functions ¶
This section is empty.
Types ¶
type BuffReader ¶
type BuffReader struct {
// contains filtered or unexported fields
}
func New ¶
func New(src io.Reader) *BuffReader
New function creates new BuffReader src - reader that will be red and data will be stored in to the internal buffer.
func (*BuffReader) Buff ¶
func (br *BuffReader) Buff() <-chan BuffRes
Buff method starts buffering in subroutine (non-blocking). Returns channel that will return buffering result when buffering is finished. Method starts buffering only the first time it is called, so can be used to get channel multiple times. If method is called after buffering is done or Close() is called it returns closed channel.
<-chan BuffRes - returns result of reading from input reader src size - is the number of bytes that were red from input reader. err - error that occurred during reading from input reader and writing to internal buffer. Err values: * there was no error err == nil * is closed err == ErrClosedBuffReader * unable to allocate memory for the buffer err == bytes.ErrTooLarge
func (*BuffReader) Close ¶
func (br *BuffReader) Close() (err error)
Close stops loading buffer and releasing memory. Source reader stays open. Returns ErrClosedBuffReader error if Close() is called more than once.
Notes ¶
Bugs ¶
If BuffReader is closed while waiting on reading source io.Reader buffering subroutine will not exit. To avoid hanging subroutine close source io.Reader.
Source Files
Directories