README
¶
readgroup
Have you ever wanted to process the exact same io.Reader data concurrently? This is library attempts to paint over the synchronization footguns in doing so.
A canonical example is transcoding a video that is being uploaded to your server into multiple formats. You don't wait for the whole video to be uploaded and have to keep the whole thing in memory before you can read over the entire set of bytes multiple times. Instead, you want to fire off multiple goroutines that all get to read the same data as it comes in and block until they're all finished or one fails.
This library was heavily by errgroup and mimics its API.
Documentation
¶
Overview ¶
Package readgroup provides synchronization for reading the same io.Reader across N goroutines concurrently.
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ReadGroup ¶
type ReadGroup struct {
// contains filtered or unexported fields
}
A ReadGroup is a collection of goroutines consuming the same io.Reader concurrently.
Example (SimpleUsage) ¶
SimpleUsage illustrates a very basic use case where you need to make transformations to the same io.Reader, but it's too large for working memory.
Instead of a large streaming file, a small string is used.
package main
import (
"bytes"
"errors"
"fmt"
"io"
"io/ioutil"
"os"
"strings"
"github.com/jzelinskie/readgroup"
)
func main() {
buf := bytes.NewBufferString("My Huge Streaming Video Reader")
rg := readgroup.NewReadGroup(buf)
rg.Go(func(r io.Reader) error {
_, err := io.Copy(os.Stdout, r)
io.WriteString(os.Stdout, "\n")
return err
})
rg.Go(func(r io.Reader) error {
hellostr, err := ioutil.ReadAll(r)
if err != nil {
return err
}
fmt.Println(strings.ToLower(string(hellostr)))
return nil
})
rg.Go(func(r io.Reader) error {
return errors.New("error that could short-circuit the go routines")
})
err := rg.Wait()
if err != nil {
fmt.Printf("\n%s", err.Error())
}
}
Output:
func NewReadGroup ¶
NewReadGroup specifies the Reader to be consumed.
Source Files