cockroach: github.com/cockroachdb/cockroach/pkg/util/ctxgroup Index | Files

package ctxgroup

import "github.com/cockroachdb/cockroach/pkg/util/ctxgroup"

Package ctxgroup wraps golang.org/x/sync/errgroup with a context func.

This package extends and modifies the errgroup API slightly to make context variables more explicit. WithContext no longer returns a context. Instead, the GoCtx method explicitly passes one to the invoked func. The goal is to make misuse of context vars with errgroups more difficult. Example usage:

ctx := context.Background()
g := ctxgroup.WithContext(ctx)
ch := make(chan bool)
g.GoCtx(func(ctx context.Context) error {
	defer close(ch)
	for _, val := range []bool{true, false} {
		select {
		case ch <- val:
		case <-ctx.Done():
			return ctx.Err()
		}
	}
	return nil
})
g.GoCtx(func(ctx context.Context) error {
	for val := range ch {
		if err := api.Call(ctx, val); err != nil {
			return err
		}
	}
	return nil
})
if err := g.Wait(); err != nil {
	return err
}
api.Call(ctx, "done")

Problems with errgroup

The bugs this package attempts to prevent are: misuse of shadowed ctx variables after errgroup closure and confusion in the face of multiple ctx variables when trying to prevent shadowing. The following are all example bugs that Cockroach has had during its use of errgroup:

ctx := context.Background()
g, ctx := errgroup.WithContext(ctx)
ch := make(chan bool)
g.Go(func() error {
	defer close(ch)
	for _, val := range []bool{true, false} {
		select {
		case ch <- val:
		case <-ctx.Done():
			return ctx.Err()
		}
	}
	return nil
})
g.Go(func() error {
	for val := range ch {
		if err := api.Call(ctx, val); err != nil {
			return err
		}
	}
	return nil
})
if err := g.Wait(); err != nil {
	return err
}
api.Call(ctx, "done")

The ctx used by the final api.Call is already closed because the errgroup has returned. This happened because of the desire to not create another ctx variable, and so we shadowed the original ctx var, but then incorrectly continued to use it after the errgroup had closed its context. So we make a modification and create new gCtx variable that doesn't shadow the original ctx:

ctx := context.Background()
g, gCtx := errgroup.WithContext(ctx)
ch := make(chan bool)
g.Go(func() error {
	defer close(ch)
	for _, val := range []bool{true, false} {
		select {
		case ch <- val:
		case <-ctx.Done():
			return ctx.Err()
		}
	}
	return nil
})
g.Go(func() error {
	for val := range ch {
		if err := api.Call(ctx, val); err != nil {
			return err
		}
	}
	return nil
})
if err := g.Wait(); err != nil {
	return err
}
api.Call(ctx, "done")

Now the final api.Call is correct. But the other api.Call is incorrect and the ctx.Done receive is incorrect because they are using the wrong context and thus won't correctly exit early if the errgroup needs to exit early.

Index

Package Files

ctxgroup.go

func GroupWorkers Uses

func GroupWorkers(ctx context.Context, num int, f func(context.Context) error) error

GroupWorkers runs num worker go routines in an errgroup.

type Group Uses

type Group struct {
    // contains filtered or unexported fields
}

Group wraps errgroup.

func WithContext Uses

func WithContext(ctx context.Context) Group

WithContext returns a new Group and an associated Context derived from ctx.

func (Group) Go Uses

func (g Group) Go(f func() error)

Go calls the given function in a new goroutine.

func (Group) GoCtx Uses

func (g Group) GoCtx(f func(ctx context.Context) error)

GoCtx calls the given function in a new goroutine.

func (Group) Wait Uses

func (g Group) Wait() error

Wait blocks until all function calls from the Go method have returned, then returns the first non-nil error (if any) from them. If Wait() is invoked after the context (originally supplied to WithContext) is canceled, Wait returns an error, even if no Go invocation did. In particular, calling Wait() after Done has been closed is guaranteed to return an error.

Package ctxgroup imports 2 packages (graph) and is imported by 51 packages. Updated 2019-07-17. Refresh now. Tools for package owners.