Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Deduplicate ¶ added in v0.2.0
func Deduplicate(in <-chan []byte, window time.Duration, ticker func(time.Duration) *time.Ticker, out chan<- []byte)
Deduplicate and order records within the given time window. A smaller window may cause duplicate or out-of-order messages. A larger window will cause higher end-to-end latency. The ticker is used every window / 10 to flush the buffer. The function returns when the `in` chan is closed.
func Execute ¶
func Execute( ctx context.Context, pf PeerFactory, rcf ReadCloserFactory, sleep func(time.Duration), ticker func(time.Duration) *time.Ticker, sink chan<- []byte, ) error
Execute creates and maintains streams of records to multiple peers. It muxes the streams of incoming records to the sink chan of records. It's designed to be invoked once per user stream request.
The sleep func is used to backoff between retries of a single peer. The ticker func is used to regularly resolve peers.
Types ¶
type PeerFactory ¶
type PeerFactory func() []string
PeerFactory should return the current set of peer addresses. Each address will be converted to an io.Reader via the ReadCloserFactory. The PeerFactory is periodically invoked to get the latest set of peers.
type ReadCloserFactory ¶ added in v0.2.0
ReadCloserFactory converts a peer address to an io.ReadCloser. ReadClosers must exit with context.Canceled when the context is canceled. Other errors will cause the managing goroutine to remanufacture.
func HTTPReadCloserFactory ¶ added in v0.2.0
func HTTPReadCloserFactory(client Doer, addr2url func(string) string) ReadCloserFactory
HTTPReadCloserFactory returns a ReadCloserFactory that converts the addr to a URL via the addr2url function, makes a GET request via the client, and returns the response body as the ReadCloser.
Source Files