Documentation
¶
Overview ¶
Package defines Sink which is a type of a task. A Sink is used as a terminal point of a data pipeline.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ElementWiseFn ¶
A function that defines the behavior of an elementwise operator.
type ElementWiseOp ¶
type ElementWiseOp[S any] struct { // contains filtered or unexported fields }
A Sink task that processes elements fed by its upstream task one by one.
func ElementWise ¶
func ElementWise[S any](fn ElementWiseFn[S]) *ElementWiseOp[S]
Create an elementwise operator from an ElementWiseFn.
func (*ElementWiseOp[S]) Concurrent ¶
func (op *ElementWiseOp[S]) Concurrent(concurrency int) Sink[S]
Create a concurrent Sink from multiple elementwise operators that have the same behavior.
type ExposeOp ¶
type ExposeOp[S any] struct { // contains filtered or unexported fields }
A Sink task that exposes elements fed by its upstream task via a gRPC service.
func Expose ¶
Create an expose operator that runs a gRPC service.
The gRPC service is bounded to the passed listener, and each message is encoded as defined with the passed marshal spec.
Use source.Pull to receive elements exposed via this operator. Or, it is also possible to consume elements exposed by directly communicating with the gRPC service. For more details on how to communicate with the service, please see the Communicator service definition.
type Sink ¶
A task that is used as a terminal point of a data pipeline.
A Sink receives elements fed by an upstream task, and consumes them, for example, print them into STDOUT or writing them into a file.
The output channel needs to be closed without sending any elements.
func Concurrent ¶
Create a Sink from multiple Sinks. The passed Sinks will run concurrently.
func ConcurrentFromFn ¶
Create a Sink to run the provided SinkFn concurrently. This is a shorthand to create a concurrent Sink from Sinks with the same function.
type SinkFn ¶
A function that defines a Sink's behavior. This function should receive elements via the passed input channel. Please note that this function should not close the passed channel because closing the input channel is the upstream task's responsibility. The whole pipeline will be aborted when the returned error is not nil.
type ToChanOp ¶ added in v0.5.0
type ToChanOp[S any] struct { ElementWiseOp[S] // contains filtered or unexported fields }
A Sink task that sends elements fed by its upstream task to the passed chan.
Source Files