Documentation ¶
Overview ¶
package allinone is an experimental mockup of an Apache Beam Go SDK API that leverages generics, and a more opinionated construction method. It exists to explore the ergonomics and feasibility of such an approach.
Core Ideas: * Single Process method, that is passed a special beam context object: DoFnC.
- User manage Iteration avoids start and finish bundle.
- Isolating certain types to passed in callbacks avoids context errors.
- Should also avoid certain issues around sampling, and
- Beam context object has specific methods that can accept a function to push iterate over.
Directories ¶
Path | Synopsis |
---|---|
Package beam is an experimental mockup of an Apache Beam Go SDK API that leverages generics, and a more opinionated construction method.
|
Package beam is an experimental mockup of an Apache Beam Go SDK API that leverages generics, and a more opinionated construction method. |
internal/extworker
Package extworker provides an external worker service and related utilities.
|
Package extworker provides an external worker service and related utilities. |
internal/pipelinex
Package pipelinex contains utilities for manipulating Beam proto pipelines.
|
Package pipelinex contains utilities for manipulating Beam proto pipelines. |
Click to show internal directories.
Click to hide internal directories.