Package runnerlib contains utilities for submitting Go pipelines to a Beam model runner.
BuildTempWorkerBinary creates a local worker binary in the tmp directory for linux/amd64. Caller responsible for deleting the binary.
BuildWorkerBinary creates a local worker binary for linux/amd64. It finds the filename by examining the call stack. We want the user entry (*), for example:
/Users/herohde/go/src/github.com/apache/beam/sdks/go/pkg/beam/runners/beamexec/main.go (skip: 2)
* /Users/herohde/go/src/github.com/apache/beam/sdks/go/examples/wordcount/wordcount.go (skip: 3)
/usr/local/go/src/runtime/proc.go (skip: 4) // not always present /usr/local/go/src/runtime/asm_amd64.s (skip: 4 or 5)
Execute executes a pipeline on the universal runner serving the given endpoint. Convenience function.
IsWorkerCompatibleBinary returns the path to itself and true if running a linux-amd64 binary that can directly be used as a worker binary.
func Prepare(ctx context.Context, client jobpb.JobServiceClient, p *pipepb.Pipeline, opt *JobOptions) (id, endpoint, stagingToken string, err error)
Prepare prepares a job to the given job service. It returns the preparation id artifact staging endpoint, and staging token if successful.
Stage stages the worker binary and any additional files to the given artifact staging endpoint. It returns the retrieval token if successful.
func StageFile(filename string, stream jobpb.ArtifactStagingService_ReverseArtifactRetrievalServiceClient) error
Submit submits a job to the given job service. It returns a jobID, if successful.
WaitForCompletion monitors the given job until completion. It logs any messages and state changes received.
JobOptions capture the various options for submitting jobs to universal runners.