beam: github.com/apache/beam/sdks/go/pkg/beam/core/runtime Index | Files | Directories

package runtime

import "github.com/apache/beam/sdks/go/pkg/beam/core/runtime"

Package runtime contains runtime hooks and utilities for pipeline options and type registration. Most functionality done in init and hence is available both during pipeline-submission and at runtime.

Index

Package Files

init.go options.go symbols.go types.go

Variables

var GlobalOptions = &Options{opt: make(map[string]string)}

GlobalOptions are the global options for the active graph. Options can be defined at any time before execution and are re-created by the harness on remote execution workers. Global options should be used sparingly.

func Init Uses

func Init()

Init is the hook that all user code must call after flags processing and other static initialization, for now.

func Initialized Uses

func Initialized() bool

Initialized exposes the initialization status for runners.

func LookupType Uses

func LookupType(key string) (reflect.Type, bool)

LookupType looks up a type in the global type registry by external key.

func RegisterFunction Uses

func RegisterFunction(fn interface{})

RegisterFunction allows function registration. It is beneficial for performance and is needed for functions -- such as custom coders -- serialized during unit tests, where the underlying symbol table is not available. It should be called in init() only. Returns the external key for the function.

func RegisterInit Uses

func RegisterInit(hook func())

RegisterInit registers an Init hook. Hooks are expected to be able to figure out whether they apply on their own, notably if invoked in a remote execution environment. They are all executed regardless of the runner.

func RegisterType Uses

func RegisterType(t reflect.Type) string

RegisterType inserts "external" types into a global type registry to bypass serialization and preserve full method information. It should be called in init() only. Returns the external key for the type.

func ResolveFunction Uses

func ResolveFunction(name string, t reflect.Type) (interface{}, error)

ResolveFunction resolves the runtime value of a given function by symbol name and type.

func TypeKey Uses

func TypeKey(t reflect.Type) (string, bool)

TypeKey returns the external key of a given type. Returns false if not a candidate for registration.

type Options Uses

type Options struct {
    // contains filtered or unexported fields
}

Options are untyped options.

func (*Options) Export Uses

func (o *Options) Export() RawOptions

Export returns a JSON-serializable copy of the options.

func (*Options) Get Uses

func (o *Options) Get(key string) string

Get returns the value of the key. If the key has not been set, it returns "".

func (*Options) Import Uses

func (o *Options) Import(opt RawOptions)

Import imports the options from previously exported data and makes the options read-only. It panics if import is called twice.

func (*Options) Set Uses

func (o *Options) Set(key, value string)

Set defines the value of the given key. If the key is already defined, it panics.

type RawOptions Uses

type RawOptions struct {
    Options map[string]string `json:"options"`
}

RawOptions represents exported options as JSON-serializable data. Exact representation is subject to change.

type RawOptionsWrapper Uses

type RawOptionsWrapper struct {
    Options      RawOptions `json:"beam:option:go_options:v1"`
    Runner       string     `json:"beam:option:runner:v1"`
    AppName      string     `json:"beam:option:app_name:v1"`
    Experiments  []string   `json:"beam:option:experiments:v1"`
    RetainDocker bool       `json:"beam:option:retain_docker_containers:v1"`
}

RawOptionsWrapper wraps RawOptions to the form expected by the harness. The extra layer is currently needed due to Dataflow expectations about this representation. Subject to change.

type SymbolResolver Uses

type SymbolResolver interface {
    // Sym2Addr returns the address pointer for a given symbol.
    Sym2Addr(string) (uintptr, error)
}

SymbolResolver resolves a symbol to an unsafe address.

var (
    // Resolver is the accessible symbol resolver the runtime uses to find functions.
    Resolver SymbolResolver
)

Directories

PathSynopsis
coderxPackage coderx contains coders for primitive types that aren't included in the beam model.
execPackage exec contains runtime plan representation and execution.
exec/optimizedPackage optimized contains type-specialized shims for faster execution.
graphxPackage graphx provides facilities to help with the serialization of pipelines into a serializable graph structure suitable for the worker.
graphx/schemaPackage schema contains utility functions for relating Go types and Beam Schemas.
graphx/v1Package v1 is a generated protocol buffer package.
harnessPackage harness implements the SDK side of the Beam FnAPI.
harness/initPackage init contains the harness initialization code defined by the FnAPI.
harness/sessionPackage session is a generated protocol buffer package.
pipelinexPackage pipelinex contains utilities for manipulating Beam proto pipelines.
xlangx

Package runtime imports 7 packages (graph) and is imported by 17 packages. Updated 2020-09-17. Refresh now. Tools for package owners.