beam: github.com/apache/beam/sdks/go/pkg/beam/util/shimx Index | Files

package shimx

import "github.com/apache/beam/sdks/go/pkg/beam/util/shimx"

Package shimx specifies the templates for generating type assertion shims for Apache Beam Go SDK pipelines.

In particular, the shims are used by the Beam Go SDK to avoid reflection at runtime, which is the default mode of operation. The shims are specialized for the code in question, using type assertion to convert arguments as required, and invoke the user code.

Similar shims are required for emitters, and iterators in order to propagate values out of, and in to user functions respectively without reflection overhead.

Registering user types is required to support user types as PCollection types, while registering functions is required to avoid possibly expensive function resolution at worker start up, which defaults to using DWARF Symbol tables.

The generator largely relies on basic types and strings to ensure that it's usable by dynamic processes via reflection, or by any static analysis approach that is used in the future.

Index

Package Files

generate.go

Variables

var (
    ExecImport     = "github.com/apache/beam/sdks/go/pkg/beam/core/runtime/exec"
    TypexImport    = "github.com/apache/beam/sdks/go/pkg/beam/core/typex"
    ReflectxImport = "github.com/apache/beam/sdks/go/pkg/beam/core/util/reflectx"
    RuntimeImport  = "github.com/apache/beam/sdks/go/pkg/beam/core/runtime"
)

Beam imports that the generated code requires.

func File Uses

func File(w io.Writer, top *Top)

File writes go code to the given writer.

func FuncName Uses

func FuncName(inNames, outNames []string) string

FuncName returns a compilable Go identifier for a function, given valid type names as generated by Name. See https://golang.org/ref/spec#Identifiers for details.

func Name Uses

func Name(t string) string

Name creates a capitalized identifier from a type string. The identifier follows the rules of go identifiers and should be compileable. See https://golang.org/ref/spec#Identifiers for details.

type Emitter Uses

type Emitter struct {
    Name, Type string // The user name of the function, the type of the emit.
    Time       bool   // if this uses event time.
    Key, Val   string // The type of the emits.
}

Emitter represents an emitter shim to be generated.

type Func Uses

type Func struct {
    Name, Type string
    In, Out    []string
}

Func represents a type assertion shim for function invocation to be generated.

type Input Uses

type Input struct {
    Name, Type string // The user name of the function, the type of the iterator (including the bool).
    Time       bool   // if this uses event time.
    Key, Val   string // The type of the inputs, pointers removed.
}

Input represents an iterator shim to be generated.

type Top Uses

type Top struct {
    FileName, ToolName, Package string

    Imports   []string // the full import paths
    Functions []string // the plain names of the functions to be registered.
    Types     []string // the plain names of the types to be registered.
    Wraps     []Wrap
    Emitters  []Emitter
    Inputs    []Input
    Shims     []Func
}

Top is the top level inputs into the template file for generating shims.

type Wrap Uses

type Wrap struct {
    Name, Type string
    Methods    []Func
}

Wrap represents a type assertion shim for Structural DoFn method invocation to be generated.

Package shimx imports 5 packages (graph) and is imported by 2 packages. Updated 2019-03-08. Refresh now. Tools for package owners.