beam: github.com/apache/beam/sdks/go/pkg/beam/core/funcx Index | Files

package funcx

import "github.com/apache/beam/sdks/go/pkg/beam/core/funcx"

Package funcx contains functions and types used to perform type analysis of Beam functions. This package is primarily used in pipeline construction to typecheck pipelines, but could be used by anyone who wants to perform type analysis of Beam user functions. For performing typechecking of code, consider the methods Try* in the beam package, as they are built on these primitives and may be more useful.

Index

Package Files

doc.go fn.go output.go sideinput.go signature.go

func IsEmit Uses

func IsEmit(t reflect.Type) bool

IsEmit returns true iff the supplied type is an emitter.

func IsEmitWithEventTime Uses

func IsEmitWithEventTime(t reflect.Type) bool

IsEmitWithEventTime return true iff the supplied type is an emitter and the first argument is the optional EventTime.

func IsIter Uses

func IsIter(t reflect.Type) bool

IsIter returns true iff the supplied type is a "single sweep functional iterator".

A single sweep functional iterator is a function taking one or more pointers of data as arguments, that returns a single boolean value. The semantics of the function are that when called, if there are values to be supplied, they will be copied into the supplied pointers. The function returns true if data was copied, and false if there is no more data available.

func IsReIter Uses

func IsReIter(t reflect.Type) bool

IsReIter returns true iff the supplied type is a functional iterator generator.

A functional iterator generator is a parameter-less function that returns single sweep functional iterators.

func MustSatisfy Uses

func MustSatisfy(fn interface{}, sig *Signature)

MustSatisfy panics if the given fn does not satisfy the signature.

func Satisfy Uses

func Satisfy(fn interface{}, sig *Signature) error

Satisfy returns nil iff the fn can satisfy the signature, respecting generics. For example, for

foo : (context.Context, X) -> bool
bar : (int) -> bool

both would satisfy a signature of (context.Context?, int) -> bool. Only "foo" would satisfy (context.Context, string) -> bool and only "bar" would satisfy (int) -> bool.

func UnfoldEmit Uses

func UnfoldEmit(t reflect.Type) ([]reflect.Type, bool)

UnfoldEmit returns the parameter types, if an emitter. For example:

func (int)                  returns {int}
func (string, int)          returns {string, int}
func (typex.EventTime, int) returns {typex.EventTime, int}

func UnfoldIter Uses

func UnfoldIter(t reflect.Type) ([]reflect.Type, bool)

UnfoldIter returns the parameter types, if a single sweep functional iterator. For example:

func (*int) bool                   returns {int}
func (*string, *int) bool          returns {string, int}
func (*typex.EventTime, *int) bool returns {typex.EventTime, int}

func UnfoldReIter Uses

func UnfoldReIter(t reflect.Type) ([]reflect.Type, bool)

UnfoldReIter returns the parameter types, if a functional iterator generator.

type Fn Uses

type Fn struct {
    Fn  reflectx.Func

    Param []FnParam
    Ret   []ReturnParam
}

Fn is the reflected user function or method, preprocessed. This wrapper is useful both at graph construction time as well as execution time.

func New Uses

func New(fn reflectx.Func) (*Fn, error)

New returns a Fn from a user function, if valid. Closures and dynamically created functions are considered valid here, but will be rejected if they are attempted to be serialized.

func (*Fn) Context Uses

func (u *Fn) Context() (pos int, exists bool)

Context returns (index, true) iff the function expects a context.Context. The context should be the first parameter by convention.

func (*Fn) Emits Uses

func (u *Fn) Emits() (pos int, num int, exists bool)

Emits returns (index, num, true) iff the function expects one or more emitters. The index returned is the index of the first emitter param in the signature. The num return value is the number of emitters in the signature. When there are multiple emitters in the signature, they will all be located contiguously, so the range of emitter params is [index, index+num).

func (*Fn) Error Uses

func (u *Fn) Error() (pos int, exists bool)

Error returns (index, true) iff the function returns an error.

func (*Fn) EventTime Uses

func (u *Fn) EventTime() (pos int, exists bool)

EventTime returns (index, true) iff the function expects an event timestamp.

func (*Fn) Inputs Uses

func (u *Fn) Inputs() (pos int, num int, exists bool)

Inputs returns (index, num, true) iff the function expects one or more inputs, consisting of the main input followed by any number of side inputs. The index returned is the index of the first input, which is always the main input. The num return value is the number of total inputs in the signature. The main input and all side inputs are located contiguously

func (*Fn) OutEventTime Uses

func (u *Fn) OutEventTime() (pos int, exists bool)

OutEventTime returns (index, true) iff the function returns an event timestamp.

func (*Fn) Params Uses

func (u *Fn) Params(mask FnParamKind) []int

Params returns the parameter indices that matches the given mask.

func (*Fn) Returns Uses

func (u *Fn) Returns(mask ReturnKind) []int

Returns returns the return indices that matches the given mask.

func (*Fn) String Uses

func (u *Fn) String() string

func (*Fn) Type Uses

func (u *Fn) Type() (pos int, exists bool)

Type returns (index, true) iff the function expects a reflect.FullType.

func (*Fn) Window Uses

func (u *Fn) Window() (pos int, exists bool)

Window returns (index, true) iff the function expects a window.

type FnParam Uses

type FnParam struct {
    Kind FnParamKind
    T    reflect.Type
}

FnParam captures the kind and type of a single user function parameter.

func SubParams Uses

func SubParams(list []FnParam, indices ...int) []FnParam

SubParams returns the subsequence of the given params with the given indices.

type FnParamKind Uses

type FnParamKind int

FnParamKind represents the kinds of parameters a user function may take.

const (
    // FnIllegal is an illegal function input parameter type.
    FnIllegal FnParamKind = 0x0
    // FnContext marks a function input parameter of type context.Context.
    FnContext FnParamKind = 0x1
    // FnEventTime indicates a function input parameter of type typex.EventTime.
    FnEventTime FnParamKind = 0x2
    // FnValue indicates a function input parameter of an ordinary Go type.
    FnValue FnParamKind = 0x4
    // FnIter indicates a function input parameter that is an iterator.
    // Examples of iterators:
    //       "func (*int) bool"
    //       "func (*string, *T) bool"
    // If there are 2 parameters, a KV input is implied.
    FnIter FnParamKind = 0x08
    // FnReIter indicates a function input parameter that is a reiterable
    // iterator.
    // The function signature is a function returning a function matching
    // the iterator signature.
    //   "func() func (*int) bool"
    //   "func() func (*string, *T) bool"
    // are reiterable versions of the FnIter examples.
    FnReIter FnParamKind = 0x10
    // FnEmit indicates a function input parameter that is an emitter.
    // Examples of emitters:
    //       "func (int)"
    //       "func (string, T)"
    //       "func (EventTime, int)"
    //       "func (EventTime, string, T)"
    // If there are 2 regular parameters, a KV output is implied. An optional
    // EventTime is allowed as well. Emitters cannot fail.
    FnEmit FnParamKind = 0x20
    // FnType indicates a function input parameter that is a type for a coder. It
    // is only valid for coders.
    FnType FnParamKind = 0x40
    // FnWindow indicates a function input parameter that implements typex.Window.
    FnWindow FnParamKind = 0x80
)

func (FnParamKind) String Uses

func (k FnParamKind) String() string

type ReturnKind Uses

type ReturnKind int

ReturnKind represents the kinds of return values a user function may provide.

const (
    RetIllegal   ReturnKind = 0x0
    RetEventTime ReturnKind = 0x1
    RetValue     ReturnKind = 0x2
    RetError     ReturnKind = 0x4
)

The supported types of ReturnKind.

func (ReturnKind) String Uses

func (k ReturnKind) String() string

type ReturnParam Uses

type ReturnParam struct {
    Kind ReturnKind
    T    reflect.Type
}

ReturnParam captures the kind and type of a single user function return value.

func SubReturns Uses

func SubReturns(list []ReturnParam, indices ...int) []ReturnParam

SubReturns returns the subsequence of the given return params with the given indices.

type Signature Uses

type Signature struct {
    // OptArgs is the optional arguments allowed in order, if any, before any
    // required arguments. Must be concrete types.
    OptArgs []reflect.Type
    // Args is the required arguments allowed in order, if any.
    Args []reflect.Type
    // Return is the required returns allowed in order, if any.
    Return []reflect.Type
    // OptReturn is the optional returns allowed in order, if any, after any
    // required returns. Must be concrete types.
    OptReturn []reflect.Type
}

Signature is a concise representation of a group of function types. The function types in a group can differ in optional leading arguments and trailing returns only. For example, a signature can represent:

(context.Context?, int, string) -> (bool, error?)

where the context arguments and error return are optional. The int and string parameters as well as the bool return are mandatory.

func MakePredicate Uses

func MakePredicate(args ...reflect.Type) *Signature

MakePredicate creates a simple N-ary predicate: <args> -> bool.

func Replace Uses

func Replace(sig *Signature, old, new reflect.Type) *Signature

Replace substitutes the old top-level type for the new one. It is intended to specialize generic signatures to concrete ones.

func (*Signature) String Uses

func (sig *Signature) String() string

type TypeMismatchError Uses

type TypeMismatchError struct {
    Got, Want reflect.Type
}

TypeMismatchError indicates we didn't get the type we expected.

func (*TypeMismatchError) Error Uses

func (e *TypeMismatchError) Error() string

Package funcx imports 6 packages (graph) and is imported by 11 packages. Updated 2019-09-24. Refresh now. Tools for package owners.