beam: github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/schema Index | Files

package schema

import "github.com/apache/beam/sdks/go/pkg/beam/core/runtime/graphx/schema"

Package schema contains utility functions for relating Go types and Beam Schemas.

Not all Go types can be converted to schemas. This is Go is more expressive than Beam schemas. Just as not all Go types can be serialized, similarly, not all Beam Schemas will have a conversion to Go types, until the correct mechanism exists in the SDK to handle them.

While efforts will be made to have conversions be reversable, this will not be possible in all instances. Eg. Go arrays as fields will be converted to Beam Arrays, but a Beam Array type will map by default to a Go slice.

Index

Package Files

logicaltypes.go schema.go

func FromType Uses

func FromType(ot reflect.Type) (*pipepb.Schema, error)

FromType returns a Beam Schema of the passed in type. Returns an error if the type cannot be converted to a Schema.

func RegisterLogicalType Uses

func RegisterLogicalType(lt LogicalType)

RegisterLogicalType registers a logical type with the beam schema system. A logical type is a type that has distinct representations and storage.

RegisterLogicalType will panic if the storage type of the LogicalType instance is not a valid storage type.

func RegisterLogicalTypeProvider Uses

func RegisterLogicalTypeProvider(rt reflect.Type, ltp LogicalTypeProvider)

RegisterLogicalTypeProvider allows registration of providers for interface types.

func RegisterType Uses

func RegisterType(ut reflect.Type)

RegisterType converts the type to it's schema representation, and converts it back to a synthetic type so we can map from the synthetic type back to the user type. Recursively registers other named struct types in any component parts.

func Registered Uses

func Registered(ut reflect.Type) bool

Registered returns whether the given type has been registered with the default schema registry.

func ToType Uses

func ToType(s *pipepb.Schema) (reflect.Type, error)

ToType returns a Go type of the passed in Schema. Types returned by ToType are always of Struct kind. Returns an error if the Schema cannot be converted to a type.

type LogicalType Uses

type LogicalType struct {
    // contains filtered or unexported fields
}

LogicalType is a mapping between custom Go types, and their schema equivalent storage types.

A LogicalType is a way to define a type that can be stored in a schema field using a known underlying type for storage. The storage type must be comprised of known schema field types, or pre-registered LogicalTypes.

LogicalTypes may not be mutually recursive at any level of indirection. LogicalTypes must map from a Go type to a single Schema Equivalent storage type.

func ToLogicalType Uses

func ToLogicalType(identifier string, goType, storageType reflect.Type) LogicalType

ToLogicalType creates a LogicalType, indicating that there's a conversion from one to the other.

func (*LogicalType) ArgumentType Uses

func (l *LogicalType) ArgumentType() reflect.Type

ArgumentType returns the Go type of the argument for parameterized types.

func (*LogicalType) ArgumentValue Uses

func (l *LogicalType) ArgumentValue() reflect.Value

ArgumentValue returns the Go value of the argument for parameterized types.

func (*LogicalType) GoType Uses

func (l *LogicalType) GoType() reflect.Type

GoType returns the Go type of the logical type. This is the type in a Go field.

func (*LogicalType) ID Uses

func (l *LogicalType) ID() string

ID is a unique identifier for the logical type.

func (*LogicalType) StorageType Uses

func (l *LogicalType) StorageType() reflect.Type

StorageType is the schema equivalent representation of this logical type. The storage type is how the logical type is encoded in bytes, and if the logical type is unknown, can be decoded into a value of this type instead.

type LogicalTypeProvider Uses

type LogicalTypeProvider = func(reflect.Type) (reflect.Type, error)

LogicalTypeProvider produces a logical type for a given Go type.

If unable to produce a logical type, it instead produces an error. Typically used to handle mapping LogicalTypes from interface types to a concrete implementation. The provider will be passed a type, and will produce an appropriate LogicalType for it.

type Registry Uses

type Registry struct {
    // contains filtered or unexported fields
}

Registry retains mappings from go types to Schemas and LogicalTypes.

func NewRegistry Uses

func NewRegistry() *Registry

NewRegistry creates an initialized LogicalTypeRegistry.

func (*Registry) FromType Uses

func (r *Registry) FromType(ot reflect.Type) (*pipepb.Schema, error)

FromType returns a Beam Schema of the passed in type. Returns an error if the type cannot be converted to a Schema.

func (*Registry) RegisterLogicalType Uses

func (r *Registry) RegisterLogicalType(lt LogicalType)

RegisterLogicalType a single logical type.

func (*Registry) RegisterLogicalTypeProvider Uses

func (r *Registry) RegisterLogicalTypeProvider(rt reflect.Type, ltp LogicalTypeProvider)

RegisterLogicalTypeProvider allows registration of providers for interface types.

func (*Registry) RegisterType Uses

func (r *Registry) RegisterType(ut reflect.Type)

RegisterType converts the type to it's schema representation, and converts it back to a synthetic type so we can map from the synthetic type back to the user type. Recursively registers other named struct types in any component parts.

func (*Registry) Registered Uses

func (r *Registry) Registered(ut reflect.Type) bool

Registered returns whether the given type has been registered with the schema package.

func (*Registry) ToType Uses

func (r *Registry) ToType(s *pipepb.Schema) (reflect.Type, error)

ToType returns a Go type of the passed in Schema. Types returned by ToType are always of Struct kind. Returns an error if the Schema cannot be converted to a type.

Package schema imports 9 packages (graph) and is imported by 3 packages. Updated 2021-01-27. Refresh now. Tools for package owners.