Documentation
¶
Overview ¶
Package xlangx contains various low-level utilities needed for adding cross-language transforms to the pipeline.
Index ¶
- func EncodeStructPayload(pl interface{}) ([]byte, error)
- func Expand(edge *graph.MultiEdge, ext *graph.ExternalTransform) error
- func ResolveArtifacts(ctx context.Context, edges []*graph.MultiEdge, p *pipepb.Pipeline)
- func ResolveArtifactsWithConfig(ctx context.Context, edges []*graph.MultiEdge, cfg ResolveConfig) (paths map[string]string, err error)
- type ResolveConfig
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EncodeStructPayload ¶
EncodeStructPayload takes a native Go struct and returns a marshaled ExternalConfigurationPayload proto, containing a Schema representation of the original type and the original value encoded as a Row. This is intended to be used as the expansion payload for an External transform.
func Expand ¶
func Expand(edge *graph.MultiEdge, ext *graph.ExternalTransform) error
Expand expands an unexpanded graph.ExternalTransform as a graph.ExpandedTransform and assigns it to the ExternalTransform's Expanded field. This requires querying an expansion service based on the configuration details within the ExternalTransform.
func ResolveArtifacts ¶
ResolveArtifacts acquires all dependencies for a cross-language transform
func ResolveArtifactsWithConfig ¶
func ResolveArtifactsWithConfig(ctx context.Context, edges []*graph.MultiEdge, cfg ResolveConfig) (paths map[string]string, err error)
ResolveArtifactsWithConfig acquires all dependencies for cross-language transforms, but with some additional configuration to behavior. By default, this function performs the following steps for each cross-language transform in the list of edges:
- Retrieves a list of dependencies needed from the expansion service.
- Retrieves each dependency as an artifact and stages it to a default local filepath.
- Adds the dependencies to the transform's stored environment proto.
The changes that can be configured are documented in ResolveConfig.
This returns a map of "local path" to "sdk path". By default these are identical, unless ResolveConfig.SdkPath has been set.
Types ¶
type ResolveConfig ¶
type ResolveConfig struct {
// SdkPath replaces the default filepath for dependencies, but only in the
// external environment proto to be used by the SDK Harness during pipeline
// execution. This is used to specify alternate staging directories, such
// as for staging artifacts remotely.
//
// Setting an SdkPath does not change staging behavior otherwise. All
// artifacts still get staged to the default local filepath, and it is the
// user's responsibility to stage those local artifacts to the SdkPath.
SdkPath string
// JoinFn is a function for combining SdkPath and individual artifact names.
// If not specified, it defaults to using filepath.Join.
JoinFn func(path, name string) string
}
ResolveConfig contains fields for configuring the behavior for resolving artifacts.
Source Files