beam: github.com/apache/beam/sdks/go/pkg/beam/options/jobopts Index | Files

package jobopts

import "github.com/apache/beam/sdks/go/pkg/beam/options/jobopts"

Package jobopts contains shared options for job submission. These options are exposed to allow user code to inspect and modify them.

Index

Package Files

options.go

Variables

var (
    // Endpoint is the job service endpoint.
    Endpoint = flag.String("endpoint", "", "Job service endpoint (required).")

    // JobName is the name of the job.
    JobName = flag.String("job_name", "", "Job name (optional).")

    // EnvironmentType is the environment type to run the user code.
    EnvironmentType = flag.String("environment_type", "DOCKER",
        "Environment Type. Possible options are DOCKER and PROCESS.")

    // EnvironmentConfig is the environment configuration for running the user code.
    EnvironmentConfig = flag.String("environment_config",
        "",
        "Set environment configuration for running the user code.\n"+
            "For DOCKER: Url for the docker image.\n"+
            "For PROCESS: json of the form {\"os\": \"<OS>\", "+
            "\"arch\": \"<ARCHITECTURE>\", \"command\": \"<process to execute>\", "+
            "\"env\":{\"<Environment variables 1>\": \"<ENV_VAL>\"} }. "+
            "All fields in the json are optional except command.")

    // WorkerBinary is the location of the compiled worker binary. If not
    // specified, the binary is produced via go build.
    WorkerBinary = flag.String("worker_binary", "", "Worker binary (optional)")

    // Experiments toggle experimental features in the runner.
    Experiments = flag.String("experiments", "", "Comma-separated list of experiments (optional).")

    // Async determines whether to wait for job completion.
    Async = flag.Bool("async", false, "Do not wait for job completion.")

    // Strict mode applies additional validation to user pipelines before
    // executing them and fails early if the pipelines don't pass.
    Strict = flag.Bool("beam_strict", false, "Apply additional validation to pipelines.")
)

func GetEndpoint Uses

func GetEndpoint() (string, error)

GetEndpoint returns the endpoint, if non empty and exits otherwise. Runners such as Dataflow set a reasonable default. Convenience function.

func GetEnvironmentConfig Uses

func GetEnvironmentConfig(ctx context.Context) string

GetEnvironmentConfig returns the specified configuration for specified SDK Harness, if not present, the default development container for the current user. Convenience function.

func GetEnvironmentUrn Uses

func GetEnvironmentUrn(ctx context.Context) string

GetEnvironmentUrn returns the specified EnvironmentUrn used to run the SDK Harness, if not present, returns the docker environment urn "beam:env:docker:v1". Convenience function.

func GetExperiments Uses

func GetExperiments() []string

GetExperiments returns the experiments.

func GetJobName Uses

func GetJobName() string

GetJobName returns the specified job name or, if not present, a fresh autogenerated name. Convenience function.

Package jobopts imports 9 packages (graph) and is imported by 3 packages. Updated 2019-09-24. Refresh now. Tools for package owners.