runtime

package
v0.0.0-...-90deddd Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 18, 2023 License: Apache-2.0 Imports: 56 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

RunSchema is the infra schema for a run.

Functions

func ClusterInstance

func ClusterInstance(config infra.Config) (runner.Cluster, error)

ClusterInstance returns a configured cluster and sets up repository credentials so that remote repositories can be dialed.

func HttpClient

func HttpClient(config infra.Config) (*http.Client, error)

func PredictorConfig

func PredictorConfig(cfg infra.Config, validate bool) (*infra2.PredictorConfig, error)

PredictorConfig returns a PredictorConfig (possibly nil) and an error (if any).

Types

type CommonRunFlags

type CommonRunFlags struct {
	// Assert is the policy used to assert cached flow result compatibility. e.g. never, exact.
	Assert string
	// EvalStrategy is the evaluation strategy. Supported modes are "topdown" and "bottomup".
	EvalStrategy string
	// Invalidate is a regular expression for node identifiers that should be invalidated.
	Invalidate string
	// PostUseChecksum indicates whether input filesets are checksummed after use.
	PostUseChecksum bool
	// RecomputeEmpty indicates if cache results with empty filesets be automatically recomputed.
	RecomputeEmpty bool
}

CommonRunFlags are the run flags that are common across various run modes (run, batch, etc)

func (*CommonRunFlags) Configure

func (r *CommonRunFlags) Configure(c *flow.EvalConfig) (err error)

Configure stores the RunFlags's configuration into the provided EvalConfig.

func (*CommonRunFlags) Err

func (r *CommonRunFlags) Err() error

Err checks if the flag values are consistent and valid.

func (*CommonRunFlags) Flags

func (r *CommonRunFlags) Flags(flags *flag.FlagSet)

Flags adds the common run flags to the provided flagset.

type Eval

type Eval struct {
	// InputArgs is the raw input arguments of the evaluation, as passed
	// in by user.
	InputArgs []string
	// Program stores the reflow program's path.
	Program string
	// Params stores the evaluation's module parameters and raw values.
	Params map[string]string
	// Args stores the evaluation's command line arguments.
	Args []string
	// V1 tells whether this program is a "V1" (".rf") program.
	V1 bool
	// Images is the list of images that were parsed from a syntax session.
	Images []string
	// ImageMap stores a mapping between image names and resolved
	// image names, to be used in evaluation.
	ImageMap map[string]string
	// Type is the module type of the toplevel module that has been
	// evaluated.
	Type *types.T
	// Module is the module value that was evaluated.
	Module values.Module
}

Eval represents the evaluation of a single module. Evaluations are performed by Cmd.Eval.

func (*Eval) Main

func (e *Eval) Main() *flow.Flow

Main returns the flow that represents the module's Main.

func (*Eval) MainType

func (e *Eval) MainType() *types.T

MainType returns the type of the module's Main identifier.

func (*Eval) ResolveImages

func (e *Eval) ResolveImages(sess *session.Session) (err error)

Resolve images resolves the images in an evaluated program.

func (*Eval) Run

func (e *Eval) Run(getBundle bool) (*syntax.Bundle, error)

Run evaluates a reflow program to a flow. It can evaluate both legacy (".reflow") and modern (".rf") programs. It interprets flags as module parameters. Input arguments and options are specified in the passed-in Eval; results are deposited there, too. Run will also return a reflow Bundle for the given reflow program, if getBundle is set to true and if applicable (ie, only if it is a V1 reflow program with .rf or .rfx extension)

type FlagName

type FlagName string
const (
	Unknown FlagName = "unknown"
	// CommonRunFlags flag names
	FlagNameAssert       FlagName = "assert"
	FlagNameEvalStrategy FlagName = "eval"
	FlagNameInvalidate   FlagName = "invalidate"
	// Deprecated: externs are no longer cached.
	// TODO(pfialho): remove flag.
	FlagNameNoCacheExtern   FlagName = "nocacheextern"
	FlagNamePostUseChecksum FlagName = "postusechecksum"
	FlagNameRecomputeEmpty  FlagName = "recomputeempty"
	// RunFlags flag names
	FlagNameBackgroundTimeout FlagName = "backgroundtimeout"
	FlagNameDotGraph          FlagName = "dotgraph"
	FlagNamePred              FlagName = "pred"
	FlagNameTrace             FlagName = "traceflow"
)

type ImageResolver

type ImageResolver struct {
	// Authenticator will have a nil AWS session if the config does not have
	// AWS credentials, in which case it is expected that none of the images are
	// ECR images.
	Authenticator ecrauth.Interface
	// contains filtered or unexported fields
}

ImageResolver maintains maps of image descriptions to their canonical values (fully qualified registry host, and digest based references)

func (*ImageResolver) ResolveImages

func (r *ImageResolver) ResolveImages(ctx context.Context, images []string) (map[string]string, error)

type ReflowRunner

type ReflowRunner interface {
	Go(context.Context) (runner.State, error)
	GetRunID() taskdb.RunID

	// TODO(swami):  Remove this.
	SetDotWriter(io.Writer)
}

ReflowRunner supports the ability to run a reflow program.

type ReflowRuntime

type ReflowRuntime interface {

	// Start starts the reflow runtime.
	// Start uses the provided context to run the necessary background processes,
	// upon cancellation of which, these processes are stopped and the runtime becomes unviable.
	Start(ctx context.Context)

	// WaitDone waits for the reflow runtime to be done (which happens only if ctx to `Start` was cancelled).
	WaitDone()

	// Scheduler returns the underlying scheduler.
	// TODO(swami):  Remove this.  This is exposed temporarily and shouldn't need to be eventually.
	Scheduler() *sched.Scheduler

	// NewRunner returns a new reflow runner.
	NewRunner(RunnerParams) (ReflowRunner, error)
}

ReflowRuntime represents a reflow runtime environment for submitting reflow runs. ReflowRuntime manages all underlying infrastructure components necessary to process reflow runs (eg: Cluster, Scheduler, etc)

func NewRuntime

func NewRuntime(p RuntimeParams) (ReflowRuntime, error)

NewRuntime creates a new reflow runtime from the given parameters. NewRuntime returns an error upon failure to setup and initialize a runtime.

type RunConfig

type RunConfig struct {
	// Program is the local path to the reflow bundle/program.
	Program string
	// Args is the arguments to be passed to the reflow program in the form of "--<argname>=<value>"
	Args []string
	// RunFlags is the run flags for this run.
	RunFlags RunFlags
	// Config is the config for this run.
	Config infra.Config
}

RunConfig defines all the material (configuration, program and args) for a specific run.

type RunFlags

type RunFlags struct {
	CommonRunFlags
	// LocalDir is the directory where execution state is store in Local mode.
	LocalDir string
	// Dir is the where execution state is stored in local mode (alias for backwards compatibility).
	Dir string
	// Local enables execution using the local docker instance.
	Local bool
	// Trace when set enable tracing flow evaluation.
	Trace bool
	Cache bool
	Pred  bool
	// DotGraph enables computation of an evaluation graph.
	DotGraph bool

	// BackgroundTimeout is the duration to wait for background tasks (such as cache writes, etc) to complete.
	// ie, this is the amount of time we wait after the user's program execution finishes but before reflow exits.
	BackgroundTimeout time.Duration
}

RunFlags is the supported flags and parameters for a run.

func (*RunFlags) Err

func (r *RunFlags) Err() error

Err checks if the flag values are consistent and valid.

func (*RunFlags) Flags

func (r *RunFlags) Flags(flags *flag.FlagSet)

flags adds all run flags to the provided flagset (including localflags).

func (*RunFlags) FlagsNoLocal

func (r *RunFlags) FlagsNoLocal(flags *flag.FlagSet, prefix string)

FlagsNoLocal adds run flags to the provided flagset with the given prefix excluding flags pertaining to local runs.

func (*RunFlags) Override

func (r *RunFlags) Override(overrides map[string]string) (err error)

Override overrides the appropriate RunFlags fields based on the given map of flag names to values. This is specifically used where overriding only a subset of flags is desired, without inheriting the default values of the flags that aren't being overridden.

type RunnerParams

type RunnerParams struct {
	RunConfig RunConfig
	Logger    *log.Logger

	// Status is the status object specific to a run.
	Status *status.Status
}

RunnerParams defines the set of parameters necessary to initialize a ReflowRunner.

type RuntimeParams

type RuntimeParams struct {

	// Config is the main configuration for the reflow runtime.
	Config infra.Config

	// Logger is the top-level logger passed to the various underlying components.
	// Logger is optional and overrides the Config provided logger (if any).
	Logger *log.Logger

	// Status is the top-level status object.  If provided, various internal components
	// will create groups under this status object for reporting purposes.
	Status *status.Status
}

RuntimeParams defines the set of parameters necessary to initialize a ReflowRuntime.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL