README
¶
Middleware
Each middleware implements a function with the following signature
func (middleware CatchMiddleware) Run(
invocation *models.PipelineInvocation,
result *models.RunResult,
next func(*models.PipelineInvocation, *models.RunResult),
logger *logging.MiddlewareLogger,
stack *middleware.Stack) {}
Built-in middleware
catch - Error Handler
dir - Directory Navigator
docker - Docker Executor
each - Input Duplicator
timer - Directory Timer Middleware
Writing your own middleware
Documentation
¶
Overview ¶
Package middleware provides middlewares that together implement pipedream's core functionalities
Index ¶
- func LookUpPipelineDefinition(definitionsLookup pipeline.DefinitionsLookup, identifier string, ...) (*pipeline.Definition, bool)
- type ExecutionContext
- func (executionContext *ExecutionContext) AddConnection(sourceRun *pipeline.Run, targetRun *pipeline.Run, label string)
- func (executionContext *ExecutionContext) CancelAll() error
- func (executionContext *ExecutionContext) Connections() []*pipeline.DataConnection
- func (executionContext *ExecutionContext) Execute(pipelineIdentifier string, stdoutWriter io.Writer, stderrWriter io.Writer)
- func (executionContext *ExecutionContext) FullRun(options ...FullRunOption) *pipeline.Run
- func (executionContext *ExecutionContext) PipelineFileAtPath(path string) (*pipeline.File, error)
- func (executionContext *ExecutionContext) Runs() []*pipeline.Run
- func (executionContext *ExecutionContext) SetUpCancelHandler(stdoutWriter io.Writer, stderrWriter io.Writer, handler func())
- func (executionContext *ExecutionContext) SetUpPipelines(fileFlag string) error
- func (executionContext *ExecutionContext) UserRuns() []*pipeline.Run
- func (executionContext *ExecutionContext) WaitForRun(identifier string) *pipeline.Run
- type ExecutionContextOption
- func WithDefinitionsLookup(definitions pipeline.DefinitionsLookup) ExecutionContextOption
- func WithExecutionFunction(executionFunction func(run *pipeline.Run)) ExecutionContextOption
- func WithLogger(logger *logrus.Logger) ExecutionContextOption
- func WithMiddlewareStack(stack []Middleware) ExecutionContextOption
- func WithParser(parser *parsing.Parser) ExecutionContextOption
- func WithProjectPath(projectPath string) ExecutionContextOption
- func WithUserPromptImplementation(...) ExecutionContextOption
- type FullRunOption
- func WithArguments(arguments map[string]interface{}) FullRunOption
- func WithIdentifier(identifier *string) FullRunOption
- func WithLogWriter(logWriter io.WriteCloser) FullRunOption
- func WithParentRun(parentRun *pipeline.Run) FullRunOption
- func WithSetupFunc(preCallback func(*pipeline.Run)) FullRunOption
- func WithTearDownFunc(postCallback func(*pipeline.Run)) FullRunOption
- type FullRunOptions
- type Middleware
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func LookUpPipelineDefinition ¶
func LookUpPipelineDefinition(definitionsLookup pipeline.DefinitionsLookup, identifier string, rootFileName string) (*pipeline.Definition, bool)
LookUpPipelineDefinition looks for a particular pipeline within the already parsed deinitions
Types ¶
type ExecutionContext ¶
type ExecutionContext struct {
// SelectableFiles is a list of filenames of all pipeline files in the current directory
//
// Only one file can be selected for execution, but all files will be parsed.
SelectableFiles []string
// PipelineFiles is a list of pipeline files in the current directory, including recursive imports
PipelineFiles []pipeline.File
// Definitions contains all pipeline definitions in the PipelineFiles, as well as built-in pipes
Definitions pipeline.DefinitionsLookup
// MiddlewareStack is a list of middleware items that will be executed in turn
MiddlewareStack []Middleware
// Defaults contains some execution options that can be set at file level
Defaults pipeline.DefaultSettings
// Hooks can execute certain functions before or after pipeline execution
//
// Currently not implemented or used.
Hooks pipeline.HookDefinitions
// Log is the execution context's logger
Log *logrus.Logger
// ProjectPath is the directory in which the tool is currently executing
ProjectPath string
// RootFileName is the name of the file selected for execution
RootFileName string
// UserPromptImplementation by default shows an interactive prompt to the user
//
// Can be overwritten if you need a different implementation e.g. for tests.
UserPromptImplementation func(
label string,
items []string,
initialSelection int,
size int,
input io.ReadCloser,
output io.WriteCloser,
) (int, string, error)
// contains filtered or unexported fields
}
ExecutionContext is the data model keeping track of everything required to execute a pipeline file
func NewExecutionContext ¶
func NewExecutionContext(options ...ExecutionContextOption) *ExecutionContext
NewExecutionContext creates a new ExecutionContext with the specified options
func (*ExecutionContext) AddConnection ¶ added in v0.36.0
func (*ExecutionContext) CancelAll ¶
func (executionContext *ExecutionContext) CancelAll() error
CancelAll cancels all active runs
func (*ExecutionContext) Connections ¶ added in v0.32.32
func (executionContext *ExecutionContext) Connections() []*pipeline.DataConnection
func (*ExecutionContext) Execute ¶
func (executionContext *ExecutionContext) Execute(pipelineIdentifier string, stdoutWriter io.Writer, stderrWriter io.Writer)
Execute runs a pipeline and outputs the result
func (*ExecutionContext) FullRun ¶
func (executionContext *ExecutionContext) FullRun(options ...FullRunOption) *pipeline.Run
FullRun starts the complete execution procedure for a nested pipeline, unwinding the entire middleware stack again
func (*ExecutionContext) PipelineFileAtPath ¶
func (executionContext *ExecutionContext) PipelineFileAtPath(path string) (*pipeline.File, error)
PipelineFileAtPath returns a *pipeline.File corresponding to the parsed pipeline file at the given path, if any
func (*ExecutionContext) Runs ¶
func (executionContext *ExecutionContext) Runs() []*pipeline.Run
func (*ExecutionContext) SetUpCancelHandler ¶ added in v0.33.3
func (executionContext *ExecutionContext) SetUpCancelHandler(stdoutWriter io.Writer, stderrWriter io.Writer, handler func())
SetUpCancelHandler registers a handler for interrupt signals
func (*ExecutionContext) SetUpPipelines ¶
func (executionContext *ExecutionContext) SetUpPipelines(fileFlag string) error
SetUpPipelines collects and parses all relevant pipeline files
func (*ExecutionContext) UserRuns ¶ added in v0.32.32
func (executionContext *ExecutionContext) UserRuns() []*pipeline.Run
UserRuns lists all runs of pipes that are not built-in
func (*ExecutionContext) WaitForRun ¶ added in v0.32.32
func (executionContext *ExecutionContext) WaitForRun(identifier string) *pipeline.Run
WaitForRun blocks until a run with the specified identifier is found and completes
type ExecutionContextOption ¶
type ExecutionContextOption func(*ExecutionContext)
ExecutionContextOption is an option that can be applied to an ExecutionContext or provided to the NewExecutionContext constructor
func WithDefinitionsLookup ¶
func WithDefinitionsLookup(definitions pipeline.DefinitionsLookup) ExecutionContextOption
WithDefinitionsLookup sets the execution context's definitions lookup
func WithExecutionFunction ¶
func WithExecutionFunction(executionFunction func(run *pipeline.Run)) ExecutionContextOption
WithExecutionFunction set the execution context's execution function
You don't need to replace the standard implementation, which unwinds a middleware stack, unless you want to specify a specific behaviour e.g. for tests.
func WithLogger ¶
func WithLogger(logger *logrus.Logger) ExecutionContextOption
WithLogger sets the execution context's logger
func WithMiddlewareStack ¶
func WithMiddlewareStack(stack []Middleware) ExecutionContextOption
WithMiddlewareStack sets the execution context's middleware stack
func WithParser ¶
func WithParser(parser *parsing.Parser) ExecutionContextOption
WithParser sets the execution context's parser
func WithProjectPath ¶
func WithProjectPath(projectPath string) ExecutionContextOption
WithProjectPath sets the execution context's project path
func WithUserPromptImplementation ¶
func WithUserPromptImplementation(implementation func( label string, items []string, initialSelection int, size int, input io.ReadCloser, output io.WriteCloser, ) (int, string, error)) ExecutionContextOption
WithUserPromptImplementation sets the execution context's implementation of a user prompt
By default, this will use promptui to show an interactive prompt to the user, but you may want to override it for tests.
type FullRunOption ¶
type FullRunOption func(*FullRunOptions)
FullRunOption represents an option provided to an executions context's FullRun
func WithArguments ¶
func WithArguments(arguments map[string]interface{}) FullRunOption
WithArguments sets the full run's arguments
func WithIdentifier ¶
func WithIdentifier(identifier *string) FullRunOption
WithIdentifier sets the identifier of the pipeline to be run
It will be used to look up the matching definition, if any
func WithLogWriter ¶
func WithLogWriter(logWriter io.WriteCloser) FullRunOption
WithLogWriter sets an io.WriteCloser to which logs will be written
func WithParentRun ¶
func WithParentRun(parentRun *pipeline.Run) FullRunOption
WithParentRun sets the parent run, i.e. the enclosing run that triggered the full run in question
func WithSetupFunc ¶
func WithSetupFunc(preCallback func(*pipeline.Run)) FullRunOption
WithSetupFunc sets a function that will be executed between the full run's setup and main execution function
func WithTearDownFunc ¶
func WithTearDownFunc(postCallback func(*pipeline.Run)) FullRunOption
WithTearDownFunc sets a function that will be executed after the full run's main execution function
type FullRunOptions ¶
type FullRunOptions struct {
// contains filtered or unexported fields
}
FullRunOptions collects a number of different FullRunOption options into a single structure
type Middleware ¶
type Middleware interface {
String() string
Apply(
run *pipeline.Run,
next func(*pipeline.Run),
executionContext *ExecutionContext,
)
}
Middleware slices the implementation into middlewares, each performing side effects and adapting the run based on provided arguments
each interpret a slice of the run's arguments and transform provide transforms the run during execution
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package catch provides a middleware for graceful handling of stderr output
|
Package catch provides a middleware for graceful handling of stderr output |
|
Package collect provides a middleware for gathering values into a map and saving them to disk
|
Package collect provides a middleware for gathering values into a map and saving them to disk |
|
Package dir provides a middleware for changing the current working directory
|
Package dir provides a middleware for changing the current working directory |
|
Package docker provides a middleware for execution within a Docker (Compose) container
|
Package docker provides a middleware for execution within a Docker (Compose) container |
|
Package each provides a middleware that copies some input into several child pipes running simultaneously
|
Package each provides a middleware that copies some input into several child pipes running simultaneously |
|
Package env provides a middleware managing environment variables
|
Package env provides a middleware managing environment variables |
|
Package extract provides a middleware merging values from nested maps into pipe arguments
|
Package extract provides a middleware merging values from nested maps into pipe arguments |
|
Package inherit provides a middleware that passes arguments from a parent to its children
|
Package inherit provides a middleware that passes arguments from a parent to its children |
|
Package _input provides a middleware to overwrite a pipe's input directly
|
Package _input provides a middleware to overwrite a pipe's input directly |
|
Package interpolate provides a middleware to substitute arguments or inputs into other arguments
|
Package interpolate provides a middleware to substitute arguments or inputs into other arguments |
|
Package _output provides a middleware to overwrite a pipe's output directly
|
Package _output provides a middleware to overwrite a pipe's output directly |
|
Package pipe provides a middleware to execute several commands in sequence
|
Package pipe provides a middleware to execute several commands in sequence |
|
Package _select provides a middleware that shows a selection prompt to the user
|
Package _select provides a middleware that shows a selection prompt to the user |
|
Package sequence provides a middleware that executes pipes synchronously, one after the other
|
Package sequence provides a middleware that executes pipes synchronously, one after the other |
|
Package shell provides a middleware to execute commands in a shell
|
Package shell provides a middleware to execute commands in a shell |
|
Package ssh provides a middleware enabling remote command execution via SSH
|
Package ssh provides a middleware enabling remote command execution via SSH |
|
Package stack defines a list of middleware items to be executed when a pipeline is run
|
Package stack defines a list of middleware items to be executed when a pipeline is run |
|
Package _switch provides a middleware that replaces the output using the first match from a list of regexes
|
Package _switch provides a middleware that replaces the output using the first match from a list of regexes |
|
Package sync provides a middleware that makes the execution function wait until the run has completed
|
Package sync provides a middleware that makes the execution function wait until the run has completed |
|
Package timer provides a middleware that records execution time
|
Package timer provides a middleware that records execution time |
|
Package when provides a middleware that enables conditional execution
|
Package when provides a middleware that enables conditional execution |
|
Package with provides a middleware that extracts and processes input patterns
|
Package with provides a middleware that extracts and processes input patterns |