pipeline

package

v0.9.0 Latest Latest Go to latest Published: Jul 23, 2015 License: Apache-2.0 Imports: 18 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/bnlcr/pachyderm

Links

Open Source Insights

Documentation ¶

Overview ¶

package pipeline implements a system for running data pipelines on top of the filesystem

Index ¶

Variables
func RunPipelines(pipelineDir string, inRepo string, outRepo string, commit string, ...) error
func WaitPipeline(pipelineDir, pipeline, commit string) error
type Runner
- func NewRunner(pipelineDir string, inRepo string, outPrefix string, commit string, ...) *Runner

Constants ¶

This section is empty.

Variables ¶

View Source

var (
	ErrFailed          = errors.New("pfs: pipeline failed")
	ErrCancelled       = errors.New("pfs: cancelled")
	ErrArgCount        = errors.New("pfs: illegal argument count")
	ErrUnkownKeyword   = errors.New("pfs: unknown keyword")
	ErrUnknownProtocol = errors.New("pfs: unknown protocol")
)

Functions ¶

func RunPipelines ¶

func RunPipelines(pipelineDir string, inRepo string, outRepo string, commit string, branch string, shard string, cache etcache.Cache) error

RunPipelines lets you easily run the Pipelines in one line if you don't care about cancelling them.

func WaitPipeline ¶

func WaitPipeline(pipelineDir, pipeline, commit string) error

WaitPipeline waits for a pipeline to complete. If the pipeline fails ErrFailed is returned.

Types ¶

type Runner ¶

type Runner struct {
	// contains filtered or unexported fields
}

func NewRunner ¶

func NewRunner(pipelineDir string, inRepo string, outPrefix string, commit string, branch string, shard string, cache etcache.Cache) *Runner

func (*Runner) Cancel ¶

func (r *Runner) Cancel() error

func (*Runner) Inputs ¶

func (r *Runner) Inputs() ([]string, error)

Inputs returns all of the inputs for the pipelines.

func (*Runner) Run ¶

func (r *Runner) Run() error

Run runs all of the pipelines it finds in pipelineDir. Returns the first error it encounters.

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL