models

package

v0.1.8 Latest Latest Go to latest Published: Sep 2, 2018 License: Apache-2.0 Imports: 7 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

This section is empty.

This section is empty.

This section is empty.

type Code string

Code is the script code

func (c Code) Configure(name string) error

Configure saves code to a file and builds plugin

type Data []byte

Data is what is passes by pipeline stages

func NewData() Data

NewData returns default data

func (d Data) Unmarshal(value interface{}) error

Unmarshal calls json.Unmarshal on Data into value

type DataProcessor interface {
	ProcessData(
		input Data,
		outputChan chan Data,
		killChan chan error,
	)
}

DataProcessor is a pipeline stage

type ETL interface {
	Extract() DataProcessor
	Transform() DataProcessor
	Load() DataProcessor
}

ETL defines an ETL

type Job struct {
	Name string
	Code Code
	// contains filtered or unexported fields
}

Job has the ETL description

func (j *Job) Configure() error

Configure loads the job plugin

func (j *Job) Execute() error

Execute ...

type Pipeline struct {
	// contains filtered or unexported fields
}

Pipeline contains stages and executes them in order

func NewPipeline(stages ...DataProcessor) *Pipeline

NewPipeline create pipeline with stages

func (p *Pipeline) Run() chan error

Run executes stages and return chan of error at first error on at the end of stages