scipipe: github.com/scipipe/scipipe/components Index | Files

package components

import "github.com/scipipe/scipipe/components"

Index

Package Files

concatenator.go file_reader.go file_source.go file_splitter.go maptokeys.go param_source.go streamtosubstream.go

type Concatenator Uses

type Concatenator struct {
    scipipe.BaseProcess
    OutPath string
}

Concatenator is a process that concatenates the content of multiple files received in the in-port In, into one file returned on its out-port, Out

func NewConcatenator Uses

func NewConcatenator(wf *scipipe.Workflow, name string, outPath string) *Concatenator

NewConcatenator returns a new, initialized Concatenator process

func (*Concatenator) In Uses

func (p *Concatenator) In() *scipipe.InPort

In returns the (only) in-port for this process

func (*Concatenator) Out Uses

func (p *Concatenator) Out() *scipipe.OutPort

Out returns the (only) out-port for this process

func (*Concatenator) Run Uses

func (p *Concatenator) Run()

Run runs the Concatenator process

type FileReader Uses

type FileReader struct {
    scipipe.BaseProcess
}

FileReader takes a file path on its FilePath in-port, and returns the file content as []byte on its out-port Out

func NewFileReader Uses

func NewFileReader(wf *scipipe.Workflow, name string) *FileReader

NewFileReader returns an initialized new FileReader

func (*FileReader) InFilePath Uses

func (p *FileReader) InFilePath() *scipipe.ParamInPort

InFilePath returns the parameter in-port on which a file name is read

func (*FileReader) OutLine Uses

func (p *FileReader) OutLine() *scipipe.ParamOutPort

OutLine returns an parameter out-port with lines of the files being read

func (*FileReader) Run Uses

func (p *FileReader) Run()

Run the FileReader

type FileSource Uses

type FileSource struct {
    scipipe.BaseProcess
    // contains filtered or unexported fields
}

FileSource is initiated with a set of file paths, which it will send as a stream of File IPs on its outport Out()

func NewFileSource Uses

func NewFileSource(wf *scipipe.Workflow, name string, filePaths ...string) *FileSource

NewFileSource returns a new initialized FileSource process

func (*FileSource) Out Uses

func (p *FileSource) Out() *scipipe.OutPort

Out returns the out-port, on which file IPs based on the file paths the process was initialized with, will be retrieved.

func (*FileSource) Run Uses

func (p *FileSource) Run()

Run runs the FileSource process

type FileSplitter Uses

type FileSplitter struct {
    scipipe.BaseProcess
    LinesPerSplit int
}

FileSplitter is a process that will split a file into multiple files, each with LinesPerSplit number of lines per file

func NewFileSplitter Uses

func NewFileSplitter(wf *scipipe.Workflow, name string, linesPerSplit int) *FileSplitter

NewFileSplitter returns an initialized FileSplitter process that will split a file into multiple files, each with linesPerSplit number of lines per file

func (*FileSplitter) InFile Uses

func (p *FileSplitter) InFile() *scipipe.InPort

InFile returns the port for the input file

func (*FileSplitter) OutSplitFile Uses

func (p *FileSplitter) OutSplitFile() *scipipe.OutPort

OutSplitFile returns the resulting split (part) files generated0

func (*FileSplitter) Run Uses

func (p *FileSplitter) Run()

Run runs the FileSplitter process

type MapToKeys Uses

type MapToKeys struct {
    scipipe.BaseProcess
    // contains filtered or unexported fields
}

MapToKeys is a process that runs a function provided by the user, upon initialization, that will provide a map of key:value pairs, based in IPs read on the In-port. The key:value pairs (maps) are added to the IPs on the out-port, which are identical to the incoming IPs, except for the new key:value map

func NewMapToKeys Uses

func NewMapToKeys(wf *scipipe.Workflow, name string, mapFunc func(ip *scipipe.FileIP) map[string]string) *MapToKeys

NewMapToKeys returns an initialized MapToKeys process

func (*MapToKeys) In Uses

func (p *MapToKeys) In() *scipipe.InPort

In takes input files the content of which the map function will be run, to generate keys

func (*MapToKeys) Out Uses

func (p *MapToKeys) Out() *scipipe.OutPort

Out outputs files that are supplemented with keys by the map function.

func (*MapToKeys) Run Uses

func (p *MapToKeys) Run()

Run runs the MapToKeys process

type ParamSource Uses

type ParamSource struct {
    scipipe.BaseProcess
    // contains filtered or unexported fields
}

ParamSource will feed parameters on an out-port

func NewParamSource Uses

func NewParamSource(wf *scipipe.Workflow, name string, params ...string) *ParamSource

NewParamSource returns a new ParamSource

func (*ParamSource) Out Uses

func (p *ParamSource) Out() *scipipe.ParamOutPort

Out returns the out-port, on which parameters the process was initialized with, will be retrieved.

func (*ParamSource) Run Uses

func (p *ParamSource) Run()

Run runs the process

type StreamToSubStream Uses

type StreamToSubStream struct {
    scipipe.BaseProcess
}

StreamToSubStream takes a normal stream of IP's representing individual files, and returns one IP where the incoming IPs are sent on its substream.

func NewStreamToSubStream Uses

func NewStreamToSubStream(wf *scipipe.Workflow, name string) *StreamToSubStream

NewStreamToSubStream instantiates a new StreamToSubStream process

func (*StreamToSubStream) In Uses

func (p *StreamToSubStream) In() *scipipe.InPort

In returns the in-port

func (*StreamToSubStream) OutSubStream Uses

func (p *StreamToSubStream) OutSubStream() *scipipe.OutPort

OutSubStream returns the out-port

func (*StreamToSubStream) Run Uses

func (p *StreamToSubStream) Run()

Run runs the StreamToSubStream

Package components imports 7 packages (graph). Updated 2018-05-25. Refresh now. Tools for package owners.