scipipe: github.com/scipipe/scipipe Index | Files | Directories

package scipipe

import "github.com/scipipe/scipipe"

Package scipipe is a library for writing scientific workflows (sometimes also called "pipelines") of shell commands that depend on each other, in the Go programming languages. It was initially designed for problems in cheminformatics and bioinformatics, but should apply equally well to any domain involving complex pipelines of interdependent shell commands.

Index

Package Files

audit.go baseprocess.go const.go ip.go log.go port.go process.go sink.go task.go utils.go workflow.go

Constants

const (
    // BUFSIZE is the standard buffer size used for channels connecting processes
    BUFSIZE = 16
)

Variables

var (
    // Trace is a log handler for extremely detailed level logs. It is so far
    // sparely used in scipipe.
    Trace *log.Logger
    // Debug is a log handler for debugging level logs
    Debug *log.Logger
    // Info is a log handler for information level logs
    Info *log.Logger
    // Audit is a log handler for audit level logs
    Audit *log.Logger
    // Warning is a log handler for warning level logs
    Warning *log.Logger
    // Error is a log handler for error level logs
    Error *log.Logger
)

func Check Uses

func Check(err error)

Check checks the error err, and prints the message in the error

func CheckWithMsg Uses

func CheckWithMsg(err error, errMsg string)

CheckWithMsg checks the error err, and prints both the original error message, and a custom one provided in errMsg

func ExecCmd Uses

func ExecCmd(cmd string) string

ExecCmd executes the command cmd, as a shell command via bash

func Fail Uses

func Fail(vs ...interface{})

Fail logs the error message, so that it will be possible to improve error messages in one place

func Failf Uses

func Failf(msg string, vs ...interface{})

Failf is like Fail but with msg being a formatter string for the message and vs being items to format into the message

func InitLog Uses

func InitLog(
    traceHandle io.Writer,
    debugHandle io.Writer,
    infoHandle io.Writer,
    auditHandle io.Writer,
    warningHandle io.Writer,
    errorHandle io.Writer)

InitLog initiates logging handlers

func InitLogAudit Uses

func InitLogAudit()

InitLogAudit initiate logging with level=AUDIT

func InitLogAuditToFile Uses

func InitLogAuditToFile(filePath string)

InitLogAuditToFile initiate logging with level=AUDIT, and write that to fileName

func InitLogDebug Uses

func InitLogDebug()

InitLogDebug initiates logging with level=DEBUG

func InitLogError Uses

func InitLogError()

InitLogError initiates logging with level=ERROR

func InitLogInfo Uses

func InitLogInfo()

InitLogInfo initiates logging with level=INFO

func InitLogWarning Uses

func InitLogWarning()

InitLogWarning initiates logging with level=WARNING

type AuditInfo Uses

type AuditInfo struct {
    ID          string
    ProcessName string
    Command     string
    Params      map[string]string
    Keys        map[string]string
    StartTime   time.Time
    FinishTime  time.Time
    ExecTimeMS  time.Duration
    Upstream    map[string]*AuditInfo
}

AuditInfo contains structured audit/provenance logging information for a particular task (invocation), to go with all outgoing IPs from that task

func NewAuditInfo Uses

func NewAuditInfo() *AuditInfo

NewAuditInfo returns a new AuditInfo struct

type BaseIP Uses

type BaseIP struct {
    // contains filtered or unexported fields
}

BaseIP contains foundational functionality which all IPs need to implement. It is meant to be embedded into other IP implementations.

func NewBaseIP Uses

func NewBaseIP(path string) *BaseIP

NewBaseIP creates a new BaseIP

func (*BaseIP) ID Uses

func (ip *BaseIP) ID() string

ID returns a globally unique ID for the IP

type BaseProcess Uses

type BaseProcess struct {
    // contains filtered or unexported fields
}

BaseProcess provides a skeleton for processes, such as the main Process component, and the custom components in the scipipe/components library

func NewBaseProcess Uses

func NewBaseProcess(wf *Workflow, name string) BaseProcess

NewBaseProcess returns a new BaseProcess, connected to the provided workflow, and with the name name

func (*BaseProcess) CloseAllOutPorts Uses

func (p *BaseProcess) CloseAllOutPorts()

CloseAllOutPorts closes all normal-, and parameter out ports

func (*BaseProcess) CloseOutPorts Uses

func (p *BaseProcess) CloseOutPorts()

CloseOutPorts closes all (normal) out-ports

func (*BaseProcess) CloseParamOutPorts Uses

func (p *BaseProcess) CloseParamOutPorts()

CloseParamOutPorts closes all parameter out-ports

func (*BaseProcess) Connected Uses

func (p *BaseProcess) Connected() (isConnected bool)

Connected checks whether all the process' ports are connected

func (*BaseProcess) DeleteInPort Uses

func (p *BaseProcess) DeleteInPort(portName string)

DeleteInPort deletes an InPort object from the process

func (*BaseProcess) DeleteOutPort Uses

func (p *BaseProcess) DeleteOutPort(portName string)

DeleteOutPort deletes a OutPort object from the process

func (*BaseProcess) DeleteParamInPort Uses

func (p *BaseProcess) DeleteParamInPort(portName string)

DeleteParamInPort deletes a ParamInPort object from the process

func (*BaseProcess) DeleteParamOutPort Uses

func (p *BaseProcess) DeleteParamOutPort(portName string)

DeleteParamOutPort deletes a ParamOutPort object from the process

func (*BaseProcess) InPort Uses

func (p *BaseProcess) InPort(portName string) *InPort

InPort returns the in-port with name portName

func (*BaseProcess) InPorts Uses

func (p *BaseProcess) InPorts() map[string]*InPort

InPorts returns a map of all the in-ports of the process, keyed by their names

func (*BaseProcess) InitInPort Uses

func (p *BaseProcess) InitInPort(proc WorkflowProcess, portName string)

InitInPort adds the in-port port to the process, with name portName

func (*BaseProcess) InitOutPort Uses

func (p *BaseProcess) InitOutPort(proc WorkflowProcess, portName string)

InitOutPort adds the out-port port to the process, with name portName

func (*BaseProcess) InitParamInPort Uses

func (p *BaseProcess) InitParamInPort(proc WorkflowProcess, portName string)

InitParamInPort adds the parameter port paramPort with name portName

func (*BaseProcess) InitParamOutPort Uses

func (p *BaseProcess) InitParamOutPort(proc WorkflowProcess, portName string)

InitParamOutPort initializes the parameter port paramPort with name portName to the process We need to supply the concrete process used here as well, since this method might be used as part of an embedded struct, meaning that the process in the receiver is just the *BaseProcess, which doesn't suffice.

func (*BaseProcess) Name Uses

func (p *BaseProcess) Name() string

Name returns the name of the process

func (*BaseProcess) OutPort Uses

func (p *BaseProcess) OutPort(portName string) *OutPort

OutPort returns the out-port with name portName

func (*BaseProcess) OutPorts Uses

func (p *BaseProcess) OutPorts() map[string]*OutPort

OutPorts returns a map of all the out-ports of the process, keyed by their names

func (*BaseProcess) ParamInPort Uses

func (p *BaseProcess) ParamInPort(portName string) *ParamInPort

ParamInPort returns the parameter port with name portName

func (*BaseProcess) ParamInPorts Uses

func (p *BaseProcess) ParamInPorts() map[string]*ParamInPort

ParamInPorts returns all parameter in-ports of the process

func (*BaseProcess) ParamOutPort Uses

func (p *BaseProcess) ParamOutPort(portName string) *ParamOutPort

ParamOutPort returns the parameter port with name portName

func (*BaseProcess) ParamOutPorts Uses

func (p *BaseProcess) ParamOutPorts() map[string]*ParamOutPort

ParamOutPorts returns all parameter out-ports of the process

func (*BaseProcess) Workflow Uses

func (p *BaseProcess) Workflow() *Workflow

Workflow returns the workflow the process is connected to

type FileIP Uses

type FileIP struct {
    *BaseIP

    SubStream *InPort
    // contains filtered or unexported fields
}

FileIP (Short for "Information Packet" in Flow-Based Programming terminology) contains information and helper methods for a physical file on a normal disk.

func NewFileIP Uses

func NewFileIP(path string) *FileIP

NewFileIP creates a new FileIP

func (*FileIP) AddKey Uses

func (ip *FileIP) AddKey(k string, v string)

AddKey adds the key k with value v

func (*FileIP) AddKeys Uses

func (ip *FileIP) AddKeys(keys map[string]string)

AddKeys adds a map of keys to the IPs audit info

func (*FileIP) Atomize Uses

func (ip *FileIP) Atomize()

Atomize renames the temporary file name to the final file name, thus enabling to separate unfinished, and finished files

func (*FileIP) AuditFilePath Uses

func (ip *FileIP) AuditFilePath() string

AuditFilePath returns the file path of the audit info file for the FileIP

func (*FileIP) AuditInfo Uses

func (ip *FileIP) AuditInfo() *AuditInfo

AuditInfo returns the AuditInfo struct for the FileIP

func (*FileIP) CreateFifo Uses

func (ip *FileIP) CreateFifo()

CreateFifo creates a FIFO file for the FileIP

func (*FileIP) Exists Uses

func (ip *FileIP) Exists() bool

Exists checks if the file exists (at its final file name)

func (*FileIP) FifoFileExists Uses

func (ip *FileIP) FifoFileExists() bool

FifoFileExists checks if the FIFO-file (named pipe file) exists

func (*FileIP) FifoPath Uses

func (ip *FileIP) FifoPath() string

FifoPath returns the path to use when a FIFO file is used instead of a normal file

func (*FileIP) Key Uses

func (ip *FileIP) Key(k string) string

Key returns the key value for the key with key k from the IPs audit info (don't confuse this with the keys of maps in go. Keys in this case is a SciPipe audit info concept)

func (*FileIP) Keys Uses

func (ip *FileIP) Keys() map[string]string

Keys returns the audit info's key values

func (*FileIP) Open Uses

func (ip *FileIP) Open() *os.File

Open opens the file and returns a file handle (*os.File)

func (*FileIP) OpenTemp Uses

func (ip *FileIP) OpenTemp() *os.File

OpenTemp opens the temp file and returns a file handle (*os.File)

func (*FileIP) OpenWriteTemp Uses

func (ip *FileIP) OpenWriteTemp() *os.File

OpenWriteTemp opens the file for writing, and returns a file handle (*os.File)

func (*FileIP) Param Uses

func (ip *FileIP) Param(key string) string

Param returns the parameter named key, from the IPs audit info

func (*FileIP) Path Uses

func (ip *FileIP) Path() string

Path returns the (final) path of the physical file

func (*FileIP) Read Uses

func (ip *FileIP) Read() []byte

Read reads the whole content of the file and returns the content as a byte array

func (*FileIP) RemoveFifo Uses

func (ip *FileIP) RemoveFifo()

RemoveFifo removes the FIFO file, if it exists

func (*FileIP) SetAuditInfo Uses

func (ip *FileIP) SetAuditInfo(ai *AuditInfo)

SetAuditInfo sets the AuditInfo struct for the FileIP

func (*FileIP) Size Uses

func (ip *FileIP) Size() int64

Size returns the size of an existing file, in bytes

func (*FileIP) TempFileExists Uses

func (ip *FileIP) TempFileExists() bool

TempFileExists checks if the temp-file exists

func (*FileIP) TempPath Uses

func (ip *FileIP) TempPath() string

TempPath returns the temporary path of the physical file

func (*FileIP) UnMarshalJSON Uses

func (ip *FileIP) UnMarshalJSON(v interface{})

UnMarshalJSON is a helper function to unmarshal the content of the IPs file to the interface v

func (*FileIP) Write Uses

func (ip *FileIP) Write(dat []byte)

Write writes a byte array ([]byte) to the file's temp file path

func (*FileIP) WriteAuditLogToFile Uses

func (ip *FileIP) WriteAuditLogToFile()

WriteAuditLogToFile writes the audit log to its designated file

type IP Uses

type IP interface {
    ID() string
    Atomize()
}

IP Is the base interface which all other IPs need to adhere to

type InPort Uses

type InPort struct {
    Chan chan *FileIP

    RemotePorts map[string]*OutPort
    // contains filtered or unexported fields
}

InPort represents a pluggable connection to multiple out-ports from other processes, from its own process, and with which it is communicating via channels under the hood

func NewInPort Uses

func NewInPort(name string) *InPort

NewInPort returns a new InPort struct

func (*InPort) AddRemotePort Uses

func (pt *InPort) AddRemotePort(rpt *OutPort)

AddRemotePort adds a remote OutPort to the InPort

func (*InPort) CloseConnection Uses

func (pt *InPort) CloseConnection(rptName string)

CloseConnection closes the connection to the remote out-port with name rptName, on the InPort

func (*InPort) Connect Uses

func (pt *InPort) Connect(rpt *OutPort)

Connect connects an OutPort to the InPort

func (*InPort) Connected Uses

func (pt *InPort) Connected() bool

Connected tells whether the port is connected or not

func (*InPort) Disconnect Uses

func (pt *InPort) Disconnect(rptName string)

Disconnect disconnects the (out-)port with name rptName, from the InPort

func (*InPort) Name Uses

func (pt *InPort) Name() string

Name returns the name of the InPort

func (*InPort) Process Uses

func (pt *InPort) Process() WorkflowProcess

Process returns the process connected to the port

func (*InPort) Recv Uses

func (pt *InPort) Recv() *FileIP

Recv receives IPs from the port

func (*InPort) Send Uses

func (pt *InPort) Send(ip *FileIP)

Send sends IPs to the in-port, and is supposed to be called from the remote (out-) port, to send to this in-port

func (*InPort) SetConnectedStatus Uses

func (pt *InPort) SetConnectedStatus(connected bool)

SetConnectedStatus sets the connected status of the InPort

func (*InPort) SetProcess Uses

func (pt *InPort) SetProcess(p WorkflowProcess)

SetProcess sets the process of the port to p

type OutPort Uses

type OutPort struct {
    RemotePorts map[string]*InPort
    // contains filtered or unexported fields
}

OutPort represents a pluggable connection to multiple in-ports from other processes, from its own process, and with which it is communicating via channels under the hood

func NewOutPort Uses

func NewOutPort(name string) *OutPort

NewOutPort returns a new OutPort struct

func (*OutPort) AddRemotePort Uses

func (pt *OutPort) AddRemotePort(rpt *InPort)

AddRemotePort adds a remote InPort to the OutPort

func (*OutPort) Close Uses

func (pt *OutPort) Close()

Close closes the connection between this port and all the ports it is connected to. If this port is the last connected port to an in-port, that in-ports channel will also be closed.

func (*OutPort) Connect Uses

func (pt *OutPort) Connect(rpt *InPort)

Connect connects an InPort to the OutPort

func (*OutPort) Connected Uses

func (pt *OutPort) Connected() bool

Connected tells whether the port is connected or not

func (*OutPort) Disconnect Uses

func (pt *OutPort) Disconnect(rptName string)

Disconnect disconnects the (in-)port with name rptName, from the OutPort

func (*OutPort) Name Uses

func (pt *OutPort) Name() string

Name returns the name of the OutPort

func (*OutPort) Process Uses

func (pt *OutPort) Process() WorkflowProcess

Process returns the process connected to the port

func (*OutPort) Send Uses

func (pt *OutPort) Send(ip *FileIP)

Send sends an FileIP to all the in-ports connected to the OutPort

func (*OutPort) SetConnectedStatus Uses

func (pt *OutPort) SetConnectedStatus(connected bool)

SetConnectedStatus sets the connected status of the OutPort

func (*OutPort) SetProcess Uses

func (pt *OutPort) SetProcess(p WorkflowProcess)

SetProcess sets the process of the port to p

type ParamInPort Uses

type ParamInPort struct {
    Chan chan string

    RemotePorts map[string]*ParamOutPort
    // contains filtered or unexported fields
}

ParamInPort is an in-port for parameter values of string type

func NewParamInPort Uses

func NewParamInPort(name string) *ParamInPort

NewParamInPort returns a new ParamInPort

func (*ParamInPort) AddRemotePort Uses

func (pip *ParamInPort) AddRemotePort(pop *ParamOutPort)

AddRemotePort adds a remote ParamOutPort to the ParamInPort

func (*ParamInPort) CloseConnection Uses

func (pip *ParamInPort) CloseConnection(popName string)

CloseConnection closes the connection to the remote out-port with name popName, on the ParamInPort

func (*ParamInPort) Connect Uses

func (pip *ParamInPort) Connect(pop *ParamOutPort)

Connect connects one parameter port with another one

func (*ParamInPort) ConnectStr Uses

func (pip *ParamInPort) ConnectStr(strings ...string)

ConnectStr connects a parameter port with a new go-routine feeding the strings in strings, on the fly, to the parameter port

func (*ParamInPort) Connected Uses

func (pip *ParamInPort) Connected() bool

Connected tells whether the port is connected or not

func (*ParamInPort) Name Uses

func (pip *ParamInPort) Name() string

Name returns the name of the ParamInPort

func (*ParamInPort) Process Uses

func (pip *ParamInPort) Process() WorkflowProcess

Process returns the process that is connected to the port

func (*ParamInPort) Recv Uses

func (pip *ParamInPort) Recv() string

Recv receiveds a param value over the ports connection

func (*ParamInPort) Send Uses

func (pip *ParamInPort) Send(param string)

Send sends IPs to the in-port, and is supposed to be called from the remote (out-) port, to send to this in-port

func (*ParamInPort) SetConnectedStatus Uses

func (pip *ParamInPort) SetConnectedStatus(connected bool)

SetConnectedStatus sets the connected status of the ParamInPort

func (*ParamInPort) SetProcess Uses

func (pip *ParamInPort) SetProcess(p WorkflowProcess)

SetProcess sets the process of the port to p

type ParamOutPort Uses

type ParamOutPort struct {
    RemotePorts map[string]*ParamInPort
    // contains filtered or unexported fields
}

ParamOutPort is an out-port for parameter values of string type

func NewParamOutPort Uses

func NewParamOutPort(name string) *ParamOutPort

NewParamOutPort returns a new ParamOutPort

func (*ParamOutPort) AddRemotePort Uses

func (pop *ParamOutPort) AddRemotePort(pip *ParamInPort)

AddRemotePort adds a remote ParamInPort to the ParamOutPort

func (*ParamOutPort) Close Uses

func (pop *ParamOutPort) Close()

Close closes the connection between this port and all the ports it is connected to. If this port is the last connected port to an in-port, that in-ports channel will also be closed.

func (*ParamOutPort) Connect Uses

func (pop *ParamOutPort) Connect(pip *ParamInPort)

Connect connects an ParamInPort to the ParamOutPort

func (*ParamOutPort) Connected Uses

func (pop *ParamOutPort) Connected() bool

Connected tells whether the port is connected or not

func (*ParamOutPort) Disconnect Uses

func (pop *ParamOutPort) Disconnect(pipName string)

Disconnect disonnects the (in-)port with name rptName, from the ParamOutPort

func (*ParamOutPort) Name Uses

func (pop *ParamOutPort) Name() string

Name returns the name of the ParamOutPort

func (*ParamOutPort) Process Uses

func (pop *ParamOutPort) Process() WorkflowProcess

Process returns the process that is connected to the port

func (*ParamOutPort) Send Uses

func (pop *ParamOutPort) Send(param string)

Send sends an FileIP to all the in-ports connected to the ParamOutPort

func (*ParamOutPort) SetConnectedStatus Uses

func (pop *ParamOutPort) SetConnectedStatus(connected bool)

SetConnectedStatus sets the connected status of the ParamOutPort

func (*ParamOutPort) SetProcess Uses

func (pop *ParamOutPort) SetProcess(p WorkflowProcess)

SetProcess sets the process of the port to p

type Process Uses

type Process struct {
    BaseProcess
    CommandPattern   string
    OutPortsDoStream map[string]bool
    PathFormatters   map[string]func(*Task) string
    CustomExecute    func(*Task)
    CoresPerTask     int
    Prepend          string
    Spawn            bool
}

Process is the central component in SciPipe after Workflow. Processes are long-running "services" that schedules and executes Tasks based on the IPs and parameters received on its in-ports and parameter ports

func NewProc Uses

func NewProc(workflow *Workflow, name string, cmd string) *Process

NewProc returns a new Process, and initializes its ports based on the command pattern.

func (*Process) In Uses

func (p *Process) In(portName string) *InPort

In is a short-form for InPort() (of BaseProcess), which works only on Process processes

func (*Process) Out Uses

func (p *Process) Out(portName string) *OutPort

Out is a short-form for OutPort() (of BaseProcess), which works only on Process processes

func (*Process) Run Uses

func (p *Process) Run()

Run runs the process by instantiating and executing Tasks for all inputs and parameter values on its in-ports. in the case when there are no inputs or parameter values on the in-ports, it will run just once before it terminates. note that the actual execution of shell commands are done inside Task.Execute, not here.

func (*Process) SetPathCustom Uses

func (p *Process) SetPathCustom(outPortName string, pathFmtFunc func(task *Task) (path string))

SetPathCustom takes a function which produces a file path based on data available in *Task, such as concrete file paths and parameter values,

func (*Process) SetPathExtend Uses

func (p *Process) SetPathExtend(inPortName string, outPortName string, extension string)

SetPathExtend creates an (output) path formatter that extends the path of an input IP

func (*Process) SetPathReplace Uses

func (p *Process) SetPathReplace(inPortName string, outPortName string, old string, new string)

SetPathReplace creates an (output) path formatter that uses an input's path but replaces parts of it.

func (*Process) SetPathStatic Uses

func (p *Process) SetPathStatic(outPortName string, path string)

SetPathStatic creates an (output) path formatter returning a static string file name

type Sink Uses

type Sink struct {
    BaseProcess
}

Sink is a simple component that just receives IPs on its In-port without doing anything with them. It is used to drive pipelines of processes

func NewSink Uses

func NewSink(wf *Workflow, name string) *Sink

NewSink returns a new Sink component

func (*Sink) Connect Uses

func (p *Sink) Connect(outPort *OutPort)

Connect connects an out-port to the sinks in-port

func (*Sink) ConnectParam Uses

func (p *Sink) ConnectParam(paramOutPort *ParamOutPort)

ConnectParam connects a param-out-port to the sinks param-in-port

func (*Sink) Run Uses

func (p *Sink) Run()

Run runs the Sink process

type Task Uses

type Task struct {
    Name          string
    Command       string
    CustomExecute func(*Task)
    InIPs         map[string]*FileIP
    OutIPs        map[string]*FileIP
    Params        map[string]string
    Done          chan int
    // contains filtered or unexported fields
}

Task represents a single static shell command, or go function, to be executed, and are scheduled and managed by a corresponding Process

func NewTask Uses

func NewTask(workflow *Workflow, process *Process, name string, cmdPat string, inIPs map[string]*FileIP, outPathFuncs map[string]func(*Task) string, outPortsDoStream map[string]bool, params map[string]string, prepend string, customExecute func(*Task), cores int) *Task

NewTask instantiates and initializes a new Task

func (*Task) Execute Uses

func (t *Task) Execute()

Execute executes the task (the shell command or go function in CustomExecute)

func (*Task) InIP Uses

func (t *Task) InIP(portName string) *FileIP

InIP returns an IP for the in-port with name portName

func (*Task) InPath Uses

func (t *Task) InPath(portName string) string

InPath returns the path name of an input file for the task

func (*Task) OutIP Uses

func (t *Task) OutIP(portName string) *FileIP

OutIP returns an IP for the in-port with name portName

func (*Task) OutPath Uses

func (t *Task) OutPath(portName string) string

OutPath returns the path name of an input file for the task

func (*Task) Param Uses

func (t *Task) Param(portName string) string

Param returns the value of a param, for the task

type Workflow Uses

type Workflow struct {
    // contains filtered or unexported fields
}

Workflow is the centerpiece of the functionality in SciPipe, and is a container for a pipeline of processes making up a workflow. It has various methods for coordination the execution of the pipeline as a whole, such as keeping track of the maxiumum number of concurrent tasks, as well as helper methods for creating new processes, that automatically gets plugged in to the workflow on creation

func NewWorkflow Uses

func NewWorkflow(name string, maxConcurrentTasks int) *Workflow

NewWorkflow returns a new Workflow

func NewWorkflowCustomLogFile Uses

func NewWorkflowCustomLogFile(name string, maxConcurrentTasks int, logFile string) *Workflow

NewWorkflowCustomLogFile returns a new Workflow, with

func (*Workflow) AddProc Uses

func (wf *Workflow) AddProc(proc WorkflowProcess)

AddProc adds a Process to the workflow, to be run when the workflow runs

func (*Workflow) AddProcs Uses

func (wf *Workflow) AddProcs(procs ...WorkflowProcess)

AddProcs takes one or many Processes and adds them to the workflow, to be run when the workflow runs.

func (*Workflow) DecConcurrentTasks Uses

func (wf *Workflow) DecConcurrentTasks(slots int)

DecConcurrentTasks decreases the conter for how many concurrent tasks are currently running in the workflow

func (*Workflow) IncConcurrentTasks Uses

func (wf *Workflow) IncConcurrentTasks(slots int)

IncConcurrentTasks increases the conter for how many concurrent tasks are currently running in the workflow

func (*Workflow) Name Uses

func (wf *Workflow) Name() string

Name returns the name of the workflow

func (*Workflow) NewProc Uses

func (wf *Workflow) NewProc(procName string, commandPattern string) *Process

NewProc returns a new process based on a commandPattern (See the documentation for scipipe.NewProcess for more details about the pattern) and connects the process to the workflow

func (*Workflow) Proc Uses

func (wf *Workflow) Proc(procName string) WorkflowProcess

Proc returns the process with name procName from the workflow

func (*Workflow) Procs Uses

func (wf *Workflow) Procs() map[string]WorkflowProcess

Procs returns a map of all processes keyed by their names in the workflow

func (*Workflow) Run Uses

func (wf *Workflow) Run()

Run runs all the processes of the workflow

func (*Workflow) RunTo Uses

func (wf *Workflow) RunTo(finalProcNames ...string)

RunTo runs all processes upstream of, and including, the process with names provided as arguments

func (*Workflow) RunToProcs Uses

func (wf *Workflow) RunToProcs(finalProcs ...WorkflowProcess)

RunToProcs runs all processes upstream of, and including, the process strucs provided as arguments

func (*Workflow) RunToRegex Uses

func (wf *Workflow) RunToRegex(procNamePatterns ...string)

RunToRegex runs all processes upstream of, and including, the process whose name matches any of the provided regexp patterns

func (*Workflow) SetSink Uses

func (wf *Workflow) SetSink(sink *Sink)

SetSink sets the sink of the workflow to the provided sink process

func (*Workflow) Sink Uses

func (wf *Workflow) Sink() *Sink

Sink returns the sink process of the workflow

type WorkflowProcess Uses

type WorkflowProcess interface {
    Name() string
    InPorts() map[string]*InPort
    OutPorts() map[string]*OutPort
    ParamInPorts() map[string]*ParamInPort
    ParamOutPorts() map[string]*ParamOutPort
    Connected() bool
    Run()
}

WorkflowProcess is an interface for processes to be handled by Workflow

Directories

PathSynopsis
components

Package scipipe imports 15 packages (graph) and is imported by 2 packages. Updated 2018-05-28. Refresh now. Tools for package owners.