cwl

package module
v0.0.0-...-c4d1d10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 19, 2018 License: MIT Imports: 13 Imported by: 0

README

Godoc

This is a Go library and command line tool for working with the common workflow language (CWL).

The core library includes the data type present in the CWL spec (CommandLineTool, Workflow, etc.), with a few extra utilities for loading and working with CWL documents.

For details, see the reference docs.

Subpackages

The process library contains experimental, unfinished code for processing CWL documents in order to execute commands and workflows.

The expr library contains utilities for parsing CWL expressions out of strings. This parser is not yet robust (see the known issues below).

Alpha quality

At the time of this writing, this library is only a couple weeks old. I feel that the core CWL document loading library is fairly stable, but I can't promise that there aren't plenty of bugs lurking.

The command line tool is far from stable, and needs work before becoming useful.

The process library is highly experimental. Processing CWL tools and workflows in a robust manner is not a trivial task.

CLI

The cwl command line tool includes a few commands for loading, inspecting, and experimental support for running commands. See the releases page to download the command line tool binary.

Usage (CLI)

The command line tool is still young and therefore fairly useless. Still:

cwl dump https://raw.githubusercontent.com/buchanae/cwl/master/examples/array-inputs.cwl
// ...outputs the normalized document in JSON.

cwl run exists and is experimental. This command will run a CWL document, similar cwltool.

Usage (library)

package main

import (
  "log"
  "github.com/buchanae/cwl"
)

func main() {
  // Load a CWL CommandLineTool document.
  path := "https://raw.githubusercontent.com/buchanae/cwl/master/examples/array-inputs.cwl"
  doc, err := cwl.Load(path)
  if err != nil {
    log.Fatal(err)
  }
  
  tool := doc.(*cwl.Tool)
  
  // Print the ID of all the input fields
  for _, input := range tool.Inputs {
    log.Println(input.ID)
  }
}

go run main.go
2018/03/12 18:16:22 filesA
2018/03/12 18:16:22 filesB
2018/03/12 18:16:22 filesC

Normalization

The CWL spec allows multiple different types for some fields, e.g. CommandLineTool.inputs may be a string, a list of strings, a map of string to string, a map of string to object, and so on. This is rather difficult to program against, especially in a statically typed language without generics or union types (i.e. Go).

This library normalizes all fields to a single type, often prefering a list where a string and map might be allowed. In the example above, CommandLineTool.inputs is a list.

Similarly, many fields might be a CWL expression. In this library, any field which might be an expression has the type Expression.

Notable changes and known issues

I've taken some liberties with the CWL spec:

  • CommandLineTool is named Tool instead, for brevity.
  • Schema Salad is not implemented and likely won't be implemented.
  • $include and $import statements are not yet implemented, but will be.
  • The CWL expression parser is not robust and will not correctly parse complex expressions, especially those containing $() and escaping.
  • documentation and examples are still sparse, more on the way soon.

Documentation

Index

Constants

View Source 
const (
	DotProduct         ScatterMethod = "dotproduct"
	NestedCrossProduct               = "nested_crossproduct"
	FlatCrossProduct                 = "flat_crossproduct"
)

Variables

This section is empty.

Functions

func ValidateTool 

func ValidateTool(tool *Tool) error

Types

type Any 

type Any struct{}

func (Any) MarshalText 

func (Any) MarshalText() ([]byte, error)

func (Any) String 

func (Any) String() string

type Boolean 

type Boolean struct{}

func (Boolean) MarshalText 

func (Boolean) MarshalText() ([]byte, error)

func (Boolean) String 

func (Boolean) String() string

type CommandInput 

type CommandInput struct {
	ID         string `json:"id,omitempty"`
	Label      string `json:"label,omitempty"`
	Doc        string `json:"doc,omitempty"`
	Streamable bool   `json:"streamable,omitempty"`
	Default    Value  `json:"default,omitempty"`

	Type []InputType `json:"type,omitempty"`

	SecondaryFiles []Expression `json:"secondaryFiles,omitempty"`
	Format         []Expression `json:"format,omitempty"`

	InputBinding *CommandLineBinding `json:"inputBinding,omitempty"`
}

type CommandLineBinding 

type CommandLineBinding struct {
	LoadContents  bool       `json:"loadContents,omitempty"`
	Position      int        `json:"position,omitempty"`
	Prefix        string     `json:"prefix,omitempty"`
	ItemSeparator string     `json:"itemSeparator,omitempty"`
	ValueFrom     Expression `json:"valueFrom,omitempty"`
	Separate      OptOut     `json:"separate,omitempty"`
	ShellQuote    OptOut     `json:"shellQuote,omitempty"`
}

func (*CommandLineBinding) GetItemSeparator 

func (clb *CommandLineBinding) GetItemSeparator() string

func (*CommandLineBinding) GetLoadContents 

func (clb *CommandLineBinding) GetLoadContents() bool

func (*CommandLineBinding) GetPosition 

func (clb *CommandLineBinding) GetPosition() int

func (*CommandLineBinding) GetPrefix 

func (clb *CommandLineBinding) GetPrefix() string

func (*CommandLineBinding) GetValueFrom 

func (clb *CommandLineBinding) GetValueFrom() Expression

type CommandOutput 

type CommandOutput struct {
	ID         string `json:"id,omitempty"`
	Label      string `json:"label,omitempty"`
	Doc        string `json:"doc,omitempty"`
	Streamable bool   `json:"streamable,omitempty"`

	Type []OutputType `json:"type,omitempty"`

	SecondaryFiles []Expression `json:"secondaryFiles,omitempty"`
	Format         []Expression `json:"format,omitempty"`

	OutputBinding *CommandOutputBinding `json:"outputBinding,omitempty"`
}

type CommandOutputBinding 

type CommandOutputBinding struct {
	Glob         []Expression `json:"glob,omitempty"`
	LoadContents bool         `json:"loadContents,omitempty"`
	OutputEval   Expression   `json:"outputEval,omitempty"`
}

type DefaultResolver 

type DefaultResolver struct{}

DefaultResolver is a document location resolver which resolves local file paths.

func (DefaultResolver) Resolve 

func (DefaultResolver) Resolve(base, loc string) ([]byte, string, error)

type Directory 

type Directory struct {
	Location string    `json:"location,omitempty"`
	Path     string    `json:"path,omitempty"`
	Basename string    `json:"basename,omitempty"`
	Listing  []FileDir `json:"listing,omitempty"`
}

func (Directory) MarshalJSON 

func (i Directory) MarshalJSON() ([]byte, error)

type DirectoryType 

type DirectoryType struct{}

func (DirectoryType) MarshalText 

func (DirectoryType) MarshalText() ([]byte, error)

func (DirectoryType) String 

func (DirectoryType) String() string

type Dirent 

type Dirent struct {
	Entry     Expression `json:"entry,omitempty"`
	Entryname Expression `json:"entryname,omitempty"`
	Writable  bool       `json:"writeable,omitempty"`
}

type DockerRequirement 

type DockerRequirement struct {
	Pull            string `json:"dockerPull,omitempty"`
	Load            string `json:"dockerLoad,omitempty"`
	File            string `json:"dockerFile,omitempty"`
	Import          string `json:"dockerImport,omitempty"`
	ImageID         string `json:"dockerImageID,omitempty"`
	OutputDirectory string `json:"dockerOutputDirectory,omitempty"`
}

func (DockerRequirement) MarshalJSON 

func (x DockerRequirement) MarshalJSON() ([]byte, error)

type Document 

type Document interface {
	Doctype() string
}

func Load 

func Load(loc string) (Document, error)

func LoadDocumentBytes 

func LoadDocumentBytes(b []byte, base string, r Resolver) (Document, error)

func LoadWithResolver 

func LoadWithResolver(loc string, r Resolver) (Document, error)

type DocumentRef 

type DocumentRef struct {
	Location string
}

func (DocumentRef) Doctype 

func (DocumentRef) Doctype() string

func (DocumentRef) MarshalText 

func (d DocumentRef) MarshalText() ([]byte, error)

type Double 

type Double struct{}

func (Double) MarshalText 

func (Double) MarshalText() ([]byte, error)

func (Double) String 

func (Double) String() string

type EnvVarRequirement 

type EnvVarRequirement struct {
	EnvDef map[string]Expression `json:"envDef,omitempty"`
}

func (EnvVarRequirement) MarshalJSON 

func (x EnvVarRequirement) MarshalJSON() ([]byte, error)

type Expression 

type Expression string

type ExpressionTool 

type ExpressionTool struct {
	CWLVersion string `json:"cwlVersion,omitempty"`
	Class      string `json:"class,omitempty"`

	ID    string `json:"id,omitempty"`
	Label string `json:"label,omitempty"`
	Doc   string `json:"doc,omitempty"`

	Hints        []Requirement `json:"hints,omitempty"`
	Requirements []Requirement `json:"requirements,omitempty"`

	Inputs  []CommandInput  `json:"inputs,omitempty"`
	Outputs []CommandOutput `json:"outputs,omitempty"`

	Expression Expression `json:"expression,omitempty"`
}

func (ExpressionTool) Doctype 

func (ExpressionTool) Doctype() string

type File 

type File struct {
	Location       string    `json:"location,omitempty"`
	Path           string    `json:"path,omitempty"`
	Basename       string    `json:"basename,omitempty"`
	Dirname        string    `json:"dirname,omitempty"`
	Nameroot       string    `json:"nameroot,omitempty"`
	Nameext        string    `json:"nameext,omitempty"`
	Checksum       string    `json:"checksum,omitempty"`
	Size           int64     `json:"size"`
	Format         string    `json:"format,omitempty"`
	Contents       string    `json:"contents,omitempty"`
	SecondaryFiles []FileDir `json:"secondaryFiles,omitempty"`
}

func (File) MarshalJSON 

func (i File) MarshalJSON() ([]byte, error)

type FileDir 

type FileDir interface {
	// contains filtered or unexported methods
}

type FileType 

type FileType struct{}

func (FileType) MarshalText 

func (FileType) MarshalText() ([]byte, error)

func (FileType) String 

func (FileType) String() string

type Float 

type Float struct{}

func (Float) MarshalText 

func (Float) MarshalText() ([]byte, error)

func (Float) String 

func (Float) String() string

type Graph 

type Graph struct {
	CWLVersion string     `json:"cwlVersion,omitempty"`
	Docs       []Document `json:"$graph"`
}

func (Graph) Doctype 

func (Graph) Doctype() string

type InitialWorkDirListing 

type InitialWorkDirListing struct{}

type InitialWorkDirRequirement 

type InitialWorkDirRequirement struct {
	// TODO the most difficult union type
	Listing InitialWorkDirListing `json:"listing,omitempty"`
}

func (InitialWorkDirRequirement) MarshalJSON 

func (x InitialWorkDirRequirement) MarshalJSON() ([]byte, error)

type InlineJavascriptRequirement 

type InlineJavascriptRequirement struct {
	ExpressionLib []string `json:"expressionLib,omitempty"`
}

func (InlineJavascriptRequirement) MarshalJSON 

func (x InlineJavascriptRequirement) MarshalJSON() ([]byte, error)

type InputArray 

type InputArray struct {
	Label        string              `json:"label,omitempty"`
	Items        []InputType         `json:"items,omitempty"`
	InputBinding *CommandLineBinding `json:"inputBinding,omitempty"`
}

func (InputArray) MarshalJSON 

func (i InputArray) MarshalJSON() ([]byte, error)

func (InputArray) String 

func (InputArray) String() string

type InputEnum 

type InputEnum struct {
	Label        string              `json:"label,omitempty"`
	Symbols      []string            `json:"symbols,omitempty"`
	InputBinding *CommandLineBinding `json:"inputBinding,omitempty"`
}

func (InputEnum) String 

func (InputEnum) String() string

type InputField 

type InputField struct {
	Name         string              `json:"name,omitempty"`
	Doc          string              `json:"doc,omitempty"`
	Label        string              `json:"label,omitempty"`
	Type         []InputType         `json:"type,omitempty"`
	InputBinding *CommandLineBinding `json:"inputBinding,omitempty"`
}

type InputRecord 

type InputRecord struct {
	Label  string       `json:"label,omitempty"`
	Fields []InputField `json:"fields,omitempty"`
}

func (InputRecord) MarshalJSON 

func (i InputRecord) MarshalJSON() ([]byte, error)

func (InputRecord) String 

func (InputRecord) String() string

type InputType 

type InputType interface {
	String() string
	// contains filtered or unexported methods
}

type Int 

type Int struct{}

func (Int) MarshalText 

func (Int) MarshalText() ([]byte, error)

func (Int) String 

func (Int) String() string

type LinkMergeMethod 

type LinkMergeMethod string
const (
	MergeNested    LinkMergeMethod = "merge_nested"
	MergeFlattened                 = "merge_flattened"
)

type Long 

type Long struct{}

func (Long) MarshalText 

func (Long) MarshalText() ([]byte, error)

func (Long) String 

func (Long) String() string

type MultipleInputFeatureRequirement 

type MultipleInputFeatureRequirement struct {
}

func (MultipleInputFeatureRequirement) MarshalJSON 

func (x MultipleInputFeatureRequirement) MarshalJSON() ([]byte, error)

type Null 

type Null struct{}

func (Null) MarshalText 

func (Null) MarshalText() ([]byte, error)

func (Null) String 

func (Null) String() string

type OptOut 

type OptOut struct {
	// contains filtered or unexported fields
}

OptOut provides a boolean flag that defaults to true.

func (*OptOut) Clear 

func (o *OptOut) Clear()

func (*OptOut) MarshalText 

func (o *OptOut) MarshalText() ([]byte, error)

func (*OptOut) Set 

func (o *OptOut) Set(v bool)

func (*OptOut) Value 

func (o *OptOut) Value() bool

type OutputArray 

type OutputArray struct {
	Label         string                `json:"label,omitempty"`
	Items         []OutputType          `json:"items,omitempty"`
	OutputBinding *CommandOutputBinding `json:"outputBinding,omitempty"`
}

func (OutputArray) MarshalJSON 

func (i OutputArray) MarshalJSON() ([]byte, error)

func (OutputArray) String 

func (OutputArray) String() string

type OutputEnum 

type OutputEnum struct {
	Label         string                `json:"label,omitempty"`
	Symbols       []string              `json:"symbols,omitempty"`
	OutputBinding *CommandOutputBinding `json:"outputBinding,omitempty"`
}

func (OutputEnum) String 

func (OutputEnum) String() string

type OutputField 

type OutputField struct {
	Name          string                `json:"name,omitempty"`
	Doc           string                `json:"doc,omitempty"`
	Type          []OutputType          `json:"type,omitempty"`
	OutputBinding *CommandOutputBinding `json:"outputBinding,omitempty"`
}

type OutputRecord 

type OutputRecord struct {
	Label  string        `json:"label,omitempty"`
	Fields []OutputField `json:"fields,omitempty"`
}

func (OutputRecord) MarshalJSON 

func (i OutputRecord) MarshalJSON() ([]byte, error)

func (OutputRecord) String 

func (OutputRecord) String() string

type OutputType 

type OutputType interface {
	String() string
	// contains filtered or unexported methods
}

type Requirement 

type Requirement interface {
	// contains filtered or unexported methods
}

type Resolver 

type Resolver interface {
	// Resolve is given the current `base`,
	// such as the directory of the root document,
	// and the `location` referenced by the CWL document.
	//
	// Upon success, the document bytes and the new `base`
	// should be returned.
	Resolve(base, location string) (doc []byte, newBase string, err error)
}

Resolver describes a type which resolves docment by location, such as loading relative file paths referenced in a CWL document.

func NoResolve 

func NoResolve() Resolver

NoResolve is a special case resolver which does not resolve documents, but instead creates `DocumentRef` instances in the document tree.

type ResourceRequirement 

type ResourceRequirement struct {
	CoresMin  Expression `json:"coresMin,omitempty"`
	CoresMax  Expression `json:"coresMax,omitempty"`
	RAMMin    Expression `json:"ramMin,omitempty"`
	RAMMax    Expression `json:"ramMax,omitempty"`
	TmpDirMin Expression `json:"tmpdirMin,omitempty"`
	TmpDirMax Expression `json:"tmpdirMax,omitempty"`
	OutDirMin Expression `json:"outdirMin,omitempty"`
	OutDirMax Expression `json:"outdirMax,omitempty"`
}

func (ResourceRequirement) MarshalJSON 

func (x ResourceRequirement) MarshalJSON() ([]byte, error)

type ScatterFeatureRequirement 

type ScatterFeatureRequirement struct {
}

func (ScatterFeatureRequirement) MarshalJSON 

func (x ScatterFeatureRequirement) MarshalJSON() ([]byte, error)

type ScatterMethod 

type ScatterMethod string

type SchemaDef 

type SchemaDef struct {
	Name string `json:"name,omitempty"`
	Type SchemaType
}

type SchemaDefRequirement 

type SchemaDefRequirement struct {
	Types []SchemaDef `json:"types,omitempty"`
}

func (SchemaDefRequirement) MarshalJSON 

func (x SchemaDefRequirement) MarshalJSON() ([]byte, error)

type SchemaType 

type SchemaType interface {
	// contains filtered or unexported methods
}

type ShellCommandRequirement 

type ShellCommandRequirement struct {
}

func (ShellCommandRequirement) MarshalJSON 

func (x ShellCommandRequirement) MarshalJSON() ([]byte, error)

type SoftwarePackage 

type SoftwarePackage struct {
	Package string   `json:"package,omitempty"`
	Version []string `json:"version,omitempty"`
	Specs   []string `json:"specs,omitempty"`
}

type SoftwareRequirement 

type SoftwareRequirement struct {
	Packages []SoftwarePackage `json:"packages,omitempty"`
}

func (SoftwareRequirement) MarshalJSON 

func (x SoftwareRequirement) MarshalJSON() ([]byte, error)

type Stderr 

type Stderr struct{}

func (Stderr) MarshalText 

func (Stderr) MarshalText() ([]byte, error)

func (Stderr) String 

func (Stderr) String() string

type Stdout 

type Stdout struct{}

func (Stdout) MarshalText 

func (Stdout) MarshalText() ([]byte, error)

func (Stdout) String 

func (Stdout) String() string

type Step 

type Step struct {
	ID    string `json:"id,omitempty"`
	Label string `json:"label,omitempty"`
	Doc   string `json:"doc,omitempty"`

	Hints        []Requirement `json:"hints,omitempty"`
	Requirements []Requirement `json:"requirements,omitempty"`

	In  []StepInput  `json:"in,omitempty"`
	Out []StepOutput `json:"out,omitempty"`

	Run Document `json:"run,omitempty"`

	Scatter       []string      `json:"scatter,omitempty"`
	ScatterMethod ScatterMethod `json:"scatterMethod,omitempty"`
}

type StepInput 

type StepInput struct {
	ID        string          `json:"id,omitempty"`
	Source    []string        `json:"source,omitempty"`
	LinkMerge LinkMergeMethod `json:"linkMerge,omitempty"`
	Default   Value           `json:"default,omitempty"`
	ValueFrom Expression      `json:"valueFrom,omitempty"`
}

type StepInputExpressionRequirement 

type StepInputExpressionRequirement struct {
}

func (StepInputExpressionRequirement) MarshalJSON 

func (x StepInputExpressionRequirement) MarshalJSON() ([]byte, error)

type StepOutput 

type StepOutput struct {
	ID string `json:"id,omitempty"`
}

type String 

type String struct{}

func (String) MarshalText 

func (String) MarshalText() ([]byte, error)

func (String) String 

func (String) String() string

type SubworkflowFeatureRequirement 

type SubworkflowFeatureRequirement struct {
}

func (SubworkflowFeatureRequirement) MarshalJSON 

func (x SubworkflowFeatureRequirement) MarshalJSON() ([]byte, error)

type Tool 

type Tool struct {
	CWLVersion string `json:"cwlVersion,omitempty"`
	ID         string `json:"id,omitempty"`
	Label      string `json:"label,omitempty"`
	Doc        string `json:"doc,omitempty"`

	Hints        []Requirement `json:"hints,omitempty"`
	Requirements []Requirement `json:"requirements,omitempty"`

	Inputs  []CommandInput  `json:"inputs,omitempty"`
	Outputs []CommandOutput `json:"outputs,omitempty"`

	BaseCommand []string              `json:"baseCommand,omitempty"`
	Arguments   []*CommandLineBinding `json:"arguments,omitempty"`

	Stdin  Expression `json:"stdin,omitempty"`
	Stderr Expression `json:"stderr,omitempty"`
	Stdout Expression `json:"stdout,omitempty"`

	SuccessCodes       []int `json:"successCodes,omitempty"`
	TemporaryFailCodes []int `json:",omitempty"`
	PermanentFailCodes []int `json:",omitempty"`
}

func (Tool) Doctype 

func (Tool) Doctype() string

func (Tool) MarshalJSON 

func (x Tool) MarshalJSON() ([]byte, error)

func (*Tool) RequiresDocker 

func (t *Tool) RequiresDocker() (*DockerRequirement, bool)

func (*Tool) RequiresInlineJavascript 

func (t *Tool) RequiresInlineJavascript() ([]string, bool)

func (*Tool) RequiresSchemaDef 

func (t *Tool) RequiresSchemaDef() (*SchemaDefRequirement, bool)

func (*Tool) RequiresShellCommand 

func (t *Tool) RequiresShellCommand() bool

func (*Tool) ResolveSchemaDefs 

func (t *Tool) ResolveSchemaDefs() error

type TypeRef 

type TypeRef struct {
	Name string
}

func (TypeRef) MarshalText 

func (t TypeRef) MarshalText() ([]byte, error)

func (TypeRef) String 

func (t TypeRef) String() string

type UnknownRequirement 

type UnknownRequirement struct {
	Name string
}

type Value 

type Value interface{}

type Values 

type Values map[string]Value

func LoadValuesBytes 

func LoadValuesBytes(b []byte) (Values, error)

func LoadValuesFile 

func LoadValuesFile(p string) (Values, error)

type Workflow 

type Workflow struct {
	CWLVersion string `json:"cwlVersion,omitempty"`
	ID         string `json:"id,omitempty"`
	Label      string `json:"label,omitempty"`
	Doc        string `json:"doc,omitempty"`

	Hints        []Requirement `json:"hints,omitempty"`
	Requirements []Requirement `json:"requirements,omitempty"`

	Inputs  []WorkflowInput  `json:"inputs,omitempty"`
	Outputs []WorkflowOutput `json:"outputs,omitempty"`
	Steps   []Step           `json:"steps,omitempty"`
}

func (Workflow) Doctype 

func (Workflow) Doctype() string

func (Workflow) MarshalJSON 

func (x Workflow) MarshalJSON() ([]byte, error)

type WorkflowInput 

type WorkflowInput struct {
	ID         string `json:"id,omitempty"`
	Label      string `json:"label,omitempty"`
	Doc        string `json:"doc,omitempty"`
	Streamable bool   `json:"streamable,omitempty"`
	Default    Value  `json:"default,omitempty"`

	Type []InputType `json:"type,omitempty"`

	SecondaryFiles []Expression `json:"secondaryFiles,omitempty"`
	Format         []Expression `json:"format,omitempty"`

	InputBinding *CommandLineBinding `json:"inputBinding,omitempty"`
}

TODO exactly the same as CommandInput? Changing in v1.1?

type WorkflowOutput 

type WorkflowOutput struct {
	ID         string          `json:"id,omitempty"`
	Label      string          `json:"label,omitempty"`
	Doc        string          `json:"doc,omitempty"`
	Streamable bool            `json:"streamable,omitempty"`
	LinkMerge  LinkMergeMethod `json:"linkMerge,omitempty"`

	Type           []OutputType `json:"type,omitempty"`
	SecondaryFiles []Expression `json:"secondaryFiles,omitempty"`
	Format         []Expression `json:"format,omitempty"`

	OutputBinding *CommandOutputBinding `json:"outputBinding,omitempty"`
	OutputSource  []string              `json:"outputSource,omitempty"`
}

type WorkflowRequirement 

type WorkflowRequirement interface {
	// contains filtered or unexported methods
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
cwl
env/simple
exec/simple
fs/local

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.