scriptish

package module
v1.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 8, 2019 License: BSD-3-Clause Imports: 15 Imported by: 8

README

Welcome to scriptish

Introduction

Scriptish is a Golang library. It helps you port UNIX shell scripts to Golang.

It is released under the 3-clause New BSD license. See LICENSE.md for details.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountLines(),
).Exec().ParseInt()

(We're going to create Scriptish for other languages too, and we'll update this README when those are available!)

Table of Contents

Why Use Scriptish?

Who Is Scriptish For?

We've built Scriptish for anyone who needs to replace UNIX shell scripts with compiled Golang binaries.

We're going to be doing that ourselves for some of our projects:

  • Dockhand - Docker management utility
  • HubFlow - the GitFlow extension for Git
  • SimpleCA - local TLS certificate authority for internal infrastructure

We'll add links to those projects when they're available.

Why UNIX Shell Scripts?

UNIX shell scripts are one of the most practical inventions in computer programming.

  • They're very quick to write.
  • They're very powerful.
  • They treat everything as text.

They're fantastic for knocking up utilities in a matter of minutes, for automating things, and for gluing things together. Our hard drives are littered with little shell scripts - and some large ones too! - and we create new ones all the time.

If you're using any sort of UNIX system (Linux, or MacOS), shell scripting is a must-have skill - whether you're a developer or a sysadmin.

Why Not Use UNIX Shell Scripts?

UNIX shell scripts are great until you want to share them with other people. They're just not a great choice if you want to distribute your work outside your team, organisation or community.

  • If someone else is going to run your shell scripts, they need to make sure that they've installed all the commands that your shell scripts call. This can end up being a trial-and-error process. And what happens if they can't install those commands for any reason?

  • Creating portable shell scripts (e.g. scripts that run on both Linux and MacOS) isn't always easy, and is very difficult (if not impossible) to test via a CI process.

  • What about your Windows users? UNIX shell scripts don't work on a vanilla Windows box.

Enter Scriptish

If you want to distribute shell scripts, it's best not to write them as shell scripts. Use Scriptish to quickly do the same thing in Golang:

  • There's one binary to ship to your users.
  • Scriptish is self-contained. No need to worry about installing additional commands (unless you call scriptish.Exec() ...)
  • Use Golang's go test to create tests for your tools.
  • Use the power of Golang to cross-compile binaries for Linux, MacOS and Windows.

How Does It Work?

Getting Started

Import Scriptish into your Golang code:

import scriptish "github.com/ganbarodigital/go_scriptish"

Create a pipeline, and provide it with a list of commands:

pipeline := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
)

Once you have your pipeline, run it:

pipeline.Exec()

Once you've run your pipeline, call one of the capture methods to find out what happened:

result, err := pipeline.ParseInt()
What Is A Pipeline?

UNIX shell scripts compose UNIX commands into a pipeline:

cat /path/to/file.txt | wc -w

The UNIX commands execute from left to right. The output (known as stdout) of each command becomes the input (known as stdin) of the next command.

The output of the final command can be captured by your shell script to become the value of a variable:

current_branch=$(git branch --no-color | grep "^[*] " | sed -e 's/^[*] //')

Scriptish works the same way. You create a pipeline of Scriptish commands:

pipeline := scriptish.NewPipeline(
    scriptish.Exec("git", "branch", "--no-color"),
    scriptish.Grep("^[* ]"),
    scriptish.Tr([]string{"* "}, []string{""}),
)

and then you run it:

pipeline.Exec()

The output of the final command can be captured by your Golang code to become the value of a variable, using capture methods:

current_branch, err := pipeline.TrimmedString()
What Happens When A Pipeline Runs?

UNIX commands in a pipeline:

  • read text input from stdin
  • write their results (as text!) to stdout
  • write any errors (as text!) out to stderr
  • return a status code to indicate what happened

Each Scriptish command works the same way:

  • they read text input from the pipeline's Stdin
  • they write their results to the pipeline's Stdout
  • they write any error messages out to the pipeline's Stderr
  • they return a status code and a Golang error to indicate what happened

When a single command has finished, its Stdout becomes the Stdin for the next command in the pipeline.

How Are Errors Handled?

One difference between UNIX commands and Golang is error handling. Scriptish combines the best of both.

  • UNIX commands return a status code to indicate what happened. A status code of 0 (zero) means success.
  • Scriptish commands return the UNIX-like status code, and any Golang error that has occurred. We store these in the pipeline.

If you're calling external commands using scriptish.Exec(), you've still got access to the UNIX status code exactly like a shell script does. And you've always got access to any Golang errors that have occurred too.

Unlike UNIX shell scripts, a Scriptish pipeline stops executing if any command returns an error.

You might not be aware of it, but by default, a pipeline in a UNIX shell script continues to run even if one of the commands returns an error. This causes error values to propagate - and error propagation is a major cause of robustness issues in software.

Philosophically, we believe that good software engineering practices are more important than UNIX shell compatibility.

Sources, Filters, Sinks, Logic and Capture Methods

Scriptish commands fall into one of four categories:

  • Sources create content in the pipeline, e.g. scriptish.CatFile(). They ignore whatever's already in the pipeline.
  • Filters do something with (or to) the pipeline's content, and they write the results back into the pipeline. These results form the input content for the next pipeline command.
  • Sinks do something with (or two) the pipeline's content, and don't write any new content back into the pipeline.
  • Logic implement support for if-like statements directly in Scriptish.

A pipeline normally:

  • starts with a source
  • applies one or more filters
  • finishes with a sink to send the results somewhere

But what if we want to get the results back into our Golang code, to reuse in some way? Instead of using a sink, use a capture method instead.

A capture method isn't a Scriptish command. It's a method on the Pipeline struct:

fileExists = scriptish.ExecPipeline(
    scriptish.TestFilepathExists("/path/to/file.txt")
).Okay()

Creating A Pipeline

You can create a pipeline in several ways.

Pipeline Produces Best For
scriptish.NewPipeline() Pipeline that's ready to run Reusable pipelines
scriptish.NewPipelineFunc() Function that will run your pipeline Getting results back into Golang
scriptish.ExecPipeline() Pipeline that has been run once Throwaway pipelines
NewPipeline()

Call NewPipeline() when you want to build a pipeline:

pipeline := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
)

pipeline can now be executed as often as you want.

result, err := pipeline.Exec().ParseInt()

Most of the examples in this README (and most of the unit tests) use scriptish.NewPipeline().

NewPipelineFunc()

NewPipelineFunc() builds the pipeline and turns it into a function.

fileExistsFunc := scriptish.NewPipelineFunc(
    scriptish.FileExists("/path/to/file")
)

Whenever you call the function, the pipeline executes. The function returns a *Pipeline. Use any of the capture methods to find out what happened when the pipeline executed.

fileExists := fileExistsFunc().Okay()

You can re-use the function as often as you want.

NewPipelineFunc() is great for pipelines where you want to get the results back into your Golang code:

getCurrentBranch := scriptish.NewPipelineFunc(
    scriptish.Exec("git", "branch", "--no-color"),
    scriptish.Grep("^[* ]"),
    scriptish.Tr([]string{"* "}, []string{""}),
)

currentBranch, err := getCurrentBranch().TrimmedString()
ExecPipeline()

ExecPipeline() builds a pipeline and executes it in a single step.

pipeline := scriptish.ExecPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
)

Behind the scenes, it simply does a scriptish.NewPipeline(...).Exec() for you.

You can then use any of the capture methods to find out what happened:

result, err = pipeline.ParseInt()

You can re-use the resulting pipeline as often as you want.

ExecPipeline() is great for pipelines that you want to throw away after use:

result, err := scriptish.ExecPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
).ParseInt()

Running An Existing Pipeline

Once you have built a pipeline, call the Exec() method to execute it:

pipeline := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
)
pipeline.Exec()

Exec() always returns a pointer to the same pipeline, so that you can use method chaining to create nicer-looking code.

// in this example, `pipeline` is available to be used more than once
pipeline := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
)
result, err := pipeline.Exec().ParseInt()

// in this example, we don't keep a reference to the pipeline
result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
).Exec().ParseInt()

Passing Parameters Into Pipelines

Both Pipeline.Exec() and the function returned by NewPipelineFunc() accept a list of parameters.

pipeline := scriptish.NewPipeline(
    scriptish.CatFile("$1"),
    scriptish.CountWords()
)
wordCount, _ := pipeline.Exec("/path/to/file").ParseInt()
fmt.Printf("file has %d words\n", wordCount)

countWordsInFile := scriptish.NewPipelineFunc(
    scriptish.CatFile("$1"),
    scriptish.CountWords()
)
wordCount, _ := countWordsInFile("/path/to/file").ParseInt()
fmt.Printf("file has %d words\n", wordCount)

The positional variables $1, $2, $3 et al are available inside the pipeline, just like they would be in a UNIX shell script function.

Calling A Pipeline From Another Pipeline

UNIX shell scripts can be broken up into functions to make them easier to maintain. You can do something similar in Scriptish, by calling a pipeline from another pipeline:

// this will parse the output of Git to find the selected branch
//
// the selected branch depends on the Git command called
filterSelectedBranch := scriptish.NewPipeline(
    scriptish.Grep("^[*] "),
    scriptish.Tr([]string{"* "}, []string{""}),
)

// which local branch are we working on?
localBranch, err := scriptish.NewPipeline(
    scriptish.Exec("git branch --no-color"),
    scriptish.RunPipeline(filterSelectedBranch),
).Exec().TrimmedString()

// what's the tracking branch?
remoteBranch, err := scriptish.NewPipeline(
    scriptish.Exec("git branch -av --no-color"),
    scriptish.RunPipeline(filterSelectedBranch),
).Exec().TrimmedString()

Capturing The Output

If you're familiar with UNIX shell scripting, you'll know that every shell command creates three different outputs:

  • stdout - normal text output
  • stderr - any error messages
  • status code - an integer representing what happened. 0 (zero) means success, any other value means an error occurred.

Scriptish commands work the same way. They also track any Golang errors that occur when the commands run.

Property Description
pipeline.Stdout normal text output
pipeline.Stderr an error messages (this is normally blank, because we have Golang errors too)
pipeline.Err Golang errors
pipeline.StatusCode an integer representing what happened. Normally 0 for success

When the pipeline has executed, you can call one of the capture methods to find out what happened:

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.CountWords()
).Exec().ParseInt()

// if the pipeline worked ...
// - result now contains the number of words in the file
// - err is nil
//
// and if the pipeline didn't work ...
// - result is 0
// - err contains a Golang error

If you want to run a Scriptish command and you don't care about capturing the output, call Pipeline.Okay():

success := scriptish.NewPipeline(
    scriptish.RmFile("/path/to/file.txt")
).Exec().Okay()

// if the pipeline worked ...
// - success is `true`
//
// and if the pipeline didn't work ...
// - success is `false`

Pipelines vs Lists

UNIX shell scripts support two main ways (known as sequences) to string individual commands together:

  • pipelines feed the output from one command into the next one
  • lists simply append the output from each command to stdout and stderr

Most of the time, you'll want to stick to pipelines, and port the rest of your shell script's behaviour over to native Golang code. That gives you the convenience of Scriptish's emulation of classic UNIX shell commands and the power of everything that Golang can do.

Sometimes, you'll find it less effort to use a few lists too.

A classic example is die(). It's very common for UNIX shell scripts to define their own die() function like this:

die() {
    echo "*** error: $*"
    exit 1
}

[[ -e ./Dockerfile ]] || die "cannot find Dockerfile"

Here's the equivalent Scriptish:

dieFunc := scriptish.NewList(
    scriptish.Echo("*** error: $*"),
    scriptish.ToStderr(),
    scriptish.Exit(1),
)

scriptish.ExecList(
    scriptish.TestFileExists("./Dockerfile"),
    scriptish.Or(dieFunc("cannot find Dockerfile")),
)

Creating A List

You can create a list in several ways.

Pipeline Produces Best For
scriptish.NewList() List that's ready to run Reusable lists
scriptish.NewListFunc() Function that will run your list Getting results back into Golang
scriptish.ExecList() List that has been run once Throwaway lists
NewList()

Call NewList() when you want to build a list:

list := scriptish.NewList(
    scriptish.Echo("*** warning: $*"),
)

list can now be executed as often as you want.

list.Exec("cannot find Dockerfile")
NewListFunc()

NewListFunc() builds the list and turns it into a function.

fileExistsFunc := scriptish.NewListFunc(
    scriptish.FileExists("/path/to/file")
)

Whenever you call the function, the list executes. The function returns a *List. Use any of the capture methods to find out what happened when the list executed.

fileExists := fileExistsFunc().Okay()

You can re-use the function as often as you want.

NewListFunc() is great for lists where you want to get the results back into your Golang code.

ExecList()

ExecList() builds a list and executes it in a single step.

list := scriptish.ExecList(
    scriptish.CatFile("/path/to/file1.txt"),
    scriptish.CatFile("/path/to/file2.txt"),
)

Behind the scenes, it simply does a scriptish.NewList(...).Exec() for you.

You can then use any of the capture methods to find out what happened:

result, err = list.ParseInt()

You can re-use the resulting list as often as you want.

ExecList() is great for lists that you want to throw away after use.

Running An Existing List

Once you have built a list, call the Exec() method to execute it:

list := scriptish.NewList(
    scriptish.CatFile("/path/to/file1.txt"),
    scriptish.CatFile("/path/to/file2.txt"),
)
list.Exec()

Exec() always returns a pointer to the same list, so that you can use method chaining to create nicer-looking code.

// in this example, `list` is available to be used more than once
list := scriptish.NewList(
    scriptish.CatFile("/path/to/file1.txt"),
    scriptish.CatFile("/path/to/file2.txt"),
)
result, err := pipeline.Exec().String()

// in this example, we don't keep a reference to the list
result, err := scriptish.NewList(
    scriptish.CatFile("/path/to/file1.txt"),
    scriptish.CatFile("/path/to/file2.txt"),
).Exec().String()

Passing Parameters Into Lists

Both List.Exec() and the function returned by NewListFunc() accept a list of parameters.

list := scriptish.NewPipeline(
    scriptish.CatFile("$1"),
    scriptish.CatFile("$2"),
)
fileContents := list.Exec("/path/to/file1.txt", "/path/to/file2.txt").String()

getTwoFileContents := scriptish.NewPipelineFunc(
    scriptish.CatFile("$1"),
    scriptish.CatFile("$2"),
)
fileContents := getTwoFileContents("/path/to/file1.txt", "/path/to/file2.txt").String()

The positional variables $1, $2, $3 et al are available inside the list, just like they would be in a UNIX shell script function.

Calling A List From Another List Or Pipeline

UNIX shell scripts can be broken up into functions to make them easier to maintain. You can do something similar in Scriptish, by calling a list from another pipeline or list:

// this will fetch the latest changes from the upstream Git repo
fetch_changes_from_origin := scriptish.NewList(
    scriptish.Exec("git", "remote", "update", "$ORIGIN"),
    scriptish.Or(dieFunc("Unable to get list of changes from '$ORIGIN'")),
    scriptish.Exec("git", "fetch", "$ORIGIN"),
    scriptish.Or(dieFunc("Unable to fetch latest changes from '$ORIGIN'")),
    scriptish.Exec("git", "fetch", "--tags"),
    scriptish.Or(dieFunc("Unable to fetch latest tags from '$ORIGIN'")),
)

// this will fetch the latest changes from upstream, and then
// merge them into local branches
merge_latest_changes = scriptish.NewList(
    scriptish.RunList(fetch_changes_from_origin),
    ...
)

Pipelines, Lists and Sequences

In UNIX shell programming, pipelines and lists are both examples of a sequence of commands. Each one is a set of commands that are wrapped in slightly different execution logic.

In Scriptish, a Pipeline and a List are type aliases for a Sequence. A call to NewPipeline() or NewList() creates a Sequence that also has the right execution logic for a pipeline or a list. We've done it this way so that you can use both pipelines and lists in our logic calls.

All of our logic calls accept Sequence parameters. You can pass in a Pipeline or a List to suit, and either will work just fine.

UNIX Shell String Expansion

What Is String Expansion?

One of the things that makes UNIX shells so powerful is the way they expand a string, or a line of code, before executing it.

#!/usr/bin/env bash

PARAM1=hello world

# output: "Hello world"
echo ${PARAM1^H}

We've integrated the ShellExpand package so that string expansion is available to you.

Setting Positional Parameters

The positional parameters are $1, $2, $3 all the way up to $9, as well as $# and $*. These are exactly the same as their equivalents in shell scripts.

To set these, pass parameters into pipeline or into lists.

Setting Local Variables

Every Pipeline and List struct comes with a LocalVars member. You can call its Setenv() method to create more variables to use in string expansion:

// create a reusable List
fetch_changes_from_remote := scriptish.NewList(
    scriptish.Exec("git", "remote", "update", "$REMOTE"),
    scriptish.Or(dieFunc("Unable to get list of changes from '$REMOTE'")),
    scriptish.Exec("git", "fetch", "$REMOTE"),
    scriptish.Or(dieFunc("Unable to fetch latest changes from '$REMOTE'")),
    scriptish.Exec("git", "fetch", "--tags"),
    scriptish.Or(dieFunc("Unable to fetch latest tags from '$REMOTE'")),
)

// set the value of '$REMOTE'
fetch_changes_from_remote.LocalVars.Setenv("REMOTE", "origin")

// run it
fetch_changes_from_remote.Exec()

Any local variables that you set will remain set if you reuse the pipeline or list - ie, they are persistent.

Escaping Strings

The one downside of string expansion is that you will need to escape characters in your strings, to avoid them being interpreted as instructions to the string expansion engine.

The basic rule of thumb is that if you'd need to escape it in a shell script, you'll also need to escape it in a string passed into Scriptish.

Filename Globbing / Pathname Expansion

At the moment, the string expansion does not support globbing (properly known as pathname expansion). That means you can't use wildcards in filepaths anywhere.

This is something we might add in a future release.

From Bash To Scriptish

Here's a handy table to help you quickly translate an action from a Bash shell script to the equivalent Scriptish command.

Bash Scriptish
$(...) scriptish.Exec()
${x%.*} scriptish.StripExtension()
${x%$y}%z `scriptish.SwapExtensions()
${x%$y} scriptish.TrimSuffix()
[[ -e $x ]] scriptish.TestFilepathExists()
[[ -n $x ]] scriptish.TestNotEmpty()
[[ -z $x ]] scriptish.TestEmpty()
> $file scriptish.WriteToFile()
>> $file scriptish.AppendToFile()
`
&& scriptish.And()
basename ... scriptish.Basename()
cat "..." scriptish.CatFile(...)
cat /dev/null > $x scriptish.TruncateFile($x)
chmod scriptish.Chmod()
cut -f scriptish.CutFields()
dirname ... scriptish.Dirname()
echo "..." scriptish.Echo(...)
echo "$@" scriptish.EchoArgs()
exit ... scriptish.Exit()
function scriptish.RunPipeline()
grep ... scriptish.Grep()
grep -v .. scriptish.GrepV()
head -n X scriptish.Head(X)
if expr ; then body ; fi scriptish.If()
if expr ; then body ; else elseBlock ; fi scriptish.IfElse()
ls -1 ... scriptish.ListFiles(...)
`ls -l awk '{ print $1 }'`
mkdir scriptish.Mkdir()
mktemp scriptish.MkTempFile()
mktemp -d scriptish.MkTempDir()
mktemp -u scriptish.MkTempFilename()
return scriptish.Return()
rm -f scriptish.RmFile()
rm -r scriptish.RmDir()
sort scriptish.Sort()
sort -r scriptish.Rsort()
tail -n X scriptish.Tail(X)
touch scriptish.Touch()
tr old new scriptish.Tr(old, new)
uniq scriptish.Uniq()
wc -l scriptish.CountLines()
wc -w scriptish.CountWords()
which scriptish.Which()
xargs cat scriptish.XargsCat()
xargs rm scriptish.XargsRmFile()
xargs test -e scriptish.XargsTestFilepathExists()

Sources

Sources get data from outside the pipeline, and write it into the pipeline's Stdout.

Every pipeline normally begins with a source, and is then followed by one or more filters.

Basename()

Basename() treats the input as a filepath. Any parent elements are stripped from the input, and the results written to the pipeline's Stdout.

Any blank lines are preserved.

result, err := scriptish.NewPipeline(
    scriptish.Basename("/path/to/folder/or/file"),
).Exec().TrimmedString()
CatFile()

CatFile() writes the contents of a file to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("a/file.txt"),
).Exec().String()
CatStdin()

CatStdin() copies the program's stdin (os.Stdin in Golang terms) to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.CatStdin(),
).Exec().String()
Dirname()

Dirname() treats the input as a filepath. It removes the last element from the input. It writes the result to the pipeline's Stdout.

If the input is blank, Dirname() returns a '.'

result, err := scriptish.NewPipeline(
    scriptish.Dirname("/path/to/folder/or/file")
).Exec().TrimmedString()
Echo()

Echo() writes a string to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.Echo("hello world"),
).Exec().String()
EchoArgs()

EchoArgs() writes the program's arguments to the pipeline's Stdout, one line per argument.

result, err := scriptish.NewPipeline(
    scriptish.EchoArgs(),
).Exec().String()
EchoSlice()

EchoSlice() writes an array of strings to the pipeline's Stdout, one line per array entry.

myStrings := []string{"hello world", "have a nice day"}

result, err := scriptish.NewPipeline(
    scriptish.EchoSlice(myStrings),
).Exec().String()
EchoToStderr()

EchoToStderr() writes a string to the pipeline's Stderr.

result, err := scriptish.NewPipeline(
    scriptish.EchoToStderr("*** error: file not found"),
).Exec()
Exec()

Exec() executes an operating system command. The command's stdin will be the pipeline's Stdin, and it will write to the pipeline's Stdout and Stderr.

The command's status code will be stored in the pipeline's StatusCode.

localBranch, err := scriptish.ExecPipeline(
    scriptish.Exec("git", "branch", "--no-color"),
    scriptish.Grep("^[* ]"),
    scriptish.Tr([]string{"* "}, []string{""}),
).TrimmedString()

Use the .Okay() capture method if you simply want to know if the command worked or not:

success := scriptish.ExecPipeline(scriptish.Exec("git", "push")).Okay()

Golang will set err to an exec.ExitError if the command's status code is not 0 (zero).

Golang will set err to an os.PathError if the command could not be found in the first place.

ListFiles()

ListFiles() writes a list of matching files to the pipeline's Stdout, one line per filename found.

  • If path is a file, ListFiles writes the file to the pipeline's Stdout
  • If path is a folder, ListFiles writes the contents of the folder to the pipeline's Stdout. The path to the folder is included.
  • If path contains wildcards, ListFiles writes any files that matches to the pipeline's Stdout. ListFiles() uses Golang's os.Glob() to expand the wildcards.
// list a single file, if it exists
result, err := scriptish.NewPipeline(
    scriptish.ListFiles("path/to/file"),
).Exec().String()

// list all files in a folder, if the folder exists
result, err := scriptish.NewPipeline(
    scriptish.ListFiles("path/to/folder"),
).Exec().String()

// list all files in a folder that match wildcards, if the folder exists
result, err := scriptish.NewPipeline(
    scriptish.ListFiles("path/to/folder/*.txt"),
).Exec().String()
Lsmod()

Lsmod() writes the permissions of the given filepath to the pipe's stdout.

  • Symlinks are followed.
  • Permissions are in the form '-rwxrwxrwx'.

It ignores the contents of the pipeline.

result, err := scriptish.NewPipeline(
    scriptish.Lsmod("/path/to/file"),
).Exec().TrimmedString()
MkTempDir()

MkTempDir() creates a temporary folder, and writes the filename to the pipeline's Stdout.

tmpDir, err := scriptish.NewPipeline(
    scriptish.MkTempFile(os.TempDir(), "scriptish-")
).Exec().TrimmedString()
MkTempFile()

MkTempFile() creates a temporary file, and writes the filename to the pipeline's Stdout.

tmpFilename, err := scriptish.NewPipeline(
    scriptish.MkTempFile(os.TempDir(), "scriptish-*")
).Exec().TrimmedString()
MkTempFilename()

MkTempFilename() generates a temporary filename, and writes the filename to the pipeline's Stdout.

tmpFilename, err := scriptish.NewPipeline(
    scriptish.MkTempFilename(os.TempDir(), "scriptish-*")
).Exec().TrimmedString()
Which()

Which() searches the current PATH to find the given path. If one is found, the command's path is written to the pipeline's Stdout.

It ignores the contents of the pipeline.

result, err := scriptish.NewPipeline(
    scriptish.Which("git"),
).Exec().String()

Filters

Filters read the contents of the pipeline's Stdin, do something to that data, and write the results out to the pipeline's Stdout.

When you've finished adding filters to your pipeline, you should either add a sink, or call one of the capture methods to get the results back into your Golang code.

AppendToTempFile()

AppendToTempFile() writes the contents of the pipeline's Stdin to a temporary file. The temporary file's filename is then written to the pipeline's Stdout.

If the file does not exist, it is created.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.AppendToTempFile(os.TempDir(), "scriptish-*"),
).Exec().TrimmedString()

// result now contains the temporary filename
CountLines()

CountLines() counts the number of lines in the pipeline's Stdin, and writes that to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("path/to/folder/*.txt"),
    scriptish.CountLines(),
).Exec().ParseInt()
CountWords()

CountWords() counts the number of words in the pipeline's Stdin, and writes that to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("path/to/file.txt"),
    scriptish.CountWords(),
).Exec().ParseInt()
CutFields()

CutFields() retrieves only the fields specified on each line of the pipeline's Stdin, and writes them to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.Echo("one two three four five"),
    scriptish.CutFields("2-3,5")
).Exec().String()
DropEmptyLines()

DropEmptyLines() removes any lines that are blank, or that only contain whitespace.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.DropEmptyLines()
).Exec().String()
Grep()

Grep() filters out lines that do not match the given regex.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Grep("second|third"),
).Exec().String()
GrepV()

GrepV() filters out lines that match the given regex.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.GrepV("second|third"),
).Exec().String()
Head()

Head() copies the first N lines of the pipeline's Stdin to its Stdout.

If N is zero or negative, Head() copies no lines.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Head(100),
).Exec().String()
Rsort()

Rsort() sorts the contents of the pipeline into descending alphabetical order.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Rsort(),
).Exec().String()
RunPipeline()

RunPipeline() allows you to call one pipeline from another. Use it to create reusable pipelines, a bit like shell script functions.

getWordCount := scriptish.NewPipeline(
    scriptish.SplitWords(),
    scriptish.CountLines(),
)

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.RunPipeline(getWordCount),
).Exec().ParseInt()
Sort()

Sort() sorts the contents of the pipeline into ascending alphabetical order.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Sort(),
).Exec().String()
StripExtension()

StripExtension() treats every line in the pipeline as a filepath. It removes the extension from each filepath.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder"),
    scriptish.StripExtension(),
).Exec().Strings()
SwapExtensions()

SwapExtensions() treats every line in the pipeline as a filepath.

It replaces every old extension with the corresponding new one.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder"),
    scriptish.SwapExtensions([]string{"txt","yml"}, []string{"text","yaml"}),
).Exec().Strings()

If the second parameter is a string slice of length 1, every old file extension will be replaced by that parameter.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder"),
    scriptish.SwapExtensions([]string{"yml","YAML"}, []string{"yaml"}),
).Exec().Strings()

If the first and second parameters are different lengths, SwapExtensions() will return an scriptish.ErrMismatchedInputs.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder"),
    scriptish.SwapExtensions([]string{"one"}, []string{"1","2"}),
).Exec().Strings()

// err is an ErrMismatchedInputs, and result is empty
Tail()

Tail() copies the last N lines from the pipeline's Stdin to its Stdout.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Tail(50),
).Exec().String()
Tr()

Tr() replaces all occurances of one string with another.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Tr([]string{"one","two"}, []string{"1","2"}),
).Exec().String()

If the second parameter is a string slice of length 1, everything from the first parameter will be replaced by that slice.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Tr([]string{"one","two"}, []string{"numberwang"}),
).Exec().String()

If the first and second parameters are different lengths, Tr() will return an scriptish.ErrMismatchedInputs.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Tr([]string{"one","two","three"}, []string{"1","2"}),
).Exec().String()

// err is an ErrMismatchedInputs, and result is empty
TrimSuffix()

TrimSuffix() removes the given suffix from each line of the pipeline.

Use it to emulate basename(1)'s [suffix] parameter.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder/"),
    scriptish.TrimSuffix(".txt")
).Exec().Strings()
TrimWhitespace()

TrimWhitespace() removes any whitespace from the front and end of each line in the pipeline.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.TrimWhitespace(),
).Exec().String()
Uniq()

Uniq() removes duplicated lines from the pipeline.

result, err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.Uniq(),
).Exec().Strings()
XargsBasename()

XargsBasename() treats each line in the pipeline's Stdin as a filepath. Any parent elements are stripped from the line, and the results written to the pipeline's Stdout.

Any blank lines are preserved.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder/*.txt"),
    scriptish.XargsBasename()
).Exec().Strings()
XargsCat()

XargsCat() treats each line in the pipeline's Stdin as a filepath. The contents of each file are written to the pipeline's Stdout.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder/*.txt"),
    scriptish.XargsCat()
).Exec().String()
XargsDirname()

XargsDirname() treats each line in the pipeline's Stdin as a filepath. The last element is stripped from the line, and the results written to the pipeline's Stdout.

Any blank lines are turned in '.'

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder/*.txt"),
    scriptish.XargsDirname()
).Exec().Strings()
XargsRmFile()

XargsRmFile() treats every line in the pipeline as a filename. It attempts to delete each file.

It stops at the first file that cannot be deleted.

Each successfully-deleted filepath is written to the pipeline's Stdout, for use by the next command in the pipeline.

err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/files"),
    scriptish.XargsRmFile(),
    scriptish.ForXIn(
        scriptish.Printf("deleted file: $x")
    )
).Exec().Error()
XargsTestFilepathExists()

XargsTestFilepathExists() treats each line in the pipeline as a filepath. It checks to see if the given filepath exists. If the filepath exists, it is written to the pipeline's stdout.

It does not care what the filepath points at (file, folder, named pipe, and so on).

// example: find all RAW photo files that also have a corresponding
// JPEG file
result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/folder/*.raw"),
    scriptish.SwapExtensions(".raw", ".jpeg"),
    scriptish.XargsTestFilepathExists()
).Exec().Strings()
XargsTruncateFiles()

XargsTruncatesFiles() treats each line of the pipeline's Stdin as a filepath. The contents of each file are truncated. If the file does not exist, it is created.

Each filepath is written to the pipeline's Stdout, for use by the next command in the pipeline.

result, err := scriptish.NewPipeline(
    scriptish.ListFiles("/path/to/files"),
    scriptish.XargsTruncateFiles(),
).Exec().Strings()

// result now contains a list of the files that have been truncated

Sinks

Sinks take the contents of the pipeline's Stdin, and write it to somewhere outside the pipeline.

A sink should be the last command in your pipeline. You can add more commands afterwards if you really want to. Just be aware that the first command after any sink will be starting with an empty Stdin.

AppendToFile()

AppendToFile() writes the contents of the pipeline's Stdin to the given file

If the file does not exist, it is created.

err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.AppendToFile("my-app.log"),
).Exec().Error()
Exit()

Exit() terminates your Golang program with the given status code. Use with caution.

dieFunc := scriptish.NewList(
    scriptish.Echo("*** error: $*"),
    scriptish.ToStderr(),
    scriptish.Exit(1),
)

scriptish.ExecList(
    scriptish.TestFilepathExists("./Dockerfile"),
    scriptish.Or(dieFunc("cannot find Dockerfile")),
)
Return()

Return() terminates the pipeline with the given status code.

statusCode := scriptish.NewPipeline(
    scriptish.Return(3)
).Exec().StatusCode()

// statusCode will be: 3
ToStderr()

ToStdout() writes the pipeline's Stdin to the program's stderr (os.Stderr in Golang terms).

err := scriptish.NewPipeline(
    scriptish.Echo("usage: simpleca <command>"),
    scriptish.ToStderr(),
).Exec().Error()
ToStdout()

ToStdout() writes the pipeline's Stdin to the program's Stdout (os.Stdout in Golang terms).

err := scriptish.NewPipeline(
    scriptish.Echo("usage: simpleca <command>"),
    scriptish.ToStdout(),
).Exec().Error()
WriteToFile()

WriteToFile() writes the contents of the pipeline's Stdin to the given file. The existing contents of the file are replaced.

If the file does not exist, it is created.

err := scriptish.NewPipeline(
    scriptish.CatFile("/path/to/file.txt"),
    scriptish.WriteToFile("/path/to/new_file.txt"),
).Exec().Error()

Builtins

Builtins are UNIX shell commands and UNIX CLI utilities that don't fall into the sources, sinks and filters categories:

  • their input is a parameter; they ignore the pipeline
  • their only output is the status code; they don't write anything new to the pipeline
Chmod()

Chmod() attempts to change the permissions on the given filepath.

It ignores the contents of the pipeline.

On success, it returns the status code StatusOkay. On failure, it returns the status code StatusNotOkay.

result, err := scriptish.NewPipeline(
    scriptish.Chmod("/path/to/file", 0644),
).Exec().StatusError()
Mkdir()

Mkdir() creates the named directory, along with any parent folders that are needed.

It ignores the contents of the pipeline.

On success, it returns the status code StatusOkay. On failure, it returns the status code StatusNotOkay.

result, err := scriptish.NewPipeline(
    scriptish.Mkdir("/path/to/folder", 0755),
).Exec().StatusError()
RmDir()

RmDir() deletes the given folder, as long as the folder is empty.

It ignores the contents of the pipeline.

It ignores the file's file permissions, because the underlying Golang os.Remove() behaves that way.

err := scriptish.NewPipeline(
    scriptish.RmDir("/path/to/folder"),
).Exec().Error()
RmFile()

RmFile() deletes the given file.

It ignores the contents of the pipeline.

It ignores the file's file permissions, because the underlying Golang os.Remove() behaves that way.

err := scriptish.NewPipeline(
    scriptish.RmFile("/path/to/file"),
).Exec().Error()
TestEmpty()

TestEmpty() returns StatusOkay if the (expanded) input is empty; StatusNotOkay otherwise.

It is the equivalent to if [[ -z $VAR ]] in a UNIX shell script.

show_usage() {
    echo "*** error: $*"
    echo
    echo "usage: $0 <name-of-arg>"
    exit 1
}

if [[ -z $1 ]] ; then
    show_usage("missing parameter <name-of-arg>")
fi

Here's the equivalent Scriptish:

showUsage := scriptish.NewList(
    scriptish.Echo("*** error: $*"),
    scriptish.Echo(""),
    scriptish.Echo("usage: $0 <name-of-arg>")
    scriptish.Exit(1),
)

checkArgs := scriptish.NewList(
    scriptish.If(
        scriptish.NewPipeline(scriptish.TestEmpty("$1")),
        showUsage("missing argument")
    ),
)

checkArgs.Exec(os.Args...)
TestFilepathExists()

TestFilepathExists() checks to see if the given filepath exists. If it does, it returns StatusOkay. If not, it returns StatusNotOkay.

  • It does not care what the filepath points at (file, folder, named pipe, and so on).
  • It ignores the contents of the pipeline.
  • It follows symbolic links.
fileExists := scriptish.ExecPipeline(
    scriptish.TestFilepathExists("/path/to/file")
).Okay()
TestNotEmpty()

TestNotEmpty() returns StatusOkay if the (expanded) input is not empty; StatusNotOkay otherwise.

It is the equivalent to if [[ -n $VAR ]] in a UNIX shell script.

show_usage() {
    echo "*** error: $*"
    echo
    echo "usage: $0 <name-of-arg>"
    exit 1
}

[[ -n $1 ]] || show_usage("missing parameter <name-of-arg>")

Here's the equivalent Scriptish:

showUsage := scriptish.NewList(
    scriptish.Echo("*** error: $*"),
    scriptish.Echo(""),
    scriptish.Echo("usage: $0 <name-of-arg>")
    scriptish.Exit(1),
)

checkArgs := scriptish.NewList(
    scriptish.TestNotEmpty("$1"),
    scriptish.Or(showUsage("missing argument")),
)

checkArgs.Exec(os.Args...)
Touch()

Touch() creates the named file (if it doesn't exist), or updates its atime and mtime (if it does exist).

It ignores the contents of the pipeline.

On success, it returns the status code StatusOkay. On failure, it returns the status code StatusNotOkay.

pipeline := NewPipeline(
    Touch("./config.yaml")
)
success, err := pipeline.Exec().StatusError()
TruncateFile()

TruncateFile() removes the contents of the given file.

If the file does not exist, it is created.

err := scriptish.NewPipeline(
    scriptish.TruncateFile("/tmp/scriptish-test"),
).Exec().Error()

Capture Methods

Capture methods available on each Pipeline. Use them to get the output from the pipeline.

Don't forget to run your pipeline first, if you haven't already!

Bytes()

Bytes() is the standard interface's io.Reader Bytes() method.

  • It writes the contents of the pipeline's Stdout into the byte slice that you provide.
  • It returns the number of bytes written.
  • It also returns the pipeline's current Golang error value.

Normally, you wouldn't call this yourself.

Error()

Error() returns the pipeline's current Golang error status, which may be nil.

err := scriptish.ExecPipeline(scriptish.RmFile("/path/to/file")).Error()
Flush()

Flush() writes the contents of the pipeline or list out to the given destinations.

Use os.Stdout and os.Stderr to send the output to your program's terminal.

scriptish.NewPipeline(
    scriptish.Echo("usage: simpleca <command>"),
)

scriptish.Exec().Flush(os.Stdout, os.Stderr)
Okay()

Okay() returns true|false depending on the pipeline's current UNIX status code.

success := scriptish.ExecPipeline(scriptish.Exec("git push")).Okay()

success is a bool:

  • false if the pipeline's StatusCode property is not 0
  • true otherwise

All Scriptish commands set the pipeline's StatusCode, so it's safe to use Okay() to check any pipeline you create.

It's mostly there if you want to call a pipeline in a Golang if statement:

if !scriptish.ExecPipeline(scriptish.Exec("git", "push")).Okay() {
    // push failed, do something about it
}
ParseInt()

ParseInt() returns the pipeline's Stdout as an int value:

lineCount, err := scriptish.ExecPipeline(
    scriptish.CatFile("/path/to/file"),
    scriptish.CountLines(),
).ParseInt()

If the pipeline's Stdout can't be turned into an integer, then it will return 0 and the parsing error from Golang's strconv.ParseInt().

If the pipeline didn't execute successfully, it will return 0 and the pipeline's current Golang error status.

String()

String() returns the pipeline's Stdout as a single string:

contents, err := scriptish.ExecPipeline(
    scriptish.CatFile("/path/to/file")
).String()

The string will be terminated by a linefeed \n character. String() is a good choice if you want to get content into your Golang code. If you just want a single-line value, see TrimmedString() below.

If the pipeline's Stdout is empty, an empty string will be returned.

If the pipeline didn't execute successfully, the contents of the pipeline's Stderr will be returned. We might change this behaviour in the future.

Strings()

Strings() returns the pipeline's Stdout as an array of strings (aka a string slice):

files, err := scriptish.ExecPipeline(
    scriptish.ListFiles("/path/to/folder"),
    scriptish.XargsBasename(),
).Strings()

Each string will not be terminated by a linefeed \n character.

If the pipeline's Stdout is empty, an empty string slice will be returned.

If the pipeline didn't execute successfully, the contents of the pipeline's Stderr will be returned. We might change this behaviour in the future.

TrimmedString()

TrimmedString() returns the pipeline's Stdout as a single string, with leading and trailing whitespace removed:

localBranch, err := scriptish.ExecPipeline(
    scriptish.Exec("git", "branch", "--no-color"),
    scriptish.Grep("^[* ]"),
    scriptish.Tr([]string{"* "}, []string{""}),
).TrimmedString()

TrimmedString() is the right choice if you're expecting a single line of text back. This is very useful for getting results back into your Golang code!

If the pipeline's Stdout is empty, an empty string will be returned.

If the pipeline didn't execute successfully, the contents of the pipeline's Stderr will be returned. We might change this behaviour in the future.

Logic Calls

Most of the time, you will use native Golang code to write if statements for your code. Use the capture methods to get the results of your pipelines back into your Golang code.

Sometimes, it will be more convenient to use Scriptish's built-in logic support. The classic example is the die() function commonly created in UNIX shell scripts to handle errors:

die() {
    echo "*** error: $*" >&2
    exit 1
}

[[ -e ./Dockerfile ]] || die "cannot find Dockerfile"

Here's the equivalent Scriptish:

dieFunc := scriptish.NewList(
    scriptish.Echo("*** error: $*"),
    scriptish.ToStderr(),
    scriptish.Exit(1),
)

scriptish.ExecList(
    scriptish.TestFilepathExists("./Dockerfile"),
    scriptish.Or(dieFunc("cannot find Dockerfile")),
)

(It also has to be said that implementing logic support in Scriptish was a good test case for proving Scriptish's underlying design.)

Generally, all the implemented logc takes Lists or Pipelines as arguments. (Lists and Pipelines are both aliases for the Sequence struct, so you can pass either in to suit.) If there are any exceptions to this rule, we'll make sure to point it out in the documentation for the individual logic call.

And()

And() executes the given sequence only if the previous command did not return any kind of error.

The sequence starts with an empty Stdin. The sequence's output is written back to the Stdout and Stderr of the calling list or pipeline - along with the StatusCode() and Error().

It is an emulation of UNIX shell scripting's command1 && command2 feature.

NOTE that And() only works when run inside a List:

scriptish.ExecList(
    scriptish.Exec("git", "fetch"),
    // if `git fetch` fails, do not attempt the merge
    scriptish.And(
        scriptish.NewList(
            scriptish.Exec("git", "merge")
        )
    )
)

If you call And() inside a Pipeline, it'll always run the given sequence. Pipelines terminate whenever a command returns an error, so And() will only be called if the previous command succeeded.

If()

If() executes the body if (and only if) the given expr does not return any kind of error.

Both expr and body start with an empty Stdin. Their output is written back to the pipeline's Stdout and Stderr.

It is an emulation of UNIX shell scripting's if expr ; then body ; fi feature.

result, err := scriptish.ExecList(
    scriptish.If(
        // this is the `expr` or expression
        scriptish.NewPipeline(
            scriptish.TestFilepathExists("/path/to/file"),
        ),
        // this is the `body` that is executed if the `expr` succeeds
        scriptish.NewPipeline(
            scriptish.CatFile("/path/to/file"),
            scriptish.Head(3),
        ),
    )
).String()

You can safely use If() inside a pipeline, because it doesn't depend upon the result of any previous command.

IfElse()

IfElse() executes the body if (and only if) the expr completes without an error. Otherwise, it executes the elseBlock instead.

IfElse() is an emulation of UNIX shell scripting's if expr ; then body ; else elseBlock ; fi.

result, err := scriptish.ExecList(
    scriptish.IfElse(
        // this is the `expr` or expression
        scriptish.NewPipeline(
            scriptish.TestFilepathExists("/path/to/file"),
        ),
        // this is the `body` that is executed if the `expr` succeeds
        scriptish.NewPipeline(
            scriptish.CatFile("/path/to/file"),
            scriptish.Head(3),
        ),
        // and this is the `elseBlock` that is executed if the `expr` fails
        scriptish.NewPipeline(
            scriptish.Echo("*** error: file not found"),
            scriptish.ToStderr(),
        )
    )
).String()
Or()

Or() executes the given sequence only if the previous command has returned some kind of error.

The sequence starts with an empty Stdin. The sequence's output is written back to the Stdout and Stderr of the calling list or pipeline - along with the StatusCode() and Error().

It is an emulation of UNIX shell scripting's list1 || command feature.

NOTE that Or() only works when run inside a List:

statusCode, err := scriptish.NewList(
    scriptish.TestFilepathExists("/path/to/file"),
    scriptish.Or(dieFunc("cannot find file")),
).Exec().StatusError()

If you call Or() from inside a Pipeline, it will never work. Pipelines terminate when the first command returns an error. This means that either:

  • the pipeline will terminate before Or() is reached, or
  • Or() never executes the given sequence (because there's no previous error)

At the moment, we can't think of a way of detecting any attempt to call Or() from a pipeline.

Errors

ErrMismatchedInputs

ErrMismatchedInputs is returned whenever two input arrays aren't the same length.

Inspirations

Scriptish is inspired by:

Compared To Labix's Pipe

Pipe is a bit more low level, and seems to be aimed predominantly at executing external commands, as a more powerful alternative to Golang's exec package.

If that's what you need, definitely check it out!

Compared To Bitfield's Script

We started out using (and contributing to) Script, but ran into a few things that didn't suit what we were doing:

  • Built for different purposes

    script is aimed at doing minimal shell-like operations from a Golang app.

    scriptish is more about providing everything necessary to recreate any size UNIX shell script - including powerful ones like the HubFlow extension for Git - to Golang, without having to port everything to Golang.

  • Aimed at different people

    We want it to take as little thinking as possible to port UNIX shell scripts over to Golang - especially for casual or lapsed Golang programmers!

    That means (amongst other things) using function names that are similar to the UNIX shell command that they emulate, and emulating UNIX behaviour as closely as is practical, including UNIX features like status codes and stderr.

  • Extensibility

    script operations are methods on the script.Pipe struct. We found that this makes it very hard to extend script with your own methods, because Golang doesn't support inheritance, and the method chaining prevents Golang embedding from working.

    In contrast, script commands are first-order functions that take the Pipe as a function parameter. You can create your own Scriptish commands, and they can live in your own Golang package.

  • Reusability

    There's currently no way to call one pipeline from another using script alone. You can achieve that by writing your own Golang boiler plate code.

    scriptish builds first-order pipelines that you can run, pass around as values, and call from other scriptish pipelines.

We were originally attracted to script because of how incredibly easy it is to use. There's a lot to like about it, and we've definitely tried to create the same feel in scriptish too. We've borrowed concepts such as sources, filters and sinks from script, because they're such a great way to describe how different Scriptish commands behave.

You should definitely check script out if you think that scriptish is too much for what you need.

Documentation

Index

Constants

View Source 
const StatusNotOkay = pipe.StatusNotOkay

StatusNotOkay is an alias for the underlying pipe library's StatusNotOkay

It just saves us having to import the pipe library into every single file in the project.

View Source 
const StatusOkay = pipe.StatusOkay

StatusOkay is an alias for the underlying pipe library's StatusOkay

It just saves us having to import the pipe library into every single file in the project.

Variables

View Source 
var NewSourceFromReader = pipe.NewSourceFromReader

NewSourceFromReader is an alias for the underlying pipe library's NewSourceFromReader()

View Source 
var NewSourceFromString = pipe.NewSourceFromString

NewSourceFromString is an alias for the underlying pipe library's NewSourceFromString()

Functions

func IsTraceEnabled 

func IsTraceEnabled() bool

IsTraceEnabled returns true if execution tracing is currently switched on across Scriptish

func NewListFunc 

func NewListFunc(steps ...Command) func(params ...string) *List

NewListFunc creates a list, and wraps it in a function to make it easier to call.

func NewPipelineFunc 

func NewPipelineFunc(steps ...Command) func(...string) *Pipeline

NewPipelineFunc creates a pipeline, and wraps it in a function to make it easier to call.

func TraceOsStderr 

func TraceOsStderr(format string, args ...interface{})

TraceOsStderr writes a trace message about content written to the program's Stderr

func TraceOsStdout 

func TraceOsStdout(format string, args ...interface{})

TraceOsStdout writes a trace message about content written to the program's Stdout

func TraceOutput 

func TraceOutput(dest string, format string, args ...interface{})

TraceOutput writes a trace message about content written to a file or a buffer

func TracePipeStderr 

func TracePipeStderr(format string, args ...interface{})

TracePipeStderr writes a trace message about content written to a pipe's Stderr buffer

func TracePipeStdout 

func TracePipeStdout(format string, args ...interface{})

TracePipeStdout writes a trace message about content written to a pipe's Stdout buffer

func Tracef 

func Tracef(format string, args ...interface{})

Tracef writes a trace message to os.Stderr if tracing is enabled

Types

type Command 

type Command = pipe.Command

Command is an alias for the underlying pipe library's Command

It just saves us having to import the pipe library into every single file in the project.

func And 

func And(sq *Sequence) Command

And executes the given sequence only if the previous command did not return any kind of error.

The sequence starts with an empty Stdin. The sequence's output is written back to the Stdout and Stderr of the calling list or pipeline - along with the StatusCode() and Error().

It is an emulation of UNIX shell scripting's `list1 && command`

func AppendToFile 

func AppendToFile(filename string) Command

AppendToFile writes the contents of the pipeline's stdin to the given file

If the file does not exist, it is created.

func AppendToTempFile 

func AppendToTempFile(dir string, pattern string) Command

AppendToTempFile writes the contents of the pipeline's stdin to a temporary file. The temporary file's filename is then written to the pipeline's stdout.

If the file does not exist, it is created.

func Basename 

func Basename(input string) Command

Basename treats the input as a filepath.

It removes any parent elements from the filepath, and writes the result into the pipeline's `Stdout`.

func CatFile 

func CatFile(filename string) Command

CatFile writes the contents of a file to the pipeline's stdout

func CatStdin 

func CatStdin() Command

CatStdin writes the contents of the program's stdin to the pipeline's stdout

func Chmod 

func Chmod(filepath string, mode os.FileMode) Command

Chmod attempts to change the permissions on the given file.

It ignores the contents of the pipeline.

On success, it returns the status code `StatusOkay`. On failure, it returns the status code `StatusNotOkay`.

func CountLines 

func CountLines() Command

CountLines counts the number of lines in the pipeline's stdin, and writes the overall count to the pipeline's stdout

func CountWords 

func CountWords() Command

CountWords counts the number of words in the pipe's stdin, and writes the overall count to the pipe's stdout

func CutFields 

func CutFields(spec string) Command

CutFields emulates `cut -f`

func Dirname 

func Dirname(input string) Command

Dirname treats the input as a filepath. It removes the last element from the input. It writes the result to the pipeline's `Stdout`.

If the input is blank, Dirname returns a '.'

func DropEmptyLines 

func DropEmptyLines() Command

DropEmptyLines removes any lines that are blank, or that only contain whitespace

func Echo 

func Echo(input string) Command

Echo writes a string to the pipeline's stdout

func EchoArgs 

func EchoArgs() Command

EchoArgs echos the program's command-line arguments into the pipeline, one argument per line

We do not perform string expansion on the arguments.

func EchoRawSlice 

func EchoRawSlice(input []string) Command

EchoRawSlice writes an array of strings to the pipeline's stdout, one line per array entry

it does not perform string expansion

func EchoSlice 

func EchoSlice(input []string) Command

EchoSlice writes an array of strings to the pipeline's stdout, one line per array entry

it DOES perform string expansion. If you want to avoid that, use the EchoSliceRaw() filter instead

func EchoToStderr 

func EchoToStderr(input string) Command

EchoToStderr writes a string to the pipeline's stderr

func Exec 

func Exec(args ...string) Command

Exec runs an operating system command, and posts the results to the pipeline's Stdout and Stderr.

The command's status code is stored in the pipeline.StatusCode.

func Exit 

func Exit(statusCode int) Command

Exit terminates the Golang app with the given status code.

It does *NOT* flush the pipe's Stdout or Stderr to your Golang's os.Stdout / os.Stderr first.

func Grep 

func Grep(regex string) Command

Grep filters out lines which do not match the given regex

func GrepV 

func GrepV(regex string) Command

GrepV filters out lines which do match the given regex 

func Head(n int) Command

Head copies the first N lines from the pipeline's Stdin to its Stdout

If `n` is less than 1 (ie 0, or negative), no lines are copied

func If 

func If(expr, body *Sequence) Command

If executes the body if (and only if) the expr completes without an error.

It is an emulation of UNIX shell scripting's `if expr ; then body`

func IfElse 

func IfElse(expr, body, elseBlock *Sequence) Command

IfElse executes the body if (and only if) the expr completes without an error. Otherwise, it executes the elseBlock instead.

IfElse is an emulation of UNIX shell scripting's `if expr ; then body ; else elseBlock ; fi`

func ListFiles 

func ListFiles(path string) Command

ListFiles is the equivalent of `ls -1 <path>`.

If `path` is a file, ListFiles writes the file to the pipeline's stdout If `path` is a folder, ListFiles writes the contents of the folder to the pipeline's stdout. The path to the folder is included.

If `path` contains wildcards, ListFiles writes any files that matches to the pipeline's stdout.

func Lsmod 

func Lsmod(filepath string) Command

Lsmod writes the permissions of the given filepath to the pipe's stdout. Symlinks are followed.

Permissions are in the form '-rwxrwxrwx'.

It ignores the contents of the pipeline.

func MkTempDir 

func MkTempDir(dir string, prefix string) Command

MkTempDir creates a temporary directory, and writes the filepath to the pipeline's stdout.

func MkTempFile 

func MkTempFile(dir string, pattern string) Command

MkTempFile creates a temporary file, and writes the filename to the pipeline's stdout.

func MkTempFilename 

func MkTempFilename(dir string, pattern string) Command

MkTempFilename generates a temporary filename, and writes the filename to the pipeline's stdout.

func Mkdir 

func Mkdir(filepath string, mode os.FileMode) Command

Mkdir creates the named directory, along with any parent folders that are needed.

It ignores the contents of the pipeline.

On success, it returns the status code `StatusOkay`. On failure, it returns the status code `StatusNotOkay`.

func Or 

func Or(sq *Sequence) Command

Or executes the given sequence only if the previous command has returned some kind of error.

The sequence starts with an empty Stdin. The sequence's output is written back to the Stdout and Stderr of the calling list or pipeline - along with the StatusCode() and Error().

It is an emulation of UNIX shell scripting's `list1 || command`

func Return 

func Return(statusCode int) Command

Return terminates the pipeline with the given status code.

func RmDir 

func RmDir(filepath string) Command

RmDir deletes the given folder, as long as the folder is empty.

It ignores the contents of the pipeline.

It ignores the file's file permissions, because the underlying Golang os.Remove() behaves that way.

func RmFile 

func RmFile(filepath string) Command

RmFile deletes the given file.

It ignores the contents of the pipeline.

It ignores the file's file permissions, because the underlying Golang os.Remove() behaves that way.

func Rsort 

func Rsort() Command

Rsort sorts the contents of the pipeline into descending alphabetical order

func RunList 

func RunList(pl *List) Command

RunList allows you to call one list from another.

Use this to execute reusable lists.

func RunPipeline 

func RunPipeline(pl *Pipeline) Command

RunPipeline allows you to call one pipeline from another.

Use this to create reusable pipelines.

func Sort 

func Sort() Command

Sort sorts the contents of the pipeline into ascending alphabetical order

func StripExtension 

func StripExtension() Command

StripExtension treats every line in the pipeline as a filepath. It removes the extension from each filepath.

func SwapExtensions 

func SwapExtensions(old []string, new []string) Command

SwapExtensions treats every line in the pipeline as a filepath.

It replaces every old extension with the corresponding new one.

func Tail 

func Tail(n int) Command

Tail copies the last N lines from the pipeline's Stdin to its Stdout

func TestEmpty 

func TestEmpty(input string) Command

TestEmpty returns 1 if the given input string (after expansion) is not empty

func TestFilepathExists 

func TestFilepathExists(filepath string) Command

TestFilepathExists checks to see if the given filepath exists. If it does, the filepath is written to the pipeline's Stdout.

It does not care what the filepath points at (file, folder, named pipe, and so on).

It ignores the contents of the pipeline. It follows symbolic links.

func TestNotEmpty 

func TestNotEmpty(input string) Command

TestNotEmpty returns 1 if the given input string (after expansion) is empty

func ToStderr 

func ToStderr() Command

ToStderr writes the contents of the pipeline's stdin to the program's stderr

func ToStdout 

func ToStdout() Command

ToStdout writes the contents of the pipeline's stdin to the program's stdout

func Touch 

func Touch(filepath string) Command

Touch creates the named file (if it doesn't exist), or updates its atime and mtime (if it does exist)

It ignores the contents of the pipeline.

On success, it returns the status code `StatusOkay`. On failure, it returns the status code `StatusNotOkay`.

func Tr 

func Tr(old []string, new []string) Command

Tr replaces all occurances of one string with another

func TrimSuffix 

func TrimSuffix(ext string) Command

TrimSuffix removes the given suffix from each line of the pipeline.

Use it to emulate basename(1)'s `[suffix]` parameter.

func TrimWhitespace 

func TrimWhitespace() Command

TrimWhitespace removes any whitespace from the front and end of each line

func TruncateFile 

func TruncateFile(filename string) Command

TruncateFile removes the contents of the given file.

If the file does not exist, it is created.

func Uniq 

func Uniq() Command

Uniq removes duplicated lines from the pipeline

func Which 

func Which(cmd string) Command

Which searches the current PATH to find the given path. If one is found, the command's path is written to the pipeline's stdout.

It ignores the contents of the pipeline.

func WriteToFile 

func WriteToFile(filename string) Command

WriteToFile writes the contents of the pipe's stdin to the given file. The existing contents of the file are replaced.

If the file does not exist, it is created.

func XargsBasename 

func XargsBasename() Command

XargsBasename treats every line in the pipe as a filepath. It removes any parent elements from the line.

func XargsCat 

func XargsCat() Command

XargsCat treats each line in the pipeline's stdin as a filepath. It reads each file, and writes them to the pipeline's stdout.

func XargsDirname 

func XargsDirname() Command

XargsDirname treats every line in the pipeline as a filepath. It removes the last element from each filepath.

If a line is blank, XargsDirname returns a '.'

func XargsRmFile 

func XargsRmFile() Command

XargsRmFile treats every line in the pipeline as a filename. It attempts to delete each file.

It stops at the first file that cannot be deleted.

Each successfully-deleted filepath is written to the pipeline's stdout, for use by the next command in the pipeline.

func XargsTestFilepathExists 

func XargsTestFilepathExists() Command

XargsTestFilepathExists treats each line in the pipeline as a filepath. It checks to see if the given filepath exists.

If the filepath exists, it is written to the pipeline's stdout.

It does not care what the filepath points at (file, folder, named pipe, and so on).

func XargsTruncateFiles 

func XargsTruncateFiles() Command

XargsTruncateFiles treats each line of the pipeline's stdin as a filepath. The contents of each file are truncated. If the file does not exist, it is created.

Each filepath is written to the pipeline's stdout, for use by the next operation in the pipeline.

type Dest 

type Dest = pipe.Dest

Dest is an alias for the underlying pipe library's Dest

func NewDest 

func NewDest() *Dest

NewDest creates a new pipe.Dest struct

type ErrMismatchedInputs 

type ErrMismatchedInputs struct {
	// contains filtered or unexported fields
}

ErrMismatchedInputs is the error returned when two input arrays aren't the same length

func (ErrMismatchedInputs) Error 

func (e ErrMismatchedInputs) Error() string

type List 

type List = Sequence

List is an alias for a Sequence

func ExecList 

func ExecList(steps ...Command) *List

ExecList creates and runs a list. Use this for short, throwaway actions.

func NewList 

func NewList(steps ...Command) *List

NewList creates a list ready to be executed

type Pipe 

type Pipe = pipe.Pipe

Pipe is an alias for the underlying pipe library's Pipe

It just saves us having to import the pipe library into every single file in the project.

type Pipeline 

type Pipeline = Sequence

Pipeline is an alias for a Sequence

func ExecPipeline 

func ExecPipeline(steps ...Command) *Pipeline

ExecPipeline creates and runs a pipeline. Use this for short, throwaway actions.

type Range 

type Range struct {
	Lo int
	Hi int
}

Range tracks the start and end of a given range of numbers

func ParseRangeSpec 

func ParseRangeSpec(spec string) ([]Range, error)

ParseRangeSpec takes a string of the form `X1-Y1[,X2-Y2 ...]` and turns it into a list of start and end ranges

It emulates the `cut -f <range>` range support.

type Sequence 

type Sequence struct {
	// our commands read from / write to this pipe
	Pipe *pipe.Pipe

	// keep track of the steps that belong to this sequence
	Steps []Command

	// How we will run the sequence
	Controller SequenceController

	// we store local variables here
	LocalVars *envish.LocalEnv

	// the flags we pass into new pipes
	Flags int
}

Sequence is a set of commands to be executed.

Provide your own logic to do the actual command execution.

func NewPipeline 

func NewPipeline(steps ...Command) *Sequence

NewPipeline creates a pipeline that's ready to run

func NewSequence 

func NewSequence(steps ...Command) *Sequence

NewSequence creates a sequence that's ready to run

func (*Sequence) Bytes 

func (sq *Sequence) Bytes() ([]byte, error)

Bytes returns the contents of the sequence's stdout as a byte slice

func (*Sequence) Error 

func (sq *Sequence) Error() error

Error returns the sequence's error status.

func (*Sequence) Exec 

func (sq *Sequence) Exec(params ...string) *Sequence

Exec executes a sequence

If you embed the sequence in another struct, make sure to override this to return your own return type!

func (*Sequence) Flush 

func (sq *Sequence) Flush(stdout io.Writer, stderr io.Writer)

Flush writes the output from running this sequence to the given stdout and stderr

func (*Sequence) NewPipe 

func (sq *Sequence) NewPipe()

NewPipe replaces the Sequence's existing pipe with a brand new (and empty) one. This is very useful for reusing Sequences.

This is called from various places right before a Sequence is run.

You shouldn't need to call it yourself, but it's exported just in case it's useful in some way.

func (*Sequence) Okay 

func (sq *Sequence) Okay() bool

Okay returns false if a sequence operation set the StatusCode to anything other than StatusOkay. It returns true otherwise.

func (*Sequence) ParseInt 

func (sq *Sequence) ParseInt() (int, error)

ParseInt returns the pipe's stdout as an integer

If the integer conversion fails, error will be the conversion error. If the integer conversion succeeds, error will be the pipe's error (which may be nil)

func (*Sequence) SetParams 

func (sq *Sequence) SetParams(params ...string)

SetParams sets $#, $1... and $* in the pipe's Var store

func (*Sequence) StatusCode 

func (sq *Sequence) StatusCode() int

StatusCode returns the UNIX-like status code from the last step to execute

func (*Sequence) StatusError 

func (sq *Sequence) StatusError() (int, error)

StatusError is a shorthand to save having to call Sequence.StatusCode() followed by Sequence.Error() from your code

func (*Sequence) String 

func (sq *Sequence) String() (string, error)

String returns the pipe's stdout as a single string

func (*Sequence) Strings 

func (sq *Sequence) Strings() ([]string, error)

Strings returns the sequence's stdout, one string per line

func (*Sequence) TrimmedString 

func (sq *Sequence) TrimmedString() (string, error)

TrimmedString returns the pipe's stdout as a single string. Any leading or trailing whitespace is removed.

type SequenceController 

type SequenceController func()

SequenceController is a function that executes a given sequence

func ListController 

func ListController(sq *Sequence) SequenceController

ListController executes a sequence of commands as if they were a UNIX shell list

func PipelineController 

func PipelineController(sq *Sequence) SequenceController

PipelineController executes a sequence of commands as if they were a UNIX shell pipeline

type ShellOptions 

type ShellOptions struct {
	// contains filtered or unexported fields
}

ShellOptions holds flags and settings that change Scriptish's behaviour

func GetShellOptions 

func GetShellOptions() *ShellOptions

GetShellOptions gives you access to the package-wide behaviour flags and settings

func (*ShellOptions) DisableTrace 

func (s *ShellOptions) DisableTrace()

DisableTrace will switch off execution tracing across Scriptish

func (*ShellOptions) EnableTrace 

func (s *ShellOptions) EnableTrace(dest io.Writer)

EnableTrace will switch on execution tracing across Scriptish

func (*ShellOptions) IsTraceEnabled 

func (s *ShellOptions) IsTraceEnabled() bool

IsTraceEnabled return true if execution tracing is currently switched on

type Source 

type Source = pipe.Source

Source is an alias for the underlying pipe library's Source

Source Files

View all Source files

Directories

Path Synopsis
examples
git-current-branch

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.