goyek

goyek

Create build pipelines in Go

Please ⭐ Star this repository if you find it valuable and worth maintaining.

Table of Contents:

• Description
• Quick start
• Wrapper scripts
• Repository template
• Examples
• Features
  • Task registration
  • Task action
  • Task dependencies
  • Helpers for running programs
  • Verbose mode
  • Default task
  • Parameters
  • Supported Go versions
• Alternatives
  • Make
  • Mage
  • Task
  • Bazel
• Presentations
• Contributing
• License

Description

goyek (/ˈɡɔɪæk/ 🔊 listen) is used to create build pipelines in Go. As opposed to many other tools, it is just a Go library.

Here are some good parts:

• It is cross-platform and shell independent.
• No binary installation is needed. Simply add it to go.mod like any other Go module.
  • You can be sure that everyone uses the same version of goyek.
• It has low learning curve, thanks to the minimal API surface, documentation, and examples.
• The task's action look like a unit test. It is even possible to use testify or is for asserting.
• It is easy to debug, like a regular Go application.
• You can reuse code like in any Go application. It may be helpful to use packages like:
  • github.com/bitfield/script
  • github.com/rjeczalik/notify
  • github.com/magefile/mage/target
  • github.com/mattn/go-shellwords
• Tasks and helpers can be unit tested. For example, see cmd_test.go.
• github.com/goyek/goyek package does not use any third-party dependency other than the Go standard library.

goyek API is mainly inspired by the testing, http, and flag packages.

Quick start

Copy and paste the following code into build/build.go:

package main

import (
	"github.com/goyek/goyek"
)

func main() {
	flow := &goyek.Flow{}

	flow.Register(goyek.Task{
		Name:  "hello",
		Usage: "demonstration",
		Action: func(tf *goyek.TF) {
			tf.Log("Hello world!")
		},
	})

	flow.Main()
}

Run:

go mod tidy

Sample usage:

$ go run ./build -h
Usage: [flag(s) | task(s)]...
Flags:
  -v     Default: false    Verbose: log all tasks as they are run.
  -wd    Default: .        Working directory: set the working directory.
Tasks:
  hello    demonstration

$ go run ./build hello
ok     0.000s

$ go run ./build hello -v
===== TASK  hello
      main.go:14: Hello world!
----- PASS: hello (0.00s)
ok      0.001s

Wrapper scripts

Instead of executing go run ./build, you can use the wrapper scripts, which can be invoked from any location.

• goyek.sh
• goyek.ps1

Simply add them to your repository's root directory and make sure to add the +x permission:

curl -sSfL https://raw.githubusercontent.com/goyek/goyek/main/goyek.sh -O
curl -sSfL https://raw.githubusercontent.com/goyek/goyek/main/goyek.ps1 -O
git add --chmod=+x goyek.sh goyek.ps1

Repository template

You can use goyek/template to create a new repository

For an existing repository you can copy most of its files.

Examples

• examples
• build/build.go - this repository's own build pipeline ()
• fluentassert - a library using goyek without polluting it's root go.mod
• splunk-otel-go - a multi-module monorepo using goyek

Features

Task registration

The registered tasks are required to have a non-empty name, matching the regular expression TaskNamePattern. This means the following are acceptable:

• letters (a-z and A-Z)
• digits (0-9)
• underscore (_)
• hyphen (-) - except at the beginning
• plus (+) - except at the beginning
• colon (:) - except at the beginning

A task with a given name can be only registered once.

A task without description is not listed in CLI usage.

Task action

Task action is a function which is executed when a task is executed.

It is not required to set a action. Not having a action is very handy when registering "pipelines".

Task dependencies

During task registration it is possible to add a dependency to an already registered task. When the flow is processed, it makes sure that the dependency is executed before the current task is run. Take note that each task will be executed at most once.

Helpers for running programs

Use func (tf *TF) Cmd(name string, args ...string) *exec.Cmd to run a program inside a task's action.

You can use it create your own helpers, for example:

import (
	"fmt"
	"os/exec"

	"github.com/goyek/goyek"
	"github.com/mattn/go-shellwords"
)

func Exec(cmdLine string) func(tf *goyek.TF) {
	args, err := shellwords.Parse(cmdLine)
	if err != nil {
		panic(fmt.Sprintf("parse command line: %v", err))
	}
	return func(tf *goyek.TF) {
    tf.Logf("Run '%s'", cmdLine)
		if err := tf.Cmd(args[0], args[1:]...).Run(); err != nil {
			tf.Error(err)
		}
	}
}

Here is the explanation why argument splitting is not included out-of-the-box.

Verbose mode

Enable verbose output using the -v CLI flag. It works similar to go test -v. Verbose mode streams all logs to the output. If it is disabled, only logs from failed task are send to the output.

Use func (f *Flow) VerboseParam() BoolParam if you need to check if verbose mode was set within a task's action.

The default value for the verbose parameter can be overwritten with func (f *Flow) RegisterVerboseParam(p BoolParam) RegisteredBoolParam.

Default task

Default task can be assigned via the Flow.DefaultTask field.

When the default task is set, then it is run if no task is provided via CLI.

Parameters

The parameters can be set via CLI using the flag syntax.

On the CLI, flags can be set in the following ways:

• -param simple - for simple single-word values
• -param "value with blanks"
• -param="value with blanks"
• -param - setting boolean parameters implicitly to true

For example, ./goyek.sh test -v -pkg ./... would run the test task with v bool parameter (verbose mode) set to true, and pkg string parameter set to "./...".

Parameters must first be registered via func (f *Flow) RegisterValueParam(newValue func() ParamValue, info ParamInfo) ValueParam, or one of the provided methods like RegisterStringParam.

The registered parameters are required to have a non-empty name, matching the regular expression ParamNamePattern. This means the following are acceptable:

• letters (a-z and A-Z)
• digits (0-9)
• underscore (_) - except at the beginning
• hyphen (-) - except at the beginning
• plus (+) - except at the beginning
• colon (:) - except at the beginning

After registration, tasks need to specify which parameters they will read. Do this by assigning the RegisteredParam instance from the registration result to the Task.Params field. If a task tries to retrieve the value from an unregistered parameter, the task will fail.

When registration is done, the task's action can retrieve the parameter value using the Get(*TF) method from the registration result instance during the task's Action execution.

See examples/parameters/main.go for a detailed example.

See examples/validation/main.go for a detailed example of cross-parameter validation.

Flow will fail execution if there are unused parameters.

Supported Go versions

Minimal supported Go version is 1.11.

Alternatives

Make

While Make (Makefile) is currently the de facto standard, it has some pitfalls:

• Requires to learn Make, which is not so easy.
• It is hard to develop a Makefile which is truly cross-platform.
• Debugging and testing Make targets is not fun.

However, if you know Make and are happy with it, do not change it. Make is very powerful and a lot of stuff can be made faster, if you know how to use it.

goyek is intended to be simpler and easier to learn, more portable, while still being able to handle most use cases.

Mage

Mage is a framework/tool which magically discovers the targets from magefiles, which results in some drawbacks:

• Requires using build tags.
• Reusing tasks is hacky.
• Requires installation or using zero install option which is slow.
• Debugging would be extermly complex.
• Magical by design (of course one may like it).

goyek is a non-magical alternative for Mage. Write regular Go code. No build tags, special names for functions, tricky imports.

Task

While Task is simpler and easier to use than Make, but it still has similar problems:

• Requires to learn Task's YAML structure and the minimalistic, cross-platform interpreter which it uses.
• Debugging and testing tasks is not fun.
• Hard to make reusable tasks.
• Requires to "install" the tool.

Bazel

Bazel is a very sophisticated tool which is created to efficiently handle complex and long-running build pipelines. It requires the build target inputs and outputs to be fully specified.

goyek is just a simple Go library that is mainly supposed to create a build pipeline consisting of commands like go vet, go test, go build. However, nothing prevents you from, for example, using the Mage's target package to make your build pipeline more efficient.

Presentations

Note: goyek was named taskflow before v0.3.0.

Contributing

See CONTRIBUTING.md if you want to help us.

License

goyek is licensed under the terms of the MIT license.