goyek

package module
v2.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 17, 2024 License: MIT Imports: 17 Imported by: 15

README

goyek

Build automation in Go

Go Reference Keep a Changelog GitHub Release go.mod

Build Status Go Report Card codecov Mentioned in Awesome Go

Please ⭐ Star this repository if you find it valuable and worth maintaining.

Introduction

Slides.

Table of Contents:

Description

goyek (/ˈɡɔɪæk/ 🔊 listen) is used to create build automation in Go. As opposed to many other tools, it is just a Go library with API inspired by testing, cobra, flag, http packages.

Here are some good parts:

  • It is cross-platform and shell independent.
  • No binary installation is needed.
  • It is easy to debug, like a regular Go application.
  • The tasks are defined similarly to cobra commands.
  • The task actions look like a Go unit test. You may even use testify or fluentassert for asserting.
  • You can reuse code like in any Go application. It may be helpful to use packages like fsnotify and viper.
  • It is highly customizable.
  • It does not use any third-party dependency other than the Go standard library. You can find supplumental features in goyek/x.
  • Minimal supported Go version is 1.11.

Quick start

Supplemental packages from github.com/goyek/x are used for convinence.

The convention is to have the build automation in the /build directory (or even Go module).

Put the following content in /build/hello.go:

package main

import (
	"flag"

	"github.com/goyek/goyek/v2"
	"github.com/goyek/x/cmd"
)

var msg = flag.String("msg", "greeting message", "Hello world!")

var hello = flow.Define(goyek.Task{
	Name:  "hello",
	Usage: "demonstration",
	Action: func(a *goyek.A) {
		a.Log(*msg)
		cmd.Exec(a, "go version")
	},
})

Put the following content in /build/main.go:

package main

import (
	"github.com/goyek/goyek/v2"
	"github.com/goyek/x/boot"
)

func main() {
	goyek.SetDefault(hello)
	boot.Main()
}

Run:

$ go mod tidy

$ go run ./build -h
Usage of build: [flags] [--] [tasks]
Tasks:
  hello  demonstration
Flags:
  -dry-run
        print all tasks without executing actions
  -long-run duration
        print when a task takes longer (default 1m0s)
  -msg string
        Hello world! (default "greeting message")
  -no-color
        disable colorizing output
  -no-deps
        do not process dependencies
  -skip comma-separated tasks
        skip processing the comma-separated tasks
  -v    print all tasks as they are run

$ go run ./build -v
===== TASK  hello
      hello.go:16: greeting message
      hello.go:17: Exec: go version
go version go1.19.3 windows/amd64
----- PASS: hello (0.12s)
ok      0.123s

Repository template

You can use goyek/template to create a new repository.

For an existing repository you can copy most of its files.

Examples

Defining tasks

Use Define to register a a task.

You can add dependencies to already defineded tasks using Task.Deps. The dependencies are running in sequential order. Each task runs at most once.

The Task.Action is a function which executes when a task is running. A task can have only dependencies and no action to act as a pipeline.

The Task.Parallel can be set to allow a task to be run in parallel with other parallel tasks.

A default task can be assigned using SetDefault.

Running programs

You can use the cmd.Exec convenient function from goyek/x that should cover most use cases.

Alternatively, you may prefer create your own helpers like Exec in build/exec.go.

#60 and #307 explain why this feature is not out-of-the-box.

Wrapper scripts

Instead of executing go run ./build, you may create wrapper scripts, which you can invoke from any locationn.

Bash - goyek.sh:

#!/bin/bash
set -euo pipefail

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
cd "$DIR"
go run ./build "$@"

PowerShell - goyek.ps1:

& go run .\build $args
exit $global:LASTEXITCODE

If /build is a separate Go module, check the goyek.sh and goyek.ps1 scripts.

Using middlewares

Call the Use function to setup a task runner interceptor (middleware).

You can use a middleware, for example to: generate a task execution report, add retry logic, export build execution telemetry, etc.

You can use some reusable middlewares from the middleware package. ReportStatus is the most commonly used middleware.

Notice that the boot.Main convenient function from goyek/x sets the most commonly used middlewares and defines flags to configure them.

Customizing

You can customize the default output by using:

You can also study how github.com/goyek/x is customizing the default behavior.

Alternatives

Make

While Make (Makefile) is currently the de facto standard, it has some pitfalls:

  • Requires to learn Make (and often Bash).
  • It is hard to develop a Makefile which is truly cross-platform.
  • Debugging and testing Make targets is not fun.

goyek is intended to be simpler, easier to learn, more portable, while still being able to handle most use cases.

Mage

Mage is a framework/tool which magically discovers the targets from magefiles, which results in some drawbacks.

goyek is a non-magical alternative for Mage. It is easier to customize and extend as it is a library that offers extension points. Write regular Go code without build tags and tricky imports.

Task

While Task is simpler and easier to use than Make, but it still has similar problems:

Bazel

Bazel is a very sophisticated tool which is created to efficiently handle complex and long-running build pipelines. It requires the build target inputs and outputs to be fully specified.

goyek is just a simple Go library. However, nothing prevents you from, for example, using the github.com/magefile/mage/target package to make your automation more efficient.

Contributing

See CONTRIBUTING.md if you want to help us.

License

goyek is licensed under the terms of the MIT license.

Note: goyek was named taskflow before v0.3.0.

Documentation

Overview

Package goyek helps implementing build automation.

Example 
// define a task printing the message (configurable via flag)
hi := goyek.Define(goyek.Task{
	Name:  "hi",
	Usage: "Greetings",
	Action: func(a *goyek.A) {
		a.Log("Hello world!")
	},
})

// define a task running a command
goVer := goyek.Define(goyek.Task{
	Name:  "go-ver",
	Usage: `Run "go version"`,
	Action: func(a *goyek.A) {
		cmd := exec.CommandContext(a.Context(), "go", "version")
		cmd.Stdout = a.Output()
		cmd.Stderr = a.Output()
		if err := cmd.Run(); err != nil {
			a.Error(err)
		}
	},
})

// define a pipeline
all := goyek.Define(goyek.Task{
	Name: "all",
	Deps: goyek.Deps{hi, goVer},
})

// configure middlewares
goyek.Use(middleware.ReportStatus)

// set the pipeline as the default task
goyek.SetDefault(all)

// run the build pipeline
goyek.Main(os.Args[1:])

/*
	$ go run .
	===== TASK  hi
	      main.go:15: Hello world!
	----- PASS: hi (0.00s)
	===== TASK  go-ver
	go version go1.19.2 windows/amd64
	----- PASS: go-ver (0.04s)
	===== TASK  all
	----- NOOP: all (0.00s)
	ok      0.039s
*/ 0.039s
*/

Output:
Example (Flag) 
// use the same output for flow and flag
flag.CommandLine.SetOutput(goyek.Output())

// define a flag to configure flow output verbosity
verbose := flag.Bool("v", true, "print all tasks as they are run")

// define a flag used by a task
msg := flag.String("msg", "hello world", `message to display by "hi" task`)

// define a task printing the message (configurable via flag)
goyek.Define(goyek.Task{
	Name:  "hi",
	Usage: "Greetings",
	Action: func(a *goyek.A) {
		a.Log(*msg)
	},
})

// set the help message
usage := func() {
	fmt.Println("Usage of build: [flags] [--] [tasks]")
	goyek.Print()
	fmt.Println("Flags:")
	flag.PrintDefaults()
}

// parse the args
flag.Usage = usage
flag.Parse()

// configure middlewares
goyek.Use(middleware.ReportStatus)
if !*verbose {
	goyek.Use(middleware.SilentNonFailed)
}

// run the build pipeline
goyek.SetUsage(usage)
goyek.Main(flag.Args())

/*
	$ go run .
	no task provided
	Usage of build: [flags] [--] [tasks]
	Tasks:
	   hi   Greetings
	Flags:
	  -msg string
	        message to display by "hi" task (default "hello world")
	  -v    print all tasks as they are run (default true)
	exit status 2
*/tatus 2
*/

Output:

Index

Examples

Constants

This section is empty.

Variables

View Source 
var DefaultFlow = &Flow{}

DefaultFlow is the default flow. The top-level functions such as Define, Main, and so on are wrappers for the methods of Flow.

Functions

func Execute 

func Execute(ctx context.Context, tasks []string, opts ...Option) error

Execute runs provided tasks and all their dependencies. Each task is executed at most once. Returns nil if no task has failed, FailError if a task failed, other errors in case of invalid input or context error.

func Main 

func Main(args []string, opts ...Option)

Main runs provided tasks and all their dependencies. Each task is executed at most once. It exits the current program when after the run is finished or SIGINT interrupted the execution.

  • 0 exit code means that non of the tasks failed.
  • 1 exit code means that a task has failed or the execution was interrupted.
  • 2 exit code means that the input was invalid.

Calls Usage when invalid args are provided.

func Output 

func Output() io.Writer

Output returns the destination used for printing messages. os.Stdout is returned if output was not set or was set to nil.

func Print 

func Print()

Print prints the information about the registered tasks. Tasks with empty [Task.Usage] are not printed.

func SetDefault 

func SetDefault(task *DefinedTask)

SetDefault sets a task to run when none is explicitly provided. It panics in case of any error.

func SetLogger 

func SetLogger(logger Logger)

SetLogger sets the logger used by A's logging functions.

A uses following methods if implemented: 

Error(w io.Writer, args ...interface{})
Errorf(w io.Writer, format string, args ...interface{})
Fatal(w io.Writer, args ...interface{})
Fatalf(w io.Writer, format string, args ...interface{})
Skip(w io.Writer, args ...interface{})
Skipf(w io.Writer, format string, args ...interface{})
Helper()

func SetOutput 

func SetOutput(out io.Writer)

SetOutput sets the output destination.

func SetUsage 

func SetUsage(fn func())

SetUsage sets the function called when an error occurs while parsing tasks.

func Undefine 

func Undefine(task *DefinedTask)

Undefine unregisters the task. It panics in case of any error.

func Usage 

func Usage() func()

Usage returns a function that prints a usage message documenting the flow. It is called when an error occurs while parsing the flow. Print is returned if a function was not set or was set to nil.

func Use 

func Use(middlewares ...Middleware)

Use adds task runner middlewares (interceptors).

Types

type A 

type A struct {
	// contains filtered or unexported fields
}

A is a type passed to [Task.Action] functions to manage task state and support formatted task logs.

A task ends when its action function returns or calls any of the methods FailNow, Fatal, Fatalf, SkipNow, Skip, or Skipf. Those methods must be called only from the goroutine running the action function.

The other reporting methods, such as the variations of Log and Error, may be called simultaneously from multiple goroutines.

func (*A) Cleanup 

func (a *A) Cleanup(fn func())

Cleanup registers a function to be called when [Task.Action] function completes. Cleanup functions will be called in the last-added first-called order.

func (*A) Context 

func (a *A) Context() context.Context

Context returns the run context.

func (*A) Error 

func (a *A) Error(args ...interface{})

Error is equivalent to A.Log followed by A.Fail.

func (*A) Errorf 

func (a *A) Errorf(format string, args ...interface{})

Errorf is equivalent to A.Logf followed by A.Fail.

func (*A) Fail 

func (a *A) Fail()

Fail marks the function as having failed but continues execution.

func (*A) FailNow 

func (a *A) FailNow()

FailNow marks the function as having failed and stops its execution by calling runtime.Goexit (which then runs all deferred calls in the current goroutine). It finishes the whole flow execution. FailNow must be called from the goroutine running the [Task.Action] function, not from other goroutines created during its execution. Calling FailNow does not stop those other goroutines.

func (*A) Failed 

func (a *A) Failed() bool

Failed reports whether the function has failed.

func (*A) Fatal 

func (a *A) Fatal(args ...interface{})

Fatal is equivalent to A.Log followed by A.FailNow.

func (*A) Fatalf 

func (a *A) Fatalf(format string, args ...interface{})

Fatalf is equivalent to A.Logf followed by A.FailNow.

func (*A) Helper 

func (a *A) Helper()

Helper marks the calling function as a helper function. It calls logger's Helper method if implemented. By default, when printing file and line information, that function will be skipped.

func (*A) Log 

func (a *A) Log(args ...interface{})

Log formats its arguments using default formatting, analogous to Println, and writes the text to A.Output. A final newline is added.

func (*A) Logf 

func (a *A) Logf(format string, args ...interface{})

Logf formats its arguments according to the format, analogous to Printf, and writes the text to A.Output. A final newline is added.

func (*A) Name 

func (a *A) Name() string

Name returns the name of the running task.

func (*A) Output 

func (a *A) Output() io.Writer

Output returns the destination used for printing messages.

func (*A) Setenv 

func (a *A) Setenv(key, value string)

Setenv calls os.Setenv(key, value) and uses Cleanup to restore the environment variable to its original value after the action.

func (*A) Skip 

func (a *A) Skip(args ...interface{})

Skip is equivalent to A.Log followed by A.SkipNow.

func (*A) SkipNow 

func (a *A) SkipNow()

SkipNow marks the task as having been skipped and stops its execution by calling runtime.Goexit (which then runs all deferred calls in the current goroutine). If a test fails (see Error, Errorf, Fail) and is then skipped, it is still considered to have failed. The flow execution will continue at the next task. See also A.FailNow. SkipNow must be called from the goroutine running the [Task.Action] function, not from other goroutines created during its execution. Calling SkipNow does not stop those other goroutines.

func (*A) Skipf 

func (a *A) Skipf(format string, args ...interface{})

Skipf is equivalent to A.Logf followed by A.SkipNow.

func (*A) Skipped 

func (a *A) Skipped() bool

Skipped reports whether the task was skipped.

func (*A) TempDir 

func (a *A) TempDir() string

TempDir returns a temporary directory for the action to use. The directory is automatically removed by Cleanup when the action completes. Each subsequent call to TempDir returns a unique directory; if the directory creation fails, TempDir terminates the action by calling Fatal.

type CodeLineLogger 

type CodeLineLogger struct {
	// contains filtered or unexported fields
}

CodeLineLogger decorates the log with code line information and indentation.

func (*CodeLineLogger) Helper 

func (l *CodeLineLogger) Helper()

Helper marks the calling function as a helper function. When printing file and line information, that function will be skipped. Helper may be called simultaneously from multiple goroutines.

func (*CodeLineLogger) Log 

func (l *CodeLineLogger) Log(w io.Writer, args ...interface{})

Log is used by A logging functions.

func (*CodeLineLogger) Logf 

func (l *CodeLineLogger) Logf(w io.Writer, format string, args ...interface{})

Logf is used by A logging functions.

type DefinedTask 

type DefinedTask struct {
	// contains filtered or unexported fields
}

DefinedTask represents a task that has been defined. It can be used as a dependency for another task.

func Default 

func Default() *DefinedTask

Default returns the default task. nil is returned if default was not set.

func Define 

func Define(task Task) *DefinedTask

Define registers the task. It panics in case of any error.

func Tasks 

func Tasks() []*DefinedTask

Tasks returns all tasks sorted in lexicographical order.

func (*DefinedTask) Action 

func (r *DefinedTask) Action() func(a *A)

Action returns the action of the task.

func (*DefinedTask) Deps 

func (r *DefinedTask) Deps() Deps

Deps returns all task's dependencies.

func (*DefinedTask) Name 

func (r *DefinedTask) Name() string

Name returns the name of the task.

func (*DefinedTask) SetAction 

func (r *DefinedTask) SetAction(fn func(a *A))

SetAction changes the action of the task.

func (*DefinedTask) SetDeps 

func (r *DefinedTask) SetDeps(deps Deps)

SetDeps sets all task's dependencies.

func (*DefinedTask) SetName 

func (r *DefinedTask) SetName(s string)

SetName changes the name of the task.

func (*DefinedTask) SetUsage 

func (r *DefinedTask) SetUsage(s string)

SetUsage sets the description of the task.

func (*DefinedTask) Usage 

func (r *DefinedTask) Usage() string

Usage returns the description of the task.

type Deps 

type Deps []*DefinedTask

Deps represents a collection of dependencies.

type FailError 

type FailError struct {
	Task string
}

FailError pointer is returned by Flow.Execute when a task failed.

func (*FailError) Error 

func (err *FailError) Error() string

type Flow 

type Flow struct {
	// contains filtered or unexported fields
}

Flow is the root type of the package. Use Register methods to register all tasks and Run or Main method to execute provided tasks.

func (*Flow) Default 

func (f *Flow) Default() *DefinedTask

Default returns the default task. nil is returned if default was not set.

func (*Flow) Define 

func (f *Flow) Define(task Task) *DefinedTask

Define registers the task. It panics in case of any error.

func (*Flow) Execute 

func (f *Flow) Execute(ctx context.Context, tasks []string, opts ...Option) error

Execute runs provided tasks and all their dependencies. Each task is executed at most once. Returns nil if no task has failed, FailError if a task failed, other errors in case of invalid input or context error.

func (*Flow) Logger 

func (f *Flow) Logger() Logger

Logger returns the logger used by A's logging functions CodeLineLogger is returned if logger was not set or was set to nil.

func (*Flow) Main 

func (f *Flow) Main(args []string, opts ...Option)

Main runs provided tasks and all their dependencies. Each task is executed at most once. It exits the current program when after the run is finished or SIGINT interrupted the execution.

  • 0 exit code means that non of the tasks failed.
  • 1 exit code means that a task has failed or the execution was interrupted.
  • 2 exit code means that the input was invalid.

Calls Usage when invalid args are provided.

func (*Flow) Output 

func (f *Flow) Output() io.Writer

Output returns the destination used for printing messages. os.Stdout is returned if output was not set or was set to nil.

func (*Flow) Print 

func (f *Flow) Print()

Print prints the information about the registered tasks. Tasks with empty [Task.Usage] are not printed.

func (*Flow) SetDefault 

func (f *Flow) SetDefault(task *DefinedTask)

SetDefault sets a task to run when none is explicitly provided. Passing nil clears the default task. It panics in case of any error.

func (*Flow) SetLogger 

func (f *Flow) SetLogger(logger Logger)

SetLogger sets the logger used by A's logging functions.

A uses following methods if implemented: 

Error(w io.Writer, args ...interface{})
Errorf(w io.Writer, format string, args ...interface{})
Fatal(w io.Writer, args ...interface{})
Fatalf(w io.Writer, format string, args ...interface{})
Skip(w io.Writer, args ...interface{})
Skipf(w io.Writer, format string, args ...interface{})
Helper()

func (*Flow) SetOutput 

func (f *Flow) SetOutput(out io.Writer)

SetOutput sets the output destination.

func (*Flow) SetUsage 

func (f *Flow) SetUsage(fn func())

SetUsage sets the function called when an error occurs while parsing tasks.

func (*Flow) Tasks 

func (f *Flow) Tasks() []*DefinedTask

Tasks returns all tasks sorted in lexicographical order.

func (*Flow) Undefine 

func (f *Flow) Undefine(task *DefinedTask)

Undefine unregisters the task. It panics in case of any error.

func (*Flow) Usage 

func (f *Flow) Usage() func()

Usage returns a function that prints a usage message documenting the flow. It is called when an error occurs while parsing the flow. Flow.Print is returned if a function was not set or was set to nil.

func (*Flow) Use 

func (f *Flow) Use(middlewares ...Middleware)

Use adds task runner middlewares (interceptors).

type FmtLogger 

type FmtLogger struct{}

FmtLogger uses fmt when logging. It only appends a new line at the end.

func (FmtLogger) Log 

func (l FmtLogger) Log(w io.Writer, args ...interface{})

Log is used by A logging functions.

func (FmtLogger) Logf 

func (l FmtLogger) Logf(w io.Writer, format string, args ...interface{})

Logf is used by A logging functions.

type Input 

type Input struct {
	Context  context.Context
	TaskName string
	Parallel bool
	Output   io.Writer
	Logger   Logger
}

Input received by the task runner.

type Logger 

type Logger interface {
	Log(w io.Writer, args ...interface{})
	Logf(w io.Writer, format string, args ...interface{})
}

Logger is used by A's logging functions.

func GetLogger 

func GetLogger() Logger

GetLogger returns the logger used by A's logging functions CodeLineLogger is returned if logger was not set or was set to nil.

type Middleware 

type Middleware func(Runner) Runner

Middleware represents a task runner interceptor.

type Option 

type Option interface {
	// contains filtered or unexported methods
}

Option configures the flow execution.

func NoDeps 

func NoDeps() Option

NoDeps is an option to skip processing of all dependencies.

func Skip 

func Skip(tasks ...string) Option

Skip is an option to skip processing of given tasks.

type Result 

type Result struct {
	Status     Status
	PanicValue interface{}
	PanicStack []byte
}

Result of a task run.

type Runner 

type Runner func(Input) Result

Runner represents a task runner function.

func NewRunner 

func NewRunner(action func(a *A)) Runner

NewRunner returns a task runner used by Flow.

It can be useful for testing and debugging a task action or middleware.

The following defaults are set for Input (take notice that they are different than Flow defaults): 

Context = context.Background()
Output = ioutil.Discard
Logger = FmtLogger{}

It can be also used as a building block for a custom workflow runner if you are missing any functionalities provided by Flow (like concurrent dependencies execution).

type Status 

type Status uint8

Status of a task run. 

const (
	StatusNotRun Status = iota
	StatusPassed
	StatusFailed
	StatusSkipped
)

Statuses of task run.

func (Status) String 

func (s Status) String() string

type Task 

type Task struct {
	// Name uniquely identifies the task.
	// It cannot be empty and should be easily representable on the CLI.
	Name string

	// Usage provides information what the task does.
	Usage string

	// Action is function that is called when the task is run.
	// A task can have only dependencies and no action to act as a pipeline.
	Action func(a *A)

	// Deps is a collection of defined tasks
	// that need to be run before this task is executed.
	Deps Deps

	// Parallel marks that this task can be run in parallel
	// with (and only with) other parallel tasks.
	Parallel bool
}

Task represents a named task that can have action and dependencies.

Source Files

View all Source files

Directories

Path Synopsis
Package middleware provides reusable task runner interceptors such as ReportStatus.
 Package middleware provides reusable task runner interceptors such as ReportStatus.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.