taskrunner

package module
v0.0.0-...-a8e4c5d Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 14, 2024 License: MIT Imports: 24 Imported by: 0

README

Taskrunner

A configurable taskrunner written in Go. Taskrunner can streamline your build system by creating reusable task definitions with names, descriptions, dependencies, and a run function written in Go. Taskrunner comes with a shell interpreter, so it can run arbitrary functions on the command line.

Setup

Add taskrunner as a dependency go get -v github.com/pylon-labs/taskrunner, and create the following main.go file:

package main

import (
	"github.com/pylon-labs/taskrunner"
	"github.com/pylon-labs/taskrunner/clireporter"
)

func main() {
	taskrunner.Run(clireporter.StdoutOption)
}

Run a task

Taskrunner can be started by running go run ., it is possible to limit log output with PRINT_LOG_LEVEL=error. To run an individual task specify the task name go run . my/task. For a full list of tasks use go run . -describe, or for a list of names go run . -list. It may be a good idea to alias taskrunner to run taskrunner.

Example tasks

Add a tasks.go file, this contains all task descriptions:

package main

import (
	"context"

	"github.com/pylon-labs/taskrunner"
	"github.com/pylon-labs/taskrunner/goextensions"
	"github.com/pylon-labs/taskrunner/shell"
)

// Run a simple task.
var myTask = taskrunner.Add(&taskrunner.Task{
	Name:         "my/task",
	RunWithFlags: func(ctx context.Context, shellRun shell.ShellRun, flags map[string]taskrunner.FlagArg) error {
		return shellRun(ctx, `echo Hello World`)
	},
})

// Run a task which performs different behaviour based on
// the flags passed to it via CLI.
var myTaskWithFlags = taskrunner.Add(&taskrunner.Task{
	Name:         "my/go/task",
	Description:  "Run a go file and re-run when it changes",
	RunWithFlags: func(ctx context.Context, shellRun shell.ShellRun, flags map[string]taskrunner.FlagArg) error {
		// Note: You can also check the ShortName string i.e. flags["b"].
		if flag, ok := flags["boolFlag"]; ok {
			// Note: FlagArgs include helper fns: BoolVal, StringVal, IntVal, Float64Val
			// and DurationVal. They cast the arg passed via CLI into the appropriate
			// type based on the flag ValueType.
			// The raw value passed via CLI is also avaialble via the FlagArg helper fn Value. 
			if flag.BoolVal() {
				return ShellRun(ctx, `echo Hello World: 1`)
			}
		}
			
		return shellRun(ctx, `echo Hello World: 2`)
	},
	Flags: []taskrunner.TaskFlag{
		{
			Description: "Passing the `--boolFlag/-b` flag has X effect on the task.",
			LongName: "boolFlag",
			ShortName: []rune("b")[0],
			Default: "false",
			// Note: BoolTypeFlag, StringTypeFlag, Float64TypeFlag and DurationFlag.
			ValueType: taskrunner.BoolTypeFlag,
		}
	},
})

// Run a task depending on another task.
var myDependentTask = taskrunner.Add(&taskrunner.Task{
	Name:         "my/dependent/task",
	Dependencies: []*taskrunner.Task{myTask},
	RunWithFlags: func(ctx context.Context, shellRun shell.ShellRun flags map[string]taskrunner.FlagArg) error {
		return shellRun(ctx, `echo Hello Again`)
	},
})


// Run a task which monitors a file and reruns on changes
// a change will also invalidate dependencies.
var myGoTask = taskrunner.Add(&taskrunner.Task{
	Name:         "my/go/task",
	Description:  "Run a go file and re-run when it changes",
	RunWithFlags: func(ctx context.Context, shellRun shell.ShellRun, flags map[string]taskrunner.FlagArg) error {
		return shellRun(ctx, `cd src/example && go run .`)
	},
	Sources: []string{"src/example/**/*.go"},
})

// Run a task using WrapWithGoBuild to automatically setup
// invalidation for the task according to the import graph
// of the specified package.
var builder = goextensions.NewGoBuilder()

var myWrappedTask = taskrunner.Add(&taskrunner.Task{
	Name: "my/wrapped/task",
        RunWithFlags: func(ctx context.Context, shellRun shell.ShellRun, flags map[string]taskrunner.FlagArg) error {
		return shellRun(ctx, `example`)
	},
}, builder.WrapWithGoBuild("example"))

Default tasks to run

It is possible to add a workspace.taskrunner.json file, this contains the default tasks to run when taskrunner is run without any arguments.

{
  "path": "./",
  "desiredTasks": [
    "my/task"
  ]
}

Documentation

Index

Constants

View Source
const (
	ExecutorEventKind_TaskStarted     ExecutorEventKind = "task.started"
	ExecutorEventKind_TaskCompleted                     = "task.completed"
	ExecutorEventKind_TaskFailed                        = "task.failed"
	ExecutorEventKind_TaskStopped                       = "task.stopped"
	ExecutorEventKind_TaskInvalidated                   = "task.invalidated"
	ExecutorEventKind_TaskDiagnostic                    = "task.diagnostic"
	ExecutorEventKind_TaskLog                           = "task.log"
	ExecutorEventKind_ExecutorSetup                     = "executor.setup"
)
View Source
const (
	StringTypeFlag   string = "string"
	BoolTypeFlag     string = "bool"
	IntTypeFlag      string = "int"
	Float64TypeFlag  string = "float64"
	DurationTypeFlag string = "duration"
)
View Source
const (
	TaskHandlerExecutionState_Invalid  TaskHandlerExecutionState = "invalid"
	TaskHandlerExecutionState_Running                            = "running"
	TaskHandlerExecutionState_Error                              = "error"
	TaskHandlerExecutionState_Done                               = "done"
	TaskHandlerExecutionState_Canceled                           = "canceled"
)

Variables

Add registers a task, failing if the name has already been taken.

View Source
var DefaultRegistry = NewRegistry()

DefaultRegistry is where tasks are registered to through the package-level functions Add, Group and Tasks.

View Source
var Group = DefaultRegistry.Group

Group creates a pseudo-task that groups other tasks underneath it.

View Source
var TaskHandlerExecutionStateMap = map[taskExecutionState]TaskHandlerExecutionState{
	// contains filtered or unexported fields
}
View Source
var Tasks = DefaultRegistry.Tasks

Tasks returns all registered tasks in alphabetical order.

Functions

func IsTaskSource

func IsTaskSource(task *Task, path string) (matches bool)

IsTaskSource checks if a provided path matches a source glob for the task.

func Logf

func Logf(ctx context.Context, f string, args ...interface{})

Logf writes a log message to the stdout logger for the task. It will append a newline if one is not present at the end.

func Run

func Run(options ...RunOption)

Types

type DependencyChange

type DependencyChange struct {
	Source *Task
}

func (DependencyChange) Description

func (f DependencyChange) Description() string

func (DependencyChange) Reason

type Executor

type Executor struct {
	// contains filtered or unexported fields
}

Executor constructs and executes a DAG for the tasks specified and desired. It maintains the state of execution for individual tasks, accepts invalidation events, and schedules re-executions when necessary.

func NewExecutor

func NewExecutor(config *config.Config, tasks []*Task, opts ...ExecutorOption) *Executor

NewExecutor initializes a new executor.

func (*Executor) Config

func (e *Executor) Config() *config.Config

Config returns the taskrunner configuration.

func (*Executor) Invalidate

func (e *Executor) Invalidate(task *Task, event InvalidationEvent)

Invalidate marks a task and its dependencies as invalidated. If any tasks become invalidated from this call, Invalidate() will also schedule a re-execution of the DAG.

func (*Executor) Run

func (e *Executor) Run(ctx context.Context, taskNames []string, runtime *Runtime) error

func (*Executor) ShellRun

func (e *Executor) ShellRun(ctx context.Context, command string, opts ...shell.RunOption) error

ShellRun executes a shell.Run with some default options: Commands for tasks are automatically logged (stderr and stdout are forwarded). Commands run in a consistent environment (configurable on a taskrunner level). Commands run in taskrunner's working directory.

func (*Executor) Subscribe

func (e *Executor) Subscribe() (events <-chan ExecutorEvent)

Subscribe returns a channel of executor-level events. Each invocation of Events() returns a new channel. The done function should be called to unregister this channel.

func (*Executor) Tasks

func (e *Executor) Tasks() []*TaskHandler

type ExecutorEvent

type ExecutorEvent interface {
	Timestamp() time.Time
	Kind() ExecutorEventKind
	TaskHandler() *TaskHandler
}

type ExecutorEventKind

type ExecutorEventKind string

type ExecutorOption

type ExecutorOption func(*Executor)

func ShellRunOptions

func ShellRunOptions(opts ...shell.RunOption) ExecutorOption

func WithWatchMode

func WithWatchMode(watchMode bool) ExecutorOption

WithWatchMode controls the file watching mode.

func WithWatcherEnhancer

func WithWatcherEnhancer(we WatcherEnhancer) ExecutorOption

WithWatcherEnhancer adds a watcher enhancer to run when creating the enhancer.

type ExecutorSetupEvent

type ExecutorSetupEvent struct {
	// contains filtered or unexported fields
}

func (*ExecutorSetupEvent) Kind

func (ExecutorSetupEvent) TaskHandler

func (e ExecutorSetupEvent) TaskHandler() *TaskHandler

func (ExecutorSetupEvent) Timestamp

func (e ExecutorSetupEvent) Timestamp() time.Time

type FileChange

type FileChange struct {
	File string
}

func (FileChange) Description

func (f FileChange) Description() string

func (FileChange) Reason

func (f FileChange) Reason() InvalidationReason

type FlagArg

type FlagArg struct {
	Value       interface{}
	StringVal   func() *string
	BoolVal     func() *bool
	IntVal      func() *int
	Float64Val  func() *float64
	DurationVal func() *time.Duration
}

type InvalidationEvent

type InvalidationEvent interface {
	Reason() InvalidationReason
	Description() string
}

type InvalidationReason

type InvalidationReason int
const (
	InvalidationReason_Invalid InvalidationReason = iota
	InvalidationReason_FileChange
	InvalidationReason_DependencyChange
	InvalidationReason_KeepAliveStopped
	InvalidationReason_UserRestart
)

type KeepAliveStopped

type KeepAliveStopped struct{}

func (KeepAliveStopped) Description

func (f KeepAliveStopped) Description() string

func (KeepAliveStopped) Reason

type LogProvider

type LogProvider func(task *Task) (*Logger, error)

type Logger

type Logger struct {
	Stdout io.Writer
	Stderr io.Writer
}

func LoggerFromContext

func LoggerFromContext(ctx context.Context) *Logger

LoggerFromContext gets the logger from the context.

type Registry

type Registry struct {
	// contains filtered or unexported fields
}

Registry is an object that keeps track of task definitions.

func NewRegistry

func NewRegistry() *Registry

NewRegistry creates a new task registry.

func (*Registry) Add

func (r *Registry) Add(t *Task, opts ...TaskOption) *Task

Add registers a task, failing if the name has already been taken.

func (*Registry) Group

func (r *Registry) Group(name string, tasks ...*Task)

Group creates a pseudo-task that groups other tasks underneath it. It explicitly doesn't expose the pseudo-task because groups are not allowed to be dependencies.

func (*Registry) Tasks

func (r *Registry) Tasks() (list []*Task)

Tasks returns all registered tasks in alphabetical order.

type RunOption

type RunOption func(options *Runtime)

func ExecutorOptions

func ExecutorOptions(opts ...ExecutorOption) RunOption

type RunnerLogger

type RunnerLogger interface {
	Printf(format string, v ...interface{})
	Println(v ...interface{})
	Fatalf(format string, v ...interface{})
	Fatalln(v ...interface{})
}

type Runtime

type Runtime struct {
	// contains filtered or unexported fields
}

Runtime represents the external interface of an Executor's runtime. It is how taskrunner extensions can register themselves to taskrunner's lifecycle.

func (*Runtime) OnStart

func (r *Runtime) OnStart(f func(context.Context, *Executor) error)

OnStart is run after taskrunner has built up its task execution list.

func (*Runtime) OnStop

func (r *Runtime) OnStop(f func(context.Context, *Executor) error)

OnStop is before taskrunner exits (either because its tasks have exited or because of a SIGINT).

func (*Runtime) Subscribe

func (r *Runtime) Subscribe(f func(events <-chan ExecutorEvent) error)

Subscribe provides an events stream channel that is populated after taskrunner has loaded. The channel is read-only and is automatically closed on exit.

func (*Runtime) WithFlag

func (r *Runtime) WithFlag(f func(flags *flag.FlagSet))

WithFlag allows for external registration of flags.

type Task

type Task struct {
	Name string

	// A one or two sentence description of what the task does
	Description string

	// [DEPRECATED] This fn is deprecated in favor of RunWithFlags
	// Run is called when the task should be run. The function should gracefully
	// handle context cancelation.
	Run func(ctx context.Context, shellRun shell.ShellRun) error

	// RunWithFlags is called when the task should be run
	// and includes all flags passed to the task. The function
	// should gracefully handle context cancellation.
	RunWithFlags func(ctx context.Context, shellRun shell.ShellRun, flagArgs map[string]FlagArg) error

	// Flags is a list of supported flags for this task (e.g. --var="val", -bool).
	// Only the flags specified in this list will be considered valid arguments to a task.
	Flags []TaskFlag

	ShouldInvalidate func(event InvalidationEvent) bool

	// Dependencies specifies any tasks that this task depends on. When those
	// dependencies are invalidated, this task will also be invalidated.
	Dependencies []*Task

	// KeepAlive specifies whether or not this task is long lived.
	// If true, this task will be restarted when it exits, regardless of exit code.
	KeepAlive bool

	// Hidden specifies whether or not to show the task when -list is called.
	Hidden bool

	// Sources specifies globs that this task depends on. When those files change,
	// the task will be invalidated.
	Sources []string

	// Ignore is a set of globs subtracted from Sources.
	Ignore []string

	// IsGroup specifies whether this task is a pseudo-task that groups
	// other tasks underneath it.
	IsGroup bool
}

type TaskCompletedEvent

type TaskCompletedEvent struct {
	Duration time.Duration
	// contains filtered or unexported fields
}

func (*TaskCompletedEvent) Kind

func (TaskCompletedEvent) TaskHandler

func (e TaskCompletedEvent) TaskHandler() *TaskHandler

func (TaskCompletedEvent) Timestamp

func (e TaskCompletedEvent) Timestamp() time.Time

type TaskDiagnosticEvent

type TaskDiagnosticEvent struct {
	Error error
	// contains filtered or unexported fields
}

func (*TaskDiagnosticEvent) Kind

func (TaskDiagnosticEvent) TaskHandler

func (e TaskDiagnosticEvent) TaskHandler() *TaskHandler

func (TaskDiagnosticEvent) Timestamp

func (e TaskDiagnosticEvent) Timestamp() time.Time

type TaskFailedEvent

type TaskFailedEvent struct {
	Error error
	// contains filtered or unexported fields
}

func (*TaskFailedEvent) Kind

func (TaskFailedEvent) TaskHandler

func (e TaskFailedEvent) TaskHandler() *TaskHandler

func (TaskFailedEvent) Timestamp

func (e TaskFailedEvent) Timestamp() time.Time

type TaskFlag

type TaskFlag struct {
	// Description should be a string that describes what effect passing the flag to a task has.
	// This description is shown automatically in the generated --help message.
	// This field must be non-empty.
	Description string

	// ShortName is the single character version of the flag. If not 0, the
	// option flag can be 'activated' using -<ShortName>. Either ShortName
	// or LongName needs to be non-empty.
	ShortName rune

	// LongName is the multi-character version of the flag. If not "", the flag can be
	// activated using --<LongName>. Either ShortName or LongName needs to be non-empty.
	LongName string

	// Default is the default value for the flag.
	Default string

	// ValueType describes what the flag type is (e.g. --flag [ValueType])
	// and can be one of either: StringTypeFlag, BoolTypeFlag,
	// IntTypeFlag, Float64TypeFlag, or DurationTypeFlag.
	// Its value is shown automatically in the generated --help message.
	// This field must be non-empty.
	ValueType string
}

type TaskHandler

type TaskHandler struct {
	// contains filtered or unexported fields
}

func NewTaskHandler

func NewTaskHandler(execution *taskExecution) *TaskHandler

func (*TaskHandler) Definition

func (h *TaskHandler) Definition() *Task

func (*TaskHandler) Invalidate

func (h *TaskHandler) Invalidate(reason InvalidationEvent)

func (*TaskHandler) State

type TaskHandlerExecutionState

type TaskHandlerExecutionState string

type TaskInvalidatedEvent

type TaskInvalidatedEvent struct {
	Reasons []InvalidationEvent
	// contains filtered or unexported fields
}

func (*TaskInvalidatedEvent) Kind

func (TaskInvalidatedEvent) TaskHandler

func (e TaskInvalidatedEvent) TaskHandler() *TaskHandler

func (TaskInvalidatedEvent) Timestamp

func (e TaskInvalidatedEvent) Timestamp() time.Time

type TaskLogEvent

type TaskLogEvent struct {
	Message string
	Stream  TaskLogEventStream
	// contains filtered or unexported fields
}

func (*TaskLogEvent) Kind

func (e *TaskLogEvent) Kind() ExecutorEventKind

func (TaskLogEvent) TaskHandler

func (e TaskLogEvent) TaskHandler() *TaskHandler

func (TaskLogEvent) Timestamp

func (e TaskLogEvent) Timestamp() time.Time

type TaskLogEventStream

type TaskLogEventStream int
const (
	TaskLogEventStdout TaskLogEventStream = iota
	TaskLogEventStderr
)

type TaskOption

type TaskOption func(*Task) *Task

type TaskStartedEvent

type TaskStartedEvent struct {
	// contains filtered or unexported fields
}

func (*TaskStartedEvent) Kind

func (TaskStartedEvent) TaskHandler

func (e TaskStartedEvent) TaskHandler() *TaskHandler

func (TaskStartedEvent) Timestamp

func (e TaskStartedEvent) Timestamp() time.Time

type TaskStoppedEvent

type TaskStoppedEvent struct {
	// contains filtered or unexported fields
}

func (*TaskStoppedEvent) Kind

func (TaskStoppedEvent) TaskHandler

func (e TaskStoppedEvent) TaskHandler() *TaskHandler

func (TaskStoppedEvent) Timestamp

func (e TaskStoppedEvent) Timestamp() time.Time

type UserRestart

type UserRestart struct{}

func (UserRestart) Description

func (f UserRestart) Description() string

func (UserRestart) Reason

func (f UserRestart) Reason() InvalidationReason

type WatcherEnhancer

type WatcherEnhancer func(watcher.Watcher) watcher.Watcher

WatcherEnhancer is a function to modify or replace a watcher.

Directories

Path Synopsis
This package provides buildkite annotations of failed tasks per buildkite job.
This package provides buildkite annotations of failed tasks per buildkite job.
Package config provides configuration for taskrunner.
Package config provides configuration for taskrunner.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL