taskcluster-worker: github.com/taskcluster/taskcluster-worker Files | Directories

Command taskcluster-worker

Package main hosts the main function for taskcluter-worker.

The code is structured in 4 kinds of extension registries. The commands, config, engines and plugins packages each define interfaces and registries where implementations of these interfaces can be registered.

To add a new command to taskcluster-worker you must create new sub-package of commands/ which implements and registers commands.CommandProvider with the commands.Register(name, provider) method. The same pattern is followed for implementation of config transformers, engines, and plugins.

All the sub-packages are then imported here, which ensure that they'll all be included in the respective extension registries. Exceptions to this pattern is the runtime and worker packages. The runtime package and its sub-packages implements generic abstractions and utilities to be used by all other packages. The worker package implements task execution flow to be used by commands.

Package Files

main.go

Directories

PathSynopsis
commandsPackage commands exposes a run method for main() to call
commands/daemon
commands/helpPackage help provides the help command.
commands/qemu-build
commands/qemu-guest-toolsPackage qemuguesttools implements the command that runs inside a QEMU VM.
commands/qemu-run
commands/schema
commands/shellPackage shell provides a CommandProvider that implements a CLI tool for opening to a interactive shell to an interactive taskcluster-worker task in your terminal.
commands/shell-server
commands/versionPackage version provides a CommandProvider that displays version number and git revision, these values are also exported through methods so that they can be read from other packages.
commands/work
configPackage config provides configuration loading logic.
config/absPackage configabs implements a TransformationProvider that replaces objects on the form: {$abs: "path"} with the value of current working folder + path.
config/configtestPackage configtest provides structs and logic for declarative configuration tests.
config/envPackage configenv implements a TransformationProvider that replaces objects on the form: {$env: "VAR"} with the value of the environment variable VAR.
config/hostcredentialsPackage hostcredentials implements a TransformationProvider that fetches credentials from the (oddly named) `host-secrets` service and replaces objects of the form: {$hostcredentials: [url, url]} with the credentials.
config/packetPackage configpacket implements a TransformationProvider that replaces objects on the form: {$packet: "VARIABLE"} with a value loaded from https://metadata.packet.net/metadata, following VARIABLE values are supported: - instance-id - hostname - facility - instance-type - public-ipv4 - public-ipv6
config/secretsPackage configsecrets implements a TransformationProvider that replaces objects on the form: {$secret: "NAME", key: "KEY"} with the value of the key "KEY" taken from the secret NAME loaded from taskcluster-secrets.
enginesPackage engines specifies the interfaces that each engine must implement.
engines/enginetestPackage enginetest provides utilities for testing generic engine implementations.
engines/mockPackage mockengine implements a MockEngine that doesn't really do anything, but allows us to test plugins without having to run a real engine.
engines/mock/mocknetPackage mocknet implements a net.Listener interface that can reached with mocknet.Dial() and establishes connections using net.Pipe()
engines/nativePackage nativeengine provides an engine with minimal sandboxing relying on per-task user accounts, temporary folders and process isolation.
engines/native/systemPackage system implements cross-platform abstractions for user-management access-control and sub-process execution geared at executing sub-process with best-effort sandboxing.
engines/qemuPackage qemuengine implements a QEMU based engine for taskcluster-worker.
engines/qemu/imagePackage image exposes methods and abstractions for extracting and managing virtual machine images.
engines/qemu/metaservicePackage metaservice implements the meta-data service that the guests use to talk to the host.
engines/qemu/networkPackage network contains scripts and abstractions for setting up TAP-device based networks for a set of QEMU virtual machines.
engines/qemu/network/openvpnPackage openvpn provides a wrapper around the openvpn client.
engines/qemu/vmPackage vm provides virtual machine abstractions using QEMU.
engines/scriptPackage scriptengine provides an engine that can be configured with a script and a JSON schema, such that the worker executes declarative tasks.
pluginsPackage plugins defines interfaces to be implemented by feature plugins.
plugins/artifactsPackage artifacts provides a taskcluster-worker plugin that uploads artifacts when sandbox execution has stopped.
plugins/envPackage env provides a taskcluster-worker plugin that injects environment variables into the task environments.
plugins/interactivePackage interactive implements the plugin that serves the interactive display and shell sessions over websockets.
plugins/interactive/displayclientPackage displayclient provides a golang implementation of websockify, transforming a websocket connection to an ioext.ReadWriteCloser object.
plugins/interactive/displayconsts
plugins/interactive/pty
plugins/interactive/shellclientPackage shellclient provides a wrapper for demuxing a shell websocket and exposing the stdout/stderr streams as well as offering a way to provide the stdin stream.
plugins/interactive/shellconstsPackage shellconsts contains constants shared between shell server and client which is split into different packages to reduce the binary size of potential commandline clients.
plugins/livelogPackage livelog provides a taskcluster-worker plugin that makes the task log available as a live log during task execution and finally uploads it as a static log.
plugins/logprefixPackage logprefix provides a taskcluster-worker plugin that prefixes all task logs with useful debug information such as taskId, workerType, as well as configurable constants.
plugins/maxruntimePackage maxruntime provides a plugin for taskcluster-worker which can enforce a maximum runtime upon tasks.
plugins/plugintest
plugins/rebootPackage reboot provides a taskcluster-worker plugin that stops the worker after certain number of tasks or given amount of time.
plugins/stoponerrorPackage stoponerror implements a very simple plugin that stops the worker gracefully if an non-fatal error is encountered.
plugins/successPackage success implements a very simple plugin that looks that the ResultSet.Success() value to determine if the process from the sandbox exited successfully.
plugins/tcproxyPackage tcproxy provides a taskcluster-worker plugin that exposes a proxy that signs requests with taskcluster credentials matching task.scopes.
plugins/watchdogPackage watchdog provides a taskcluster-worker plugin that pokes a watchdog whenever a task makes progress or the worker reports that it's idle.
runtimePackage runtime contains the generic functionality that an engine and plugins use.
runtime/atomicsPackage atomics provides types that can be concurrently accessed and modified, without caller code needing to implement locking.
runtime/client
runtime/fetcherPackage fetcher provides means for plugins and engines to fetch resources with generic references.
runtime/gcPackage gc contains the GarbageCollector which allows cacheable resources to register themselves for disposal when we run low on resources.
runtime/ioextPackage ioext contains interfaces and implementations for when the default io types are not sufficient.
runtime/mocksPackage mocks contains mock implementations of various interfaces useful for writing unit-tests.
runtime/monitoringPackage monitoring provides multiple implementations of runtime.Monitor.
runtime/utilPackage util contains a few simple utilites that has no internal dependencies.
runtime/webhookserverPackage webhookserver provides implementations of the WebHookServer interface.
test
test/nativetestPackage nativetest provides integration tests for a few common configuration of native and common plugins.
workerPackage worker is responsible for managing the claiming, executing, and resolution of tasks.
worker/taskrunPackage taskrun provides abstractions to execute a run of a task given a task, engine, plugin, and other runtime objects required by plugin and engine.
worker/workertestPackage workertest provides a framework for declarative definition of worker integration tests.
worker/workertest/fakequeuePackage fakequeue provides a fake implementation of taskcluster-queue in golang, The FakeQueue server stores tasks in-memory, it doesn't validate authentication, but implements most end-points correctly.

Package main imports 34 packages (graph). Updated 2017-11-04. Refresh now. Tools for package owners.