morbyd

package module
v0.10.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 30, 2024 License: Apache-2.0 Imports: 25 Imported by: 0

README

lxkns logo

morbyd

PkgGoDev License build and test goroutines Go Report Card Coverage

morbyd is a thin layer on top of the standard Docker Go client to easily build and run throw-away test Docker images and containers. And to run commands inside these containers. In particular, morbyd hides the gory details of how to stream the output, and optionally input, of container and commands via Dockers API. You just use your io.Writers and io.Readers, for instance, to reason about the expected output.

This module makes heavy use of option functions. So you can quickly get a grip on Docker's slightly excessive knobs-for-everything API design. morbyd neatly groups the many With...() options in packages, such as run for "run container" and exec for "container execute". This design avoids stuttering option names that would otherwise clash across different API operations for common configuration elements, such as names, labels, and options.

Features of morbyd

  • testable examples for common tasks to get you quickly up and running. Please see the package documentation.

  • option function design with extensive Go Doc comments that IDEs show upon option completion. No more pseudo option function "callbacks" that are none the better than passing the original Docker config type verbatim.

  • uses the official Docker Go client in order to benefit from its security fixes, functional upgrades, and all the other nice things to get directly from upstream.

  • “auto-cleaning” that runs when creating a new test session and again at its end, removing all containers and networks especially tagged using session.WithAutoCleaning for the test.

  • uses context.Context throughout the whole module, especially integrating well with testing frameworks (such as Ginkgo) that support automatic unit test context creation.

  • extensive unit tests with large coverage. We even mock the Docker client in order to cover the "unhappy paths", also known as "error handling". In addition, we run go routine leak checks, courtesy of Gomega gleak.

Trivia

The module name morbyd is an amalgation of "Moby (Dock)" and morbid – ephemeral – test containers.

Usage

package main

import (
    "context"

    "github.com/thediveo/morbyd"
    "github.com/thediveo/morbyd/exec"
    "github.com/thediveo/morbyd/run"
    "github.com/thediveo/morbyd/session"
)

func main() {
    ctx := context.TODO()
    // note: error handling left out for brevity
    //
    // note: enable auto-cleaning of left-over containers and
    // networks, both when creating the session as well as when
    // closing the session. Use a unique label either in form of
    // "key=" or "key=value".
    sess, _ := morbyd.NewSession(ctx, session.WithAutoCleaning("test.mytest="))
    defer sess.Close(ctx)

    cntr, _ := sess.Run(ctx, "busybox",
        run.WithCommand("/bin/sh", "-c", "while true; do sleep 1; done"),
        run.WithAutRemove(),
        run.WithCombinedOutput(os.Stdout))
    defer cntr.Stop(ctx)

    cmd, _ := cntr.Exec(ctx,
        exec.WithCommand("/bin/sh", "-c", "echo \"Hellorld!\""),
        exec.WithCombinedOutput(os.Stdout))
    exitcode, _ := cmd.Wait(ctx)
}

Alternatives

Why morbyd when there are already other much bigger and long-time battle-proven tools for using Docker images and containers in Go tests?

  • for years, @ory/dockertest has served me well. Yet I eventually hit its limitations hard: for instance, dockertest cannot handle Docker's 100 CONTINUE API protocol upgrades, because of its own proprietary Docker client implementation. However, this functionatly is essential in streaming container and command output and input – and thus only allowing diagnosing tests. Such issues are unresponded and unfixed. In addition, having basically to pass functions for configuration of Docker data structures is repeating the job of option functions at each and every dockertest call site.
  • Testcontainers for Go as a much larger solution with a steep learning curve as well as some automatically installing infrastructure – while I admire this design, it is difficult to understand what exactly is happening. Better keep it simple.

Supported Go Versions

morbyd supports versions of Go that are noted by the Go release policy, that is, major versions N and N-1 (where N is the current major version).

Contributing

Please see CONTRIBUTING.md.

morbyd is Copyright 2024 Harald Albrecht, and licensed under the Apache License, Version 2.0.

The package github.com/thediveo/morbyd/run/dockercli is Copyright 2013-2017 Docker, Inc. and licensed under the Apache License Version 2.0, with the elements listed below coming from the github.com/docker/cli module in order to work around import dependency versioning problems due to @docker/cli using a managed vendor/ directory, but not providing a go.mod and the associated guarantees:

Documentation

Overview

Package morbyd is a thin layer on top of the standard Docker Go client to easily build and run ephemeral test Docker images and containers, and run commands inside containers. It especially hides the gory details of how to stream the output, and optionally input, of containers and commands. Just io.Writer and io.Reader.

morbyd makes heavy use of option functions in order to help test writers to get a grip on Docker's (slightly) excessive knobs-for-everything API design. It neatly groups the many With...() options in packages, such as run for “run container” and exec for “container execute”. This design avoids stuttering option names that would otherwise clash across different API operations for common configuration elements, such as names, labels, and options.

Features of morbyd

At a glance:

  • testable examples for common tasks to get you quickly up and running.
  • option function design with extensive Go Doc comments that IDEs show upon option completion. No more pseudo option function “callbacks” that are none the better than passing the original Docker config type verbatim.
  • uses the official Docker Go client in order to benefit from its security fixes, functional upgrades, and all the other nice things to to get directly from upstream.
  • “auto-cleaning” that runs when creating a new test session and again at its end, removing all containers and networks especially tagged using session.WithAutoCleaning for the test.
  • uses context.Context throughout the whole module, especially integrating well with testing frameworks (such as Ginkgo) that support automatic unit test context creation.
  • extensive unit tests with large coverage.

Trivia

The module name “morby” is an amalgation of “Moby (Dock)” and “morbid” – ephemeral – test containers.

Index

Examples

Constants

View Source 
const AbbreviatedIDLength = 10

AbbreviatedIDLength defines the number of hex digits of a container ID to show in error and log messages.

View Source 
const DefaultSleep = 10 * time.Millisecond

Variables

This section is empty.

Functions

func Sleep 

func Sleep(ctx context.Context, d time.Duration) error

Sleep the specified duration, returning early in case the specified context gets cancelled. When cancelled, Sleep returns the context's error, otherwise when sleeping through the specified duration, it returns nil.

Types

type Container 

type Container struct {
	Name    string
	ID      string
	Session *Session
	Details types.ContainerJSON // inspection information after start.
}

Container represents a Docker container, providing notable operations specific to it:

  • Container.IP returns an host-internal IP address where the container can be reached.
  • Container.Exec to execute a command inside the container.
  • Container.PID to retrieve the PID of the container's initial process.
  • Container.Stop to stop the container by sending it the configured signal (defaults to SIGTERM).
  • Container.Kill to forcefully kill the container using SIGKILL.

func (*Container) AbbreviatedID 

func (c *Container) AbbreviatedID() string

AbbreviatedID returns an abbreviated container ID for use in error reporting in order to not report unwieldy long IDs.

func (*Container) Exec 

func (c *Container) Exec(ctx context.Context, cmd exec.Cmd, opts ...exec.Opt) (es *ExecSession, err error)

Exec a command inside a container, using the specified command using exec.Command(cmd, args...) and optional configuration information. It returns an *ExecSession object if successful, otherwise an error.

Important: executing the command is a fully asynchronous process to the extend that the session returned might still in its startup phase with the command not yet being executed. ExecSession.PID can be re-appropriated to wait for the command have been started.

Note: when using exec.WithInput make sure to close the input reader in order to not leak go routines handling the executed input and output streams in the background.

Note: morbyd does not support executing detached commands, so we will always be attached to the executing command's input/output streams.

Example

Execute a command inside a running container using Container.Exec.

In this example, we start by creating a session using NewSession. Because unit tests may crash and leave test containers and networks behind, we enable “auto-cleaning” using the session.WithAutoCleaning option, passing it a unique label. This label can be either a unique key (“KEY=”) or a unique key-value pair (“KEY=VALUE”); either form is allowed, depending on how you like to structure and label your test containers and networks. Auto-cleaning runs automatically directly after session creation (to remove any left-overs from a previous test run) and then again when calling Session.Close.

Next, we start a container using Session.Run where this container simply sits idle in a sleep loop (so that the idling shell process reacts more quickly to SIGTERMs).

Then, we run a new command inside this container and pick up its output. Finally, we wind everything down.

Note: [safe.Buffer] is a bytes.Buffer that is safe for concurrent use. 

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/thediveo/morbyd"
	"github.com/thediveo/morbyd/exec"
	"github.com/thediveo/morbyd/run"
	"github.com/thediveo/morbyd/safe"
	"github.com/thediveo/morbyd/session"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(),
		30*time.Second)
	defer cancel()

	sess, err := morbyd.NewSession(ctx,
		session.WithAutoCleaning("test.morbyd="))
	if err != nil {
		panic(err)
	}
	defer sess.Close(ctx)

	container, err := sess.Run(ctx,
		"busybox",
		run.WithCommand("/bin/sh", "-c", "trap 'exit 1' TERM; while true; do sleep 1; done"),
		run.WithAutoRemove())
	if err != nil {
		panic(err)
	}
	defer container.Kill(context.Background()) // just to be sure

	var out safe.Buffer
	exec, err := container.Exec(ctx,
		exec.Command("/bin/echo", "Hellorld! from exec"),
		exec.WithCombinedOutput(&out))
	if err != nil {
		panic(err)
	}
	exitcode, _ := exec.Wait(ctx)
	fmt.Printf("command exited with code %d\n", exitcode)
	container.Stop(ctx)
	fmt.Println(out.String())
}

Output:

command exited with code 0
Hellorld! from exec

func (*Container) IP 

func (c *Container) IP(ctx context.Context) net.IP

IP returns an IP address (net.IP) of this container that can be used to reach the container from the host. If no suitable IP address can be found, IP return nil. IP ignores addresses on a MACVLAN network, as IP addresses on a MACVLAN network cannot reached from the host.

NOTE: the container's IP address is usable without the need to (publicly) expose container ports on the host – which often is less than desirable in tests. However, with Docker Desktop the container IPs aren't directly reachable anymore as on plain Docker hosts, so in these cases you'll need to expose a container's exposable ports on (preferably) loopback.

func (*Container) Kill 

func (c *Container) Kill(ctx context.Context)

Kill the container forcefully and also remove its volumes.

func (*Container) PID 

func (c *Container) PID(ctx context.Context) (int, error)

PID of the initial container process, as seen by the container engine. In case the container is restarting, it waits for the next Doctor, erm, container incarnation to come online.

Note to Docker Desktop users: the PID is only valid in the context of the Docker engine that in case of macOS runs in its own VM, and in case of WSL2 in its own PID namespace in the same HyperV Linux VM.

func (*Container) Refresh 

func (c *Container) Refresh(ctx context.Context) error

Refresh the details about this container, or return an error in case refreshing fails.

func (*Container) Stop 

func (c *Container) Stop(ctx context.Context)

Stop the container by sending it a termination signal. Default is SIGTERM, unless changed using run.WithStopSignal.

func (*Container) Wait 

func (c *Container) Wait(ctx context.Context) error

Wait for the container to finish, that is, become “not-running” in Docker API parlance. See also: Docker's Client.ContainerWait.

type ExecSession 

type ExecSession struct {
	ID        string     // command execution ID.
	Container *Container // container this command runs inside.
	// contains filtered or unexported fields
}

ExecSession represents a command running inside a container.

Nota bene: the Docker API doesn't have an API endpoint for deleting executions when they're finished and we've picked up the results.

func (*ExecSession) Done 

func (e *ExecSession) Done() chan struct{}

Done returns a channel that gets closed when the command has finished executing inside its container.

func (*ExecSession) PID 

func (e *ExecSession) PID(ctx context.Context) (int, error)

PID returns the executing command's PID, or an error if the command has already terminated.

Note: [Container.Run] can already return while the underlying Docker session for executing the command inside the container is still starting up. In this case, PID will wait until the executing command's PID becomes available or the passed context gets cancelled.

func (*ExecSession) Wait 

func (e *ExecSession) Wait(ctx context.Context) (exitcode int, err error)

Wait for the command executed inside its container to finish, and then return the command's exit code. If the passed context gets cancelled or there is a problem picking up the command's exit code, Wait returns an error instead.

type Network 

type Network struct {
	Name    string
	ID      string
	Session *Session
	Details types.NetworkResource
}

func (*Network) Remove 

func (n *Network) Remove(ctx context.Context) error

type PullImageOpt 

type PullImageOpt func(*pullImageOptions)

func WithPullImageOutput 

func WithPullImageOutput(w io.Writer) PullImageOpt

WithImageBuildOutput set the writer to send the output of the image pull process to.

type Session 

type Session struct {
	// contains filtered or unexported fields
}

Session represents a Docker API client connection, together with additional configuration options that are inherited to newly created images, containers, and networks.

func NewSession 

func NewSession(ctx context.Context, opts ...session.Opt) (*Session, error)

NewSession creates a new Docker client and test session, returning a Session object on success, or an error otherwise.

When [sess.WithAutoCleaning] has been specified, then NewSession will then forcefully remove all containers and then networks matching the specified auto-cleaning label. In this case, Session.Close will then run a post-session cleaning.

Note: the Docker client is created using the options client.FromEnv and client.WithAPIVersionNegotiation.

func (*Session) AutoClean 

func (s *Session) AutoClean(ctx context.Context)

AutoClean forcefully removes all left-over containers and networks that are labelled with the auto-cleaning label specified when creating this session. If no auto-cleaning label was specified, AutoClean simply returns, doing nothing. (Well, it does something: it returns ... but that is now too meta).

func (*Session) BuildImage 

func (s *Session) BuildImage(ctx context.Context, buildctxpath string, opts ...build.Opt) (id string, err error)

BuildImage builds a container image using the specified build context and further build options. These build options are applied in the order they are provided, which allows modifying (or even nuking) the defaults when building an image.

BuildImage returns the ID of the built image, or an error in case of build errors.

Unless overridden using a build option, the following defaults apply:

  • Dockerfile: "Dockerfile"
  • Remove: true
  • ForceRemove: true

If no build process output writer has been specified using build.WithOutput any output (such as build steps, et cetera) will simply be discarded.

func (*Session) Client 

func (s *Session) Client() moby.Client

Client returns the Docker client used in this test session.

func (*Session) Close 

func (s *Session) Close(ctx context.Context)

Close removes left-over containers and networks if auto-cleaning has been enabled, and then closes idle HTTP connections to the Docker daemon.

func (*Session) Container 

func (s *Session) Container(ctx context.Context, nameID string) (*Container, error)

Container returns a *Container object for the specified name or ID if it exists, otherwise it returns an error. Please note that multiple calls for the same name or ID will return different *Container objects, as there is no caching.

func (*Session) CreateNetwork 

func (s *Session) CreateNetwork(ctx context.Context, name string, opts ...net.Opt) (*Network, error)

CreateNetwork creates a new “custom” Docker network using the specified configuration options.

Notable configuration options:

See also: docker network create

Example

Create a new “custom” Docker network, then run an example container attached to this (purely internal) network.

We start by creating a session using NewSession. Because unit tests may crash and leave test containers and networks behind, we enable “auto-cleaning” using the session.WithAutoCleaning option, passing it a unique label. This label can be either a unique key (“KEY=”) or a unique key-value pair (“KEY=VALUE”); either form is allowed, depending on how you like to structure and label your test containers and networks. Auto-cleaning runs automatically directly after session creation (to remove any left-overs from a previous test run) and then again when calling Session.Close.

Please note that for this testable example we need a deterministic container IP address assignment. In your tests, you most probably just need a working IP address, but not a particular one, so you won't need net.WithIPAM in most circumstance.

In our special case, we create a custom Docker network with an IP address management (IPAM) pool of “0.0.1.0/24” that is a small part of the so-called “this” network defined in RFC5735 section 3 and RFC8190.

The default “bridge” driver will automatically allocate the first available pool IP address “0.0.1.1” to the Linux kernel bridge, so the first container IP address will be “0.0.1.2”. Please note that the IP address “0.0.1.0” is the “subnet address” of this subnet and usually is reserved, so Docker's standard IPAM driver never assigns it.

When the example container attached to this custom network (using run.WithNetwork) starts, it executes the following shell command that grabs information about the specified network interface “eth0” and then cuts out only the IPv4 address: 

ip a sh dev eth0 | awk '/inet / { print $2 }'

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/thediveo/morbyd"
	"github.com/thediveo/morbyd/ipam"
	"github.com/thediveo/morbyd/net"
	"github.com/thediveo/morbyd/run"
	"github.com/thediveo/morbyd/safe"
	"github.com/thediveo/morbyd/session"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(),
		30*time.Second)
	defer cancel()

	sess, err := morbyd.NewSession(ctx,
		session.WithAutoCleaning("test.morbyd="))
	if err != nil {
		panic(err)
	}
	defer sess.Close(ctx)

	netw, err := sess.CreateNetwork(ctx, "my-notwork",
		net.WithInternal(),
		net.WithIPAM(ipam.WithPool("0.0.1.0/24")))
	if err != nil {
		panic(err)
	}
	defer func() {
		_ = netw.Remove(ctx)
	}()

	var out safe.Buffer
	container, err := sess.Run(ctx,
		"busybox",
		run.WithCommand("/bin/sh", "-c", "ip a sh dev eth0 | awk '/inet / { print $2 }'"),
		run.WithNetwork(netw.ID),
		run.WithAutoRemove(),
		run.WithCombinedOutput(&out))
	if err != nil {
		panic(err)
	}

	_ = container.Wait(ctx)
	fmt.Print(out.String())
}

Output:

0.0.1.2/24

func (*Session) HasImage 

func (s *Session) HasImage(ctx context.Context, imageref string) (bool, error)

HasImage returns true if the image referenced by imageref is locally available, otherwise false.

func (*Session) IsDockerDesktop added in v0.10.0 

func (s *Session) IsDockerDesktop(ctx context.Context) bool

IsDockerDesktop returns true if the Docker engine is the Docker Desktop engine, as opposed to a “plain” Docker engine. This differentiation is important in some situation, as containers managed by Docker Desktop cannot be directly reached from the host, but always require ports to be published.

func (*Session) Network 

func (s *Session) Network(ctx context.Context, nameID string) (*Network, error)

Network returns a *Network object for the specified name or ID if it exists, otherwise it returns an error. Please note that multiple calls for the same name or ID will return different *Network objects, as there is no caching.

func (*Session) PullImage 

func (s *Session) PullImage(ctx context.Context, imgref string, opts ...PullImageOpt) error

PullImage pulls a container image specified by the image reference, if not already locally available. The additional pull options are applied in the order they are provided.

If no pull process output writer has been specified using WithPullImageOutput any output (such as pull progress, et cetera) will simply be discarded.

Any pull process errors will be reported.

func (*Session) Run 

func (s *Session) Run(ctx context.Context, imageref string, opts ...run.Opt) (cntr *Container, err error)

Run (create and start) a new container, using the referenced image and optional configuration information, returning a *Container object if successful. Otherwise, it returns an error without leaving behind any container.

Additionally, Run attaches to the container's input and output streams which can be accessed using run.WithInput, and either run.WithCombinedOutput or run.WithDemuxedOutput.

If the session has configured with labels, the new container inherits them. Use run.ClearLabels before run.WithLabel or run.WithLabels in order to remove any inherited labels first.

Example

Run a container and gather its output.

We start by creating a session using NewSession. Because unit tests may crash and leave test containers and networks behind, we enable “auto-cleaning” using the session.WithAutoCleaning option, passing it a unique label. This label can be either a unique key (“KEY=”) or a unique key-value pair (“KEY=VALUE”); either form is allowed, depending on how you like to structure and label your test containers and networks. Auto-cleaning runs automatically directly after session creation (to remove any left-overs from a previous test run) and then again when calling Session.Close.

Next, Session.Run creates the container and then runs (in our example) a command that we supplied as part of the run configuration.

Because running the container and gathering its output are asynchronous operations, we Container.Wait for the container to have terminated before we pick up its output.

Note: [safe.Buffer] is a bytes.Buffer that is safe for concurrent use. 

package main

import (
	"context"
	"fmt"
	"time"

	"github.com/thediveo/morbyd"
	"github.com/thediveo/morbyd/run"
	"github.com/thediveo/morbyd/safe"
	"github.com/thediveo/morbyd/session"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(),
		30*time.Second)
	defer cancel()

	sess, err := morbyd.NewSession(ctx,
		session.WithAutoCleaning("test.morbyd="))
	if err != nil {
		panic(err)
	}
	defer sess.Close(ctx)

	var out safe.Buffer
	container, err := sess.Run(ctx,
		"busybox",
		run.WithCommand("/bin/sh", "-c", "echo \"Hellorld!\""),
		run.WithAutoRemove(),
		run.WithCombinedOutput(&out))
	if err != nil {
		panic(err)
	}

	_ = container.Wait(ctx)
	fmt.Print(out.String())
}

Output:

Hellorld!

Source Files

View all Source files

Directories

Path Synopsis
Package build provides configuring options for building images and handling build arguments.
 Package build provides configuring options for building images and handling build arguments.
Package exec provides configuration options for executing programs inside containers.
 Package exec provides configuration options for executing programs inside containers.
Package internal: keep your dirty paws off.
 Package internal: keep your dirty paws off.
ipamint
Package ipamint provides package “ipam”-internal APIs for sole use within the morbyd module, not littering the public API.
 Package ipamint provides package “ipam”-internal APIs for sole use within the morbyd module, not littering the public API.
netint
Package netint provides package “net”-internal APIs for sole use within the morbyd module, not littering the public API.
 Package netint provides package “net”-internal APIs for sole use within the morbyd module, not littering the public API.
Package ipam provides options for IP address management (“IPAM”), making Docker's IPAM-related API data structures more accessible.
 Package ipam provides options for IP address management (“IPAM”), making Docker's IPAM-related API data structures more accessible.
Package labels provides parsing of labels into a key-value map.
 Package labels provides parsing of labels into a key-value map.
net
Package net provides options to configure new Docker (custom) networks.
 Package net provides options to configure new Docker (custom) networks.
bridge
Package bridge provides Docker “bridge”-driver specific network configuration options.
 Package bridge provides Docker “bridge”-driver specific network configuration options.
ipvlan
Package ipvlan provides Docker “ipvlan”-driver specific network configuration options.
 Package ipvlan provides Docker “ipvlan”-driver specific network configuration options.
macvlan
Package macvlan provides Docker “macvlan”-driver specific network configuration options.
 Package macvlan provides Docker “macvlan”-driver specific network configuration options.
run
Package run provides configuration options for running containers.
 Package run provides configuration options for running containers.
dockercli
Source: https://github.com/docker/cli/blob/v25.0.1/opts/mount.go
 Source: https://github.com/docker/cli/blob/v25.0.1/opts/mount.go
Package safe provides a concurrency-safe buffer io.Writer.
 Package safe provides a concurrency-safe buffer io.Writer.
Package sess provides options for creating (test) sessions.
 Package sess provides options for creating (test) sessions.
Package strukt provides parsing strings with delimiter-separated fields into structs consisting of sufficient exported string fields.
 Package strukt provides parsing strings with delimiter-separated fields into structs consisting of sufficient exported string fields.
Package timestamper provides an io.Writer that adds timestamps at the beginning of each line of output.
 Package timestamper provides an io.Writer that adds timestamps at the beginning of each line of output.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.