go-docker-utils

:toc:
:toc-title:
:toclevels: 2
:sectnums:

= Go Docker Utility Belt (GoDub)

The `Go Docker Utility Belt`, or short `GoDub`, is a tool to simplify and support the configuration of applications on Docker images.
One major feature is the templating support which allows it to render link:https://pkg.go.dev/text/template[Go templates] on the command line using environment variables.

It is directly inspired by the original link:https://github.com/confluentinc/confluent-docker-utils/blob/master/confluent/docker_utils/dub.py[Docker Utility Belt] written by Confluent. The original is a Python tool, which for example uses link:https://jinja.palletsprojects.com/en/latest/[Jinja2] for rendering templates.

In addition, it is also inspired by link:https://github.com/subfuzion/envtpl[envtpl] tool, which is written in Go. The major difference between _envtpl_ and _GoDub_ is, that _GoDub_ includes additional templating functions, for example to transform environment variable keys to a property format.

The main reason to develip _GoDob_ was, to provide templating support for configuration files which need to be part of a Docker image, without the need to install Python.

== Get it

[source, bash]
----
go get github.com/ueisele/go-docker-utils
----

== Usage

----
godub [command]

Available Commands:
  template    Uses Go template and environment variables to generate configuration files.
  ensure      Ensures that environment variables are defined.
  path        Checks a path on the filesystem for permissions.
----

`GoDub` provides the three base functions `template`, `ensure` and `path`.

=== Template

----
godub template [flags]

Flags:
  -i, --in strings       The template files (glob pattern). If not provided, it is read from stdin.
  -o, --out string       The output file or directory. If not provided, it is written to stdout.
  -r, --refs strings     Reference templates (glob pattern).
  -v, --values strings   Values files (glob pattern). Can be used with '.Values.' prefix.
  -f, --files strings    Available files inside templates (directories). It should be noted that all files are immediately loaded into memory. Can be used with '.Files.' prefix.
  -s, --strict           In strict mode, rendering is aborted on missing field.
----

* If `--in` is not provided, `GoDub` reads from `stdin`
* If `--out` is not provided, `GoDub` writes to `stdout`

==== Examples

.Renders a template from `stdin` ...
[source, bash]
----
export GREET="Hey :)"
echo "{{ .Env.GREET }}" | ./godub template
----

.\... and outputs `Hey :)` to `stdout`.
[source]
----
Hey :)
----

.Renders the template from file `examples/server.properties.gotpl` ... 
[source, bash]
----
export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
export KAFKA_NODE_ID=1
export KAFKA_LOG4J_LOGGERS=kafka.server.KafkaApis=TRACE
./godub template < examples/server.properties.gotpl > examples/server.properties
----

.\... and writes the output to file `examples/server.properties`
----
advertised.listeners=PLAINTEXT://127.0.0.1:9092
node.id=1
----

The template link:examples/server.properties.gotpl[] makes use of the `envToProp` function provided by `GoDub`, which transforms environment variable keys to properties. This feature was ported from Confluent`s link:https://github.com/confluentinc/confluent-docker-utils/blob/master/confluent/docker_utils/dub.py[Docker Utility Belt].

The reason why the environment variable `KAFKA_LOG4J_LOGGERS` is not added is that it is defined as an property to exclude (`$excluded_props`).

.Excerpt from link:examples/server.properties.gotpl[]
[source, go]
----
{{- range $key, $value := envToProp "KAFKA_" "" $excluded_props -}}
{{ $key }}={{ tpl $value $ }}
{{ end }}
----

.Can make usage of reference templates
[source, bash]
---
./godub template -i examples/define.gotpl -r examples/helpers.gotpl
---

.A reference templates can contain template definitions...
[source, go]
----
{{- define "T2" -}}
{{- range $key, $value := . -}}
{{ $key }}={{ $value }}
{{ end }}
{{- end -}}
----

.which can be used in templates
[source, go]
----
{{ template "T2" . }}
{{ template "T2" envToMap "KAFKA_" }}
----

.You can also load additional value files in formats `yaml`, `json`, `toml` and `properties`
[source, bash]
----
./godub template -i examples/deployment.yaml.gotpl --values examples/values.yaml
----

.examples/values.yaml
[source, yaml]
----
deployment:
  name: templated-name-from-yaml
----

.examples/deployment.yaml.gotpl
[source, yaml]
----
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.deployment.name }}
----

.It is possible to use files in templates
[source, bash]
----
./godub template -i examples/filesglob.gotpl -f examples/filestest
----

.The usage is basically identical to Helm: https://helm.sh/docs/chart_template_guide/accessing_files/
[source, go]
----
{{- range $path, $content := .Files.Glob "**/*.json" }}
file: {{ $path }}
{{ $content | fromJSON | toYAML }}
{{- end }}
----

=== Ensure

----
godub ensure [flags]

Flags:
  -a, --at-least-one   By the default it is ensured that all environment variables are defined. If this flag is set, it is enough if at least one is defined.
----

==== Examples

.Ensures that the environment variable `KAFKA_ADVERTISED_LISTENERS` is defined ...
[source, bash]
----
export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
./godub ensure KAFKA_ADVERTISED_LISTENERS
----


.\... and completes with exit status `0`
[source, bash]
----
echo $?
0
----

.For a environment variable like in this example `KAFKA_OPTS`, which is not defined ...
[source, bash]
----
export KAFKA_OPTS=
./godub ensure KAFKA_OPTS
----

.\... `GoDub` completes with exit status `1`
[source, bash]
----
Error: environment variables are missing: [KAFKA_OPTS]
echo $?
1
----

.Ensures that at least one of the environment variables `KAFKA_ADVERTISED_LISTENERS` and `KAFKA_LISTENERS` are defined ...
[source, bash]
----
export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
export KAFKA_LISTENERS=
./godub ensure --at-least-one KAFKA_ADVERTISED_LISTENERS KAFKA_LISTENERS
----

.\... and completes with exit status  `0`, because `KAFKA_ADVERTISED_LISTENERS` is defined
[source, bash]
----
echo $?
0
----

=== Path

----
godub path [flags]

Flags:
  -e, --existence          Path must be existence (default true)
  -r, --readable           Path must be readable
  -w, --writeable          Path must be writeable
  -x, --executable         Path must be executable
  -t, --timeout duration   Time to wait for the URL to be retrievable (default 0s)
----

==== Examples

.Could wait for a given amount of time for a specified file
[source,bash]
----
./godub path -t 5s /app/file-which-should-exist
----

== Template Functions

=== Sprig

The Sprig library is included and therefore all Sprig functions are supported (see https://masterminds.github.io/sprig/).

=== Custom

The following additional functions are suppored:

* Test Functions
** heygodub
* Env functions
** hasEnv
** fromEnv
** envToMap
** envToProp
* Map functions
** excludeKeys
** replaceKeyPrefix
** toPropertiesKey
* String functions
** kvCsvToMap
* List functions
** filterHasPrefix
* Verify functions
** required
* Network functions
** ipAddresses
** ipAddress
** anyIpAddress
* Format functions
** toYAML
** fromYAML
** toJSON
** toJSONPretty
** fromJSON
** toTOML
** fromTOML
** toProperties
** fromProperties

The functions are implemented in link:pkg/template/functions.go[].

**ToDo:** Detailed documentation of functions!

== Development

_GoDub_ provides a Visual Studio Code Remote Development in Containers set up. 
Just re-open this folder as Remote-Container, and you have a Goland development environment.

Instructions about how to configure Remote Development: https://code.visualstudio.com/docs/remote/containers-tutorial

=== Build

[source, bash]
----
./build.sh
----

.You can also cross-compile _GoDub_ with the following command:
[source, bash]
----
GOOS=linux GOARCH=arm64 ./build.sh
----

=== Run

[source, bash]
----
./godub
----

Hint: At the moment, the _godub_ binary is not a dynamic executable.
If, during some changes, the binary is no longer statically linked by default, it can be achieved by adding the following parameter to the _go build_ command.
[source, bash]
----
-ldflags "-s -w -linkmode external -extldflags -static"
----

=== Tests

[source, bash]
----
go test -v ./...
----

**ToDo:** Implement tests!

.functions_test.go
[source, go]
----
package template

import (
  "os"
  "testing"
  "github.com/stretchr/testify/assert"
)

func TestHasEnv(t *testing.T) {
  os.Setenv("GODUB_TEST_HAS_ENV_EXISTING", "value")
  assert.True(t, hasEnv("GODUB_TEST_HAS_ENV_EXISTING"))
  assert.False(t, hasEnv("GODUB_TEST_HAS_ENV_SOMETHING_ELSE"))
}
----

=== Initialize Project

Typically, this is not required, because go.mod is added to Git. However, to re-create it run_

[source, bash]
----
go mod init github.com/ueisele/go-docker-utils
go mod tidy
----

== Similar Tools

* https://github.com/confluentinc/confluent-docker-utils
* https://github.com/subfuzion/envtpl