azure-key-vault-agent

Usage

akva --config=akva.yaml

Authentication

You can specify named credentials in the config file under the top-level key credentials.

Additionally, you can specify the following environment variables (or specify these key=value pairs in a file called .env):

AZURE_TENANT_ID=<tenant id>
AZURE_CLIENT_ID=<SPN name including http>
AZURE_CLIENT_SECRET=<SPN password>

They will be loaded as the credential name default.

Config

Create a yaml file which holds configuration for one or more credentials and one or more workers.

Each worker can pull one or more resources and use those resources to write to one or more file sinks.

A simple example is given below:

credentials:
  -
    name: default
    tenantID: test-id
    clientID: https://test-client-id
    clientSecret: test-secret

workers:
  -
    resources:
      - kind: secret
        name: password
        vaultBaseURL: https://test-kv.vault.azure.net/
        # No credential specified, so "default" will be used
    sinks:
      - path: ./password
        template: "{{ .Secrets.password.Value }}"

Credentials

The credentials section is a list of one or more named credentials used for fetching resources. Each credential has a tenantID, clientID, clientSecret.

The ENV vars (or .env file) will be injected as a credential with the name default if you don't override default within your config file.

Resources

The resources section is a list of one or more resources to fetch. Each resource has a kind, vaultBaseURL, and optional credential field.

Valid kinds are: cert, secret, all-secrets, and key.

Note: The all-secrets fetches all of the secrets found in the vault, and cannot be used in conjunction with any specific secrets for the same vaultBaseURL

Unless a resource has a kind of all-secrets, there is also a required name field for the resource.

If you don't specify credential, a credential with the name default will be used (you can either specify the default credential in the credentials array, or as ENV vars / .env file)

Aliases

A resource with a kind set to cert, secret, or key may specify an alias. This alias may be used to reference the resource in your specified sink:

workers:
  -
    resources:
      - kind: secret
        name: my-application-password
        alias: pass
        vaultBaseURL: https://test-kv.vault.azure.net/
    sinks:
      - path: ./password
        template: "{{ .Secrets.pass.Value }}"

Sinks

The sinks section is a list of one or more files to write to. Each sink has a path and either template (inline template) or templatePath (path to template on the filesystem). The template syntax is golang's text/template library (with sprig helpers).

sinks also support configuring file ownership and permission bits via the owner, group, and mode settings.

• owner and group are the names of the respective entity and must both be present. If omitted the executing user and group will be applied.
• mode accepts file modes in either 3 or 4 digit notation 777, 1644, 0600 are all valid examples. If omitted a default of 0644 will be used.

Each template has access to all of the resources specified in the resources section above, separated by kind and resource name. The fields available to you for any given resource can be found by looking at the corresponding source structs:

• Secret
• Cert
• Key

For example, if you wanted to read the Value attribute of a Secret whose name was test, the template for that would be: {{ .Secrets.test.Value }}

Other fields

Other worker-level fields that you can specify are:

• frequency: How often the worker should poll its resources and see if there are any changes. Defaults to 60s
• preChange: If the newly rendered sink contents differ from the file contents already on disk, the command specified here will be executed before the file is written
• postChange: If the newly rendered sink contents differ from the file contents already on disk, the command specified here will be executed after the file is written

Examples

SSL Cert + Private key

When you create a Cert in azure key vault, it automatically creates a Secret and Key with the same name. In the associated Secret, the value will be a blob that contains both the private key and cert.

To fetch the private key, you'll need to ensure that the Secret is in your resources section. You will also need to use the built-in privateKey and cert helpers to parse the blob into its respective pieces.

Note: cert helper will only return the leaf certificate

In the example below, it is assumed you have created a PEM format certificate with the name pem-test:

workers:
  -
    resources:
      - kind: secret
        name: pem-test
        vaultBaseURL: https://test-kv.vault.azure.net/
    frequency: 60s
    postChange: service nginx restart
    sinks:
      - path: ./pem-test.key
        template: '{{ index .Secrets "pem-test" | privateKey }}'
        owner: myuser
        group: mygroup
        mode: 0600
      - path: ./pem-test.cert
        template: '{{ index .Secrets "pem-test" | cert }}'

Complete List of Cert Helpers:

cert - returns PEM formatted leaf certificate.

privateKey - returns PEM formatted private key.

issuers - returns sorted issuers in PEM format.

fullChain - returns full certificate chain including leaf cert in PEM format.

expandFullChain - returns a map of secrets, including separate PEM and keys.

Note:

• The resource type cert does not contain any chain information due to the way Azure stores the data. If you wish to use issuers or fullChain helpers, you must do so on a secret resource.
• The issuers and fullChain helpers will do their best to reconstruct the chain, but can only work with the data given. So if you did not store your certificate with its chain an empty string will be returned.

Multiple secrets in a file

Let's suppose you had 4 secrets in a given key vault, dbHost, dbName, dbUser, dbPass.

Here's some sample configs:

Using individual secret lookups

workers:
  -
    resources:
      - kind: secret
        name: dbHost
        vaultBaseURL: https://test-kv.vault.azure.net/
      - kind: secret
        name: dbName
        vaultBaseURL: https://test-kv.vault.azure.net/
      - kind: secret
        name: dbUser
        vaultBaseURL: https://test-kv.vault.azure.net/
      - kind: secret
        name: dbPass
        vaultBaseURL: https://test-kv.vault.azure.net/
    frequency: 60s
    postChange: docker restart webapp
    sinks:
      - path: ./config.yml
        template: "databaseUrl: psql://{{ .Secrets.dbUser.Value }}:{{ .Secrets.dbPass.Value }}@{{ .Secrets.dbHost.Value }}/{{ .Secrets.dbName.Value }}"

Using all-secrets

workers:
  -
    resources:
      - kind: all-secrets
        vaultBaseURL: https://test-kv.vault.azure.net/
    frequency: 60s
    postChange: docker restart webapp
    sinks:
      - path: ./config.yml
        template: "databaseUrl: psql://{{ .Secrets.dbUser.Value }}:{{ .Secrets.dbPass.Value }}@{{ .Secrets.dbHost.Value }}/{{ .Secrets.dbName.Value }}"

You can also use the built-in toValues helper to get key/value pairs of all of your secrets.

workers:
  -
    resources:
      - kind: all-secrets
        vaultBaseURL: https://test-kv.vault.azure.net/
    frequency: 60s
    postChange: docker restart webapp
    sinks:
      - path: ./config.json
        template: "{{ index .Secrets | toValues | toJson }"

will output the following in the config.json file

{ "dbHost": "my-host", "dbName": "my-db", "dbUser": "my-user", "dbPass": "my-pass" }

Using the built-in expandFullChain helper will separate the PEM and key from certificates if present in your secrets, and return the pem and key as separate secrets along with any original secrets from a given keyvault.

workers:
  -
    resources:
      - kind: all-secrets
        vaultBaseURL: https://test-kv.vault.azure.net/
    frequency: 60s
    postChange: docker restart webapp
    sinks:
      - path: ./config.json
        template: "{{ index .Secrets | expandFullChain | toValues | toJson }"

will output the following in the config.json file

{"test-cert":"...","test-cert.key":"...","test-cert.pem":"...","some-string-secret":"...","different-cert":"...","different-cert.key":"...","different-cert.pem":"..."}

Resources with special characters in their name

Go's text/template syntax cannot handle reading fields with special characters (including hyphens) in the name directly. If you have a resource with a hyphen (or other funky character) you will have to use the built-in index function for fetching the appropriate value:

workers:
  -
    resources:
      - kind: secret
        name: my-test
        vaultBaseURL: https://test-kv.vault.azure.net/
    frequency: 60s
    postChange: service nginx restart
    sinks:
      - path: ./my-test
        template: '{{ index .Secrets "pem-test.Value" }}'

Different credentials for different resources

Using two sets of credentials, one named default and one named shared, fetch multiple resources. Don't specify any credential for the resources using default.

credentials:
  -
    name: default
    tenantID: my-tenant-id
    clientID: http://cjohnson-test-spn
    clientSecret: cjohnson-test-secret
  -
    name: shared
    tenantID: my-tenant-id
    clientID: http://shared-test-spn
    clientSecret: shared-test-secret

workers:
  -
    resources:
      - kind: secret
        name: thing1
        vaultBaseURL: https://test-kv.vault.azure.net/
        # No credential specified, so "default" will be used
      - kind: secret
        name: thing2
        vaultBaseURL: https://test-kv.vault.azure.net/
        # No credential specified, so "default" will be used
      - kind: secret
        name: thing3
        vaultBaseURL: https://test-kv.vault.azure.net/
        credential: shared # Refers to credentials.name == "shared" above
    frequency: 60s
    sinks:
      - path: ./secret.txt
        template: "{{ .Secrets.thing1.Value }}{{ .Secrets.thing2.Value }}{{ .Secrets.thing3.Value }}"

Workers

Workers default to working in a loop, whose frequency is controlled by the frequency field in your config. Each iteration of the loop, the worker performs the following:

• Fetch all of the specified resources
• If any errors occur, fail the iteration and:
  • For high-frequency workers (<60s) just wait for the next iteration and try again
  • For low-frequency workers (>60s), enter a retry/backoff cycle, with jitter to avoid the thundering herd problem
• If no errors occurred, then for each sink specified:
  • Load and/or parse the specified template, and render it using the fetched resources
  • Compare the results of the template to the contents of the destination path
  • If the contents differ, trigger any preChange hook, write the contents to the path, and trigger any postChange hook

If you want to run your workers once and then exit, pass the --once option to the executable.

Config watcher

A filesystem watch is placed on the specified config file, and if the file is changed, the config will be re-parsed and all of the workers will be killed and recreated based on the new config

Known Issues

• Using a 4 digit mode on MacOS will only support sticky (i.e. 1644). setuid and setgid do not work.

Development

Building locally

• Run go mod download to download dependencies in the module cache
• Add any test configuration to a local akva.yaml file
• Run go build . && ./azure-key-vault-agent -c ./akva.yaml to build and run

Troubleshooting builds

• If you run into any issues when running go build ., you may need to update package dependencies
• You can update a single package with go get -u <package name>

Releasing a new version

1. Update the CHANGELOG accordingly
2. Merge the PR
3. Determine the most recent deployment tag version: git checkout master && git fetch && git tag --sort=-creatordate | head -n1 - the new version tag should be above this using semVer
4. Tag and push the new release; example:

git tag -a v1.7.0 -m "version 1.7.0"
git push origin v1.7.0