terrafirma

Terrafirma

Terrafirma is a composable alternative to Terraform. Like Terraform, Terrafirma deploys infrastructure. It abstracts away the cloud provider and provides a CLI that reliably creates and destroys servers.

Unlike Terraform, Terrafirma is designed to work alongside and with other tools. It is very much designed in the UNIX philosphy of doing one thing well.

• Composable with other tools, so it easily works with binp to handle binpacking and cup for zero-downtime, high-speed, parallel, rolling deploys. Mix-and-match to find your perfect deployment strategy.
• State is discovered, not stored, and thus is always in sync.
• Standard config language rather than tf. Works alongside everything else using json. jq to your heart's content.

Note that this is not a "batteries-included" tool. Terrafirma provides the bare minimum to interface with cloud providers and manage infrastructure.

Currently only Google Cloud is supported. Patches for additional features and cross-cloud support are welcome.

How it works

Terrafirma deploys infrastructure in 3 steps:

1. Plan: Terrafirma retrieves the current state from your cloud provider, compares that against your desired state, and generates a minimal diff of machines that need to be created or destroyed to reach your desired state.
2. Create: Create each of the machines listed in the plan.
3. Destroy: Delete each of the machines listed in the plan.

Unlike Terraform which executes a plan in a single command, Terrafirma separates the create and destroy steps.

This is because Terrafirma is designed to compose well with other tools and doesn't make any assumptions around how you provision servers, start your services, check their health, or notify your reverse proxy of new IPs.

In a high-availability production environment, all of that must happen after create but before destroy. A single step that both creates and destroys would require Terrafirma to handle that logic, which would greatly complicate the tool's code, configuration, and use.

Instead, ensure your new servers are provisioned, services are running, and your reverse proxy is directing traffic to the new instances prior to deleting old instances. Usually this can be accomplished with a simple bash script gluing everything together.

Usage example

Terrafirma requires a services.json file describing the services you want to deploy and the types of boxes you'll use. Here's an example:

{
	"services": {
		"openbsd": {
			"app-1": {
				"cpu": 2,
				"ram": "12 GB",
				"disk": "20 GB",
				"count": 5
			},
			"app-2": {
				"cpu": 3,
				"ram": "4 GB",
				"disk": "30 GB",
				"count": 3
			}
		},
		"debian": {
			"redis": {
				"cpu": 1,
				"ram": "2 GB",
				"disk": "10 GB",
				"count": 5
			}
		}
	},
	"boxes": {
		"openbsd": {
			"cpu": 5,
			"ram": "16 GB",
			"disk": "100 GB",
			"image": "projects/{your-project}/global/images/openbsd-66"
		},
		"debian": {
			"cpu": 1,
			"ram": "3 GB",
			"disk": "10 GB",
			"image": "projects/gce-uefi-images/global/images/family/debian-10"
		}
	},
	"providers": {
		"google": {
			"project": "{your-project}",
			"region": "us-central1",
			"zone": "us-central1-b"
		}
	}
}

Note that Terrafirma requires that all your boxes are homogenous.

Since using Terrafirma requires several steps, you'll probably use a short bash script to automate your particular workflow. An example script might look like this:

#!/usr/bin/env bash
set -efux

TOKEN=$(gcloud auth print-access-token)
export TOKEN

# Plan the number of servers to be created and destroyed. This creates a
# tf_plan.json file which the remaining steps use.
#
# binp is a binpacking tool that minimizes the number of servers for our box
# type.
terrafirma -b "$(binp)" plan > tf_plan.json

# Create any new servers, wait for them to boot, then use terrafirma to build
# an inventory file mapping the new IPs to each of the services running on
# them. Other tools can convert this inventory.json file to an SSH config file,
# etc.
#
# Terrafirma will automatically create and assign static IP addresses as
# needed.
if terrafirma create; then
	echo "waiting 60s for servers to boot..." && sleep 60
fi
terrafirma inventory > inventory.json

# Provision our servers using our usual deployment tools (cup, ansible, etc.).
up -t openbsd -c provision_openbsd
up -t debian  -c provision_debian

# Now we know our reverse proxy is sending traffic to these new boxes, so we
# can bring down our old ones and clean up any temporary files when we're done.
#
# Note that Terrafirma will NOT delete static IPs. You must manually delete
# them.
terrafirma destroy
rm tf_plan.json

If you wanted to wipe out your Terrafirma-managed infrastructure (except for any static IPs):

TOKEN=... terrafirma -b '{}' plan > tf_plan.json
TOKEN=... terrafirma destroy

Future work

Patches for the following would be greatly appreciated:

• Across a larger team, you'll likely need a locking mechanism. We need a locking design, ideally using the primitives we already use (VMs+tags). Any lock should expire with a timeout in the terrafirma client.
• Right now it can only deploy Compute Engine VMs (Google's EC2 equivalent). Ideally we'd be able to manage more than that through this tool, but we have no need for that yet.
• Other clouds would be great to support.