hcbuild
The purpose of hcbuild
is to help you build multi-platform container images faster in an affordable manner. To achieve this, it leverages Hetzner Cloud's powerful computing resources, including both AMD and ARM virtual machines, as well as SSD NVMe volumes to persist the state of your builds.
Table of Contents
Why should I find this useful?
If you are happy with the time it takes for you to build multi-platform container images, then this tool is not for you.
If you are building container images for multiple platforms (e.g. Linux amd64, arm64, arm/v6, arm/v7, etc) and:
- Your local machine doesn't have enough resources to build multi-platform container images.
- Your multi-platform builds are slow because they're using emulation.
- You don't want or can't afford to spend large amounts of money on cloud VMs from other big cloud providers such as AWS, GCP or Azure.
Then, hcbuild
is for you.
What makes it interesting?
- Governance : you have full control over the cloud resources created: you choose what VMs to use to build your images and the size of the volumes to persist the BuildKit cache.
- Fast provisioning: to spin up all the cloud resources (including VMs, IPs, SSH key and volumes) takes literally less than 2 minutes.
- Caching: Leverage layer caching with persistent volumes in the cloud.
Overall, less time waiting for a multi-platform docker builds to finish.
Demo
Requirements
Note
If you don't have a Hetzner Cloud account, you can use my referral link to receive 20€ in cloud credits.
Download hcbuild
hcbuild
is distributed as a static Go binary. You can visit the Releases page to download the executable for your platform.
Usage
Describing the cloud builder
A BuildKit builder instance is composed of multiple nodes targeting different platforms. The cloud-builder.yaml
file describes the number of nodes (i.e. Hetzner's Cloud VMs) that will be used to build your container images.
In the example below, we are creating 2 nodes to build container images for Linux amd64 and arm.
builder:
name: cloud-builder
sshKey:
name: "personal"
path: "~/.ssh/personal.pub"
nodes:
- name: "amd64"
location: "nbg1"
image: "docker-ce"
type: "cx21" # 2 vCPUs, 4 GB
platforms:
- "linux/amd64"
volume:
name: buildkitd-state-amd64
path: /buildkitd-state
sizeGB: 50 # 10 is the minimum value
- name: "arm64"
location: "fsn1" # ARM VMs are only available in Falkenstein atm
image: "docker-ce"
type: cax11" # 2 vCPUs, 4 GB
platforms:
- "linux/arm64"
- "linux/arm/v6"
- "linux/arm/v7"
volume:
name: buildkitd-state-arm64
path: /buildkitd-state
sizeGB: 50 # 10 is the minimum value
Note
ARM cloud VMs are only available in Falkenstein (Germany) at the moment of writing this.
In this examples, the nodes use the same image (docker-ce
), but you can use different images for each node.
Each node runs a buildkitd
instance as a docker container, which is responsible for building the container images.
The volume
section is optional, but it's recommended to use it to persist the buildkitd state between builds.
Provisioning the infrastructure
export HCLOUD_TOKEN=*****
hcbuild up --config cloud-builder.yaml
This command will provision the resources described in the cloud-builder.yaml
file and will start the buildkitd
daemon in each node. Afterwards, it will create a BuildKit builder instance in your local machine and will connect to the remote buildkitd
daemons via SSH.
At this point, you can use the builder to build container images using the buildx
command:
After the infrastructure has been successfully provisioned and the builder has been created, you can use it to build multi-platform container images.
docker buildx build \
--builder cloud-builder \
--platform linux/amd64,linux/arm64 \
-t myimage:latest .
The first time you run this command, it will take a while to build the images as it needs to download the base images and the buildkitd cache is empty. However, the next time you run it, it will be much faster as the cache will be reused.
Destroying the infrastructure
To save money, whenever you're done building images for the day, you can use hcbuild down --nodes
to remove the VMs but keep the volumes. This way, the next time you run hcbuild up
, it will reuse the volumes and the buildkitd
state will be persisted between builds.
Notice that some cloud resources such as the VMs and volumes are billed hourly, so you can save some money by removing them when you're not using them. This means that even if you remove the VMs just right after they are created, you will still be billed for 1 hour.
Note
Stopping the VMs will not save you any money as Hetzner Cloud still charges you for stopped VMs.
Therefore, I'd recommend to use hcbuild down
to remove the VMs and keep the volumes, and only use hcbuild down --all
to remove the volumes when you're not going to use the builder for a while.
Note
The volumes are not deleted by default to avoid losing the buildkitd
state.
How does it work?
hcbuild
is a Go CLI that quickly provisions Hetzner Cloud VMs and runs a BuildKit daemon in each one of them. From your local machine, you can use hcbuild
to configure and remotely connect via SSH to a BuildKit builder composed of nodes of different platforms using the cloud-builder.yaml file.
Alongside the VMs where the buildkitd
daemon is running, hcbuild
also provisions a volume to persist the buildkitd
state between builds. This is extremely convenient as you can remove the VMs after you're done with your builds, and the next time you run hcbuild
it will re-create them and mount the volumes automatically for you, thus reusing the BuildKit cache from previous builds.
What resources are created in Hetzner Cloud?
hcbuild
creates the following resources in Hetzner Cloud:
- 1 VM per node to run the
buildkitd
daemon.
- 1 SSH key to connect to the
buildkitd
daemon.
- 1 Primary IP address per node to connect to the
buildkitd
daemon.
- 1 Volume per node to persist the
buildkitd
state between builds.
For instance, in the example described above, 2 volumes, 2 servers, 1 SSH key and 2 IP addresses will be created.
Monthly cost breakdown
The cost of running the builder depends on the number of nodes, the type of VMs you choose, and the size of the volumes. Using the example described above, the monthly cost of running the builder, at the time of writing this would be:
Fixed costs
Resource |
Quantity |
Price |
Total |
Primary IP |
2 |
0.744 €/month |
1.488 €/month |
Volume 50GB SSD NVMe |
2 |
0.05324 €/GB/month |
2.662 €/month |
Total |
- |
- |
4.15 €/month |
Variable costs
Resource |
Quantity |
Price |
Total |
VM CX21: 2 vCPUs Intel Xeon Gold, 4GB |
1 |
5.87 €/month |
5.87 €/month |
VM CAX11: 2 vCPUs Ampere Altra, 4GB |
1 |
3.98 €/month |
3.98 €/month |
Total |
- |
- |
9.85€/month |
Total costs
Total (fixed + variable): 14 €/month.
Note
This assumes that the VMs are running 24/7. If you only run the VMs for a few hours a day, the cost will be significantly lower.