tinystack

TinyStack - Lightweight (Kubernetes + OpenStack)

Lightweight container and virtual machine management platform. Easy to install, half the memory, all in a binary less than 100 MB.

Great for:

• Edge
• CI
• Development
• ARM

What is this?

TinyStack is a Kubernetes distribution with the following changes:

1. It is packaged as a single binary.
2. It adds support for sqlite3 as the default storage backend. Etcd3, MySQL, and Postgres are also supported.
3. It wraps Kubernetes and KubeVirt in a single, simple launcher.
4. It has minimal to no OS dependencies (just a sane kernel and cgroup mounts needed).

TinyStack bundles the following technologies together into a single cohesive distribution:

• Containerd & runc
• Flannel for CNI
• CoreDNS
• Metrics Server
• Traefik for ingress
• Klipper-lb as an embedded service load balancer provider
• Kube-router netpol controller for network policy
• Helm-controller to allow for CRD-driven deployment of helm manifests
• Kine as a datastore shim that allows etcd to be replaced with other databases
• Local-path-provisioner for provisioning volumes using local storage
• Host utilities such as iptables/nftables, ebtables, ethtool, & socat
• KubeVirt to define and manage virtual machines

These technologies can be disabled or swapped out for technologies of your choice.

Additionally, TinyStack simplifies Kubernetes operations by maintaining functionality for:

• Managing the TLS certificates of Kubernetes components
• Managing the connection between worker and server nodes
• Auto-deploying Kubernetes resources from local manifests in realtime as they are changed
• Managing an embedded etcd cluster (work in progress)
• Integrated kubevirt API, controllers and virtctl command line

What's with the name?

We want a minimum installation for virtual machine management, better with container capabilities.

We wanted an installation of Kubernetes that was half the size in terms of memory footprint. Kubernetes is a 10 letter word stylized as k8s. So something half as big as Kubernetes would be a 5 letter word stylized as TinyStack. There is neither a long-form of TinyStack nor official pronunciation.

Is this a fork?

No, it's a distribution. A fork implies continued divergence from the original. This is not TinyStack's goal or practice. TinyStack explicitly intends not to change any core Kubernetes functionality. We seek to remain as close to upstream Kubernetes as possible. However, we maintain a small set of patches (well under 1000 lines) important to TinyStack's use case and deployment model. We maintain patches for other components as well. When possible, we contribute these changes back to the upstream projects, for example, with SELinux support in containerd. This is a common practice amongst software distributions.

TinyStack is a distribution because it packages additional components and services necessary for a fully functional cluster that go beyond vanilla Kubernetes. These are opinionated choices on technologies for components like ingress, storage class, network policy, service load balancer, and even container runtime. These choices and technologies are touched on in more detail in the What is this? section.

How is this lightweight or smaller than upstream Kubernetes?

There are two major ways that TinyStack is lighter weight than upstream Kubernetes:

1. The memory footprint to run it is smaller
2. The binary, which contains all the non-containerized components needed to run a cluster, is smaller

The memory footprint is reduced primarily by running many components inside of a single process. This eliminates significant overhead that would otherwise be duplicated for each component.

The binary is smaller by removing third-party storage drivers and cloud providers, explained in more detail below.

What have you removed from upstream Kubernetes?

This is a common point of confusion because it has changed over time. Early versions of TinyStack had much more removed than the current version. TinyStack currently removes two things:

1. In-tree storage drivers
2. In-tree cloud provider

Both of these have out-of-tree alternatives in the form of CSI and CCM, which work in TinyStack and which upstream is moving towards.

We remove these to achieve a smaller binary size. They can be removed while remaining conformant because neither affects core Kubernetes functionality. They are also dependent on third-party cloud or data center technologies/services, which may not be available in many TinyStack' use cases.

What's next?

Check out our roadmap to see what we have planned moving forward.

Documentation

Please see the official docs site for complete documentation.

Quick-Start - Install Script

The install.sh script provides a convenient way to download TinyStack and add a service to systemd or openrc.

To install TinyStack as a service, run:

./install.sh

A kubeconfig file is written to /etc/tinystack/stack.yaml and the service is automatically started or restarted. The install script will install TinyStack and additional utilities, such as kubectl, crictl, tiny-killall.sh, and tiny-uninstall.sh, for example:

sudo kubectl get nodes

Manual Download

1. Download tstac from latest release, x86_64 and arm64 are supported.
2. Run the server.

sudo tstac server &
# Kubeconfig is written to /etc/tinystack/stack.yaml
sudo tstac kubectl get nodes

# On a different node run the below. NODE_TOKEN comes from
# /var/lib/tstac/server/node-token on your server
sudo tstac agent --server https://myserver:6443 --token ${NODE_TOKEN}

Contributing

Please check out our contributing guide if you're interested in contributing to TinyStack.