rhyzome-api

Rhyzome Resource Server

tracks and manages the lifecyle of resources, and provides a REST API for users to access them

All state will be stored in postgres. Authentication happens with oauth2 via keycloak. This description is preliminary and should be carefully evaluated.

This server provides a user access to arbitrary "resources" (defined below) via a REST API. Permission scopes control specifically which resources a particular user may create, update or delete. Unless prohibited by permissions, users should be able to preform CRUD operations on resources. Eventually, a web-based user interface for this REST API will be required, although that it beyond the scope of this project. Currently management of all resource types is handled by a central daemon compiled into a single binary. Eventually it may be desierable to break these up into multiple components, either as loadable plugins or by splitting each serviece into it's own binary that can be operated independantly. There would likely be some core set of resources that must be available. In the plugin model, this would mean the core resources are built into the main binary. In the multi-service model, a central resource management service would handle these and other core APIs that other services would call back to.

Install

The HTTP API and the GRPC server are seperated into two separate packages.

You can build the binaries using Go as follows:

go build -o rhyzome-api cmd/rhyzome-api/main.go
go build -o rhyzome-grpc cmd/rhyzome-grpc/main.go

Setup

Configuration

The programs will look for a configuration file named rhyzome-api.json in the directory where it is run, or in /etc/rhyzome/.

The config requires an db and a pki options defined, as seen in the following example. These options are explained in detail in the next sections.

{
    "db": "dbname=rhyzome host=/var/run/postgresql sslmode=disable",
    "pki": {
        "ca": "/home/user/.step/certs/root_ca.crt",
        "cert": "/etc/step/certificates/step.crt",
        "key": "/etc/step/certificates/step.key"
    }
}

DB

You need to have Postgresql installed, and to create a Postgresql user.

While the project is in early development the database schemas are subject to change in ways that require DB resets.

The db-reset.sh script can be used for initializing the database, as well as reseting. THIS WILL DESTROY YOUR DATABASE

PKI

Rhyzome uses Mutual TLS (mTLS) authentication between the components. You can read about how to setup a PKI and issue certificates using Smallstep in our PKI guide.

Usage

Once the configuration is in place, we can start the GRPC server by simply running:

./rhyzome-grpc

Along the same lines, you may start the HTTP API like so:

./rhyzome-api

Rhyzome is now ready for incoming connections from other components, as well as API calls.

Concepts

Resource

Resource is an interface for things that users can create, list, update and delete. This is a common interface for things like Instances (VMs), disk images, object storage (S3) buckets, networks, etc. Each resource implements the interface, and has an associated ResourceCore object, which can be used for common fields such as unique id, unique name and access permissions. Required functions:

• Validate(boolean) []error - Called on the new object before creating or updating. The boolean argument is true when a new resource is being created. an empty list means the object is valid.
• Create(ResourceCore, *jobs.Job) error - Called after validation when a create is requested
• Update(*jobs.Job) error - Called after validation when an update is requested
• Remove(*jobs.Job) error - Called when a delete is requested
• GetCore() ResourceCore - returns the ResourceCore
• Load(ResourceCore) Resource - fetch an instance from the database by ID. This might take an sqlc row struct as an argument as well.

note: it seems like this might not end up being an interface and resources just typically have these operations

the ResourceCore struct:

type ResourceCore struct {
	ID   string
    Tags map[string]string
}

Some lists of unfinished ideas, items in no particular order:

other things to consider tracking:

• Dependant resources. ie, maybe we have disks that can be attached to instances, making the instance dependant on them.
• Price per billing interval. We haven't picked a billing interval, it should probably 1 second. Some resources might use consumption based pricing as well/in addition to the cost to run it, so there may be other billing-related fields here.
• There should be a mechanism for resources to report custom metrics that are read-only to the user

Things that might become resources as this gets implemented:

• Virtual Machines
• Disks
• Users on a particular tenant
• Networks
• WireGuard servers and peers
• Port forwards/firewall rules in general
• Reserved public IP addresses
• Internal subnets
• TLS-terminating HTTP proxies
• S3-compatible buckets
• x509 and SSH PKIs/CAs
• base images and snapshots - we should have some way to make base images globally discoverable
• VoIP numbers
• kubernetes clusters
• Postgres Databases
• Redis Instances
• static http hosting (likely tied into s3 service)
• matrix home servers
• XMPP server or domain on shared server
• GitLab runners especially with weird configurations
• owncast instances
• DNS zones and entries
• prometheus monitoring servers, targets and paging schedules (or maybe there's a good open source PagerDuty?)
• email domains, inboxes, aliases, SMTP/POP3/IMAP/JMAP access
• web-based development environment (Theia)
• literally anything else this entire thing is just an eleborate billing system

Other notes:

• There should be a way to create new resource types easily. For example a command that templates all required files, including SQL updates.
• There is no current database migration mechanism. That needs to happen before this is actually deployed for use. sqlc documentation suggests several db migration options
• each tenant should be assigned a domain in keycloak. We may want to consider extending this to having a per-tenant domain for the API.
• We need a short-term way to preform administrative actions. likely via a cli that connects to the database directly. Actions include tenant create/update/delete

Resource: Instance

A virtual machine.

• Must be assigned to a Network
• Assigned an IP address on that network. Generated if not specified, must not be in use already.

Resource: Network

A virtual network

• Configurable properties subnet and mask
• List of static DHCP leases, including a flag indicating if it was automatically assigned.
  • Automatically assigned leases are deleted associated Instance is deleted. Does not need to be right away

Resource: Base Image

this name should change, maybe instance image? what does aws call them?

Currently these are global images that can be used by anyone. Eventually there should be a way to make private base images.

Resource: Block Storage Disk

A disk attached to an Instance

Resource: Disk Snapshot

A snapshot a fixed state from a Block Storage Disk at a particular time.

Resource: Disk Base Image

TODO: figure out a better name

A published snapshot from a disk, intended for publication to and access by any users and even other tenants

Tenant

A tenant is one logical customer organization. Tenants are identified by a unique lower case alphanumeric strings, and may have other properties associated with them, for example some piece of information linking a billing system.

Developer setup

sqlc

To add or update sql queries, edit sql/queries.sql. To add or alter tables, edit sql/schema.sql. After editing either, run sqlc generate. Requires sqlc.

gRPC server

The gRPC server uses protobufs to communicate. To alter the protocol, edit protos/*.proto. You will need protobuf-compiler protobuf-compiler-grpc protoc-gen-go. also need protoc-gen-go-grpc.