README
¶
waymond - Autoscale Anything Anywhere All at once! :eyes:
waymond is
- An open-source autoscaler.
- Aiming to provide autoscaling for a wide variety of infrastructure.
- Modular and extensible.
- Built with Go.
Motivation
There are a lot of autoscalers out there like AWS EC2 Autoscaling, kubernetes autoscalers, nomad autoscaler, etc. Most of them are suited towards specific type of targets (example: nomad autoscaler could only be used for nomad). There is a good deal of overlap between those autoscalers in-terms of what and how an autoscale can happen.
The original idea for waymond came up while trying to autoscale CI/CD workloads in self-hosted infrastructure. Example: Autoscale the number of CI agents running as systemd processes inside a big EC2 VM and when we run out of limits there, try to bring up new EC2 VMs that run one CI agent per machine for a given CI job queue. At the sametime, autoscale the agents running in a kubernetes cluster when jobs are arriving in a different CI job queue.
waymond tries to support a variety of autoscaling targets from operating system processes to kubernetes clusters and everything in-between like traditional VMs. One of the main goals of the project is to make it very easy to autoscale mixed type of targets. Truly anything and anywhere!
Architecture
Concepts
Triggers
Triggers are the components which could trigger an event, which could ultimately yield to autoscaling. The decision of when and how to auto-scale will flow from here.
| type | status | description |
|---|---|---|
| cron | Available | Trigger events based on cron expressions |
| http_endpoint | Looking for contribution | Starts a HTTP server in waymond and triggers event for every HTTP request |
| http_client | Looking for contribution | Creates a HTTP client in waymond and triggers event based on the HTTP response |
| buildkite | Available | Trigger event based on the CI job queue length in Buildkite |
Propose a new trigger here.
Scalers
Scalers are the components which perform autoscaling operation. They access the target APIs (like docker or AWS API) to achieve a desired state in those systems.
| type | status | description |
|---|---|---|
| docker | Available | Autoscales docker containers |
| docker_compose | Looking for contribution | Autoscales containers in a docker compose setup |
| noop | Available | noop stands for "No-Operation". This is mainly for debugging what data is being received by a problematic scaler. |
| aws_ec2 | Looking for contribution | Autoscales AWS EC2 machines |
| aws_ec2_fleet | Looking for contribution | Autoscales AWS EC2 machines via EC2 Fleet |
| aws_ec2_asg | Available | Autoscales AWS EC2 machines via Autoscaling groups |
Propose a new scaler here.
Connectors
Connectors are the components which connect two objects to facilitate the flow of events between them. Example: A connector could connect a cron trigger to docker scaler. This will ensure that the desired number of docker containers are running in a machine periodically.
Connectors can connect a trigger to a trigger (and form a chain of triggers) that ultimately is connected to a scaler. Connectors are also a good place to do data transformation of data from a trigger to a format of data that a scaler can understand.
| type | status | description |
|---|---|---|
| direct | Available | Directly pass events between Trigger-Scaler or Trigger-Trigger |
Propose a new connector here.
Install
Download the binary for your OS and architecture from the latest release. Extract the compressed package and move it an executable PATH.
# example: for linux amd64
wget https://github.com/scriptnull/waymond/releases/latest/download/waymond_Linux_x86_64.tar.gz
tar -xf waymond_Linux_x86_64.tar.gz
mv waymond /usr/bin/waymond
Command
The only way to run waymond right now is
waymond -config waymond.toml
But the project is looking to improve the CLI experience. So, please take a look here if you would like to contribute.
Configuration
waymond accepts a toml configuration file when it boots up to configure all its components.
#
# For every 1 minute, waymond will check if there are two redis docker containers running and run them if not
#
[[trigger]]
type = "cron"
id = "global_cron"
expression = "*/1 * * * *"
[[scaler]]
type = "docker"
id = "local_redis_containers"
image_name = "redis"
image_tag = "latest"
count = 2
[[connect]]
type = "direct"
id = "run_redis_via_cron"
from = "trigger.global_cron"
to = "scaler.local_redis_containers"
Contribute
Software pre-requisites
If you are a user of nix package manager, you can enter the nix-shell to automatically download all the above dependencies.
Build
Run just build and ./waymond binary should be ready for use.
Community
Talk to a human
The project is currently in very early stages and it would be awesome if you could join us! If you are looking to talk to a human about the waymond project, feel free join the telegram group. If you are interested in requesting new features or report bugs, please do so in the issue tracker. If you are looking to contribute for the first time, try checking the issues tagged with good-first-issue.
Swag
All contributors to the waymond project can optionally choose to receive an one-time swag. If you have contributed to the project, please fill out this form to opt-in for receiving the swag.
Directories