tfrobot

tfrobot

tfrobot is tool designed to automate mass deployment of groups of VMs on ThreeFold Grid, with support of multiple retries for failed deployments

Features

• Mass Deployment: Deploy groups of vms on ThreeFold Grid simultaneously.
• Mass Cancellation: Cancel all vms on ThreeFold Grid defined in configuration file simultaneously.
• Load Deployments: Load gorups of vms deployed using TFRobot simultaneously.
• Customizable Configurations: Define Node groups, VMs groups and other configurations through YAML or JSON file.

Download

1. Download the binaries from releases
2. Extract the downloaded files
3. Move the binary to any of $PATH directories, for example:

mv tfrobot /usr/local/bin

1. Create a new configuration file, for example config.yaml:

node_groups:

  - name: battag1
    nodes_count: 2 # amount of nodes to be found
    free_cpu: 2 # number of logical cores
    free_mru: 70 # amount of memory in GB
    free_ssd: 100 # amount of ssd storage in GB
    free_hdd: 50 # amount of hdd storage in GB
    dedicated: false # are nodes dedicated
    public_ip4: false # should the nodes have free ip v4
    public_ip6: false # should the nodes have free ip v6
    certified: false # should the nodes be certified(if false the nodes could be certified of diy) 
    region: europe # region could be the name of the continents the nodes are located in (africa, americas, antarctic, antarctic ocean, asia, europe, oceania, polar)
vms:
  - name: batta_vmg1
    vms_count: 1 # amount of vms with the same configurations
    node_group: battag1 # the name of the predefined group of nodes
    cpu: 2 # number of logical cores
    mem: 2 # amount of  memory in GB
    public_ip4: false
    public_ip6: false
    flist: https://hub.grid.tf/mariobassem1.3bot/threefolddev-holochain-latest.flist
    entry_point: /usr/local/bin/entrypoint.sh
    root_size: 0 # root size in GB, 0 is the default
    ssh_key: my_key # the name of the predefined ssh key, will be defined below
    env_vars:
      key1: val1


ssh_keys: # map of ssh keys with key=name and value=the actual ssh key
  my_key: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCeq1MFCQOv3OCLO1HxdQl8V0CxAwt5AzdsNOL91wmHiG9ocgnq2yipv7qz+uCS0AdyOSzB9umyLcOZl2apnuyzSOd+2k6Cj9ipkgVx4nx4q5W1xt4MWIwKPfbfBA9gDMVpaGYpT6ZEv2ykFPnjG0obXzIjAaOsRthawuEF8bPZku1yi83SDtpU7I0pLOl3oifuwPpXTAVkK6GabSfbCJQWBDSYXXM20eRcAhIMmt79zo78FNItHmWpfPxPTWlYW02f7vVxTN/LUeRFoaNXXY+cuPxmcmXp912kW0vhK9IvWXqGAEuSycUOwync/yj+8f7dRU7upFGqd6bXUh67iMl7 ahmed@ahmedheaven"

mnemonic: REPLACE WITH YOUR MNEMONIC # mnemonic of the user
network: main # eg: main, test, qa, dev

You can use this example for further guidance

Please make sure to replace placeholders and adapt the groups based on your actual project details.

Notes:

The VMs may utilize a different number of nodes than requested due to the retries filtering out additional nodes in case of failure. Consequently, it's possible to utilize more nodes than initially requested. Duplicate field values are ignored; only last occurrence will be considered.

1. Run the deployer with path to the config file

tfrobot deploy -c path/to/your/config.yaml

Supported Configurations

Config File

Node Group

Vms Groups

Disk

Notes:

All storage resources are expected to be in GB.

In case of YAML input, floating point portion of int values will be ignored.

Ensure that memory precision does not exceed 0.001, any value greater than this threshold will be disregarded.

Usage

Subcommands

• deploy: used to mass deploy groups of vms with specific configurations

tfrobot deploy -c path/to/your/config.yaml

• cancel: used to cancel all vms deployed using specific configurations

tfrobot cancel -c path/to/your/config.yaml

• load: used to load all vms deployed using specific configurations

tfrobot load -c path/to/your/config.yaml

Flags

Notes:

Parsing is based on file extension, json format if the file had json extension, yaml format otherwise

Make sure to use every flag once. If the flag is repeated, it will ignore all values and take the last value of the flag.

Using Docker

docker build -t tfrobot -f Dockerfile ../
docker run -v $(pwd)/config.yaml:/config.yaml -it tfrobot:latest deploy -c /config.yaml

Build

To build the deployer locally clone the repo and run the following command inside the repo directory:

make build

Test

To run the deployer tests run the following command inside the repo directory:

make test

Logs

To ensure a complete log history, append 2>&1 | tee path/to/log/file to the command being executed.

tfrobot deploy -c path/to/your/config.yaml 2>&1 | tee path/to/log/file