docker-ona

docker-ona

A Docker commandline plugin that provisions complete system deployments using the infrastructure we have at CSIRO O&A.

Usage

docker ona ls <optional|domain>

List all the existing deployments managed by gitlab (named by top level DNS).

dow184@TOWER-SL:~/src/onaci/docker-ona$ docker ona ls
List all deployments on git.ona.im:
 DOMAIN              NAME                  BRANCH         STACKNAMES                                                PIPELINE                                          RUNNERS 
 ais.ona.im          swarm-infra           master         --all                                                     failed (260: 2019-09-19 10:15:37.3 +1000 AEST)    5       
 b.ona.im            docker-compose-files  master         traefik keycloak elasticsearch prometheus cronicle gitlab failed (71: 2019-08-21 16:32:32.003 +1000 AEST)   8       
 ereefs4-demo.ona.im coastal-vis-dashboard master         vis-dashboard                                             success (256: 2019-09-18 16:37:39.442 +1000 AEST) 4       
 ereefs4-demo.ona.im viz-portal            recom.csiro.au dashboard                                                 success (187: 2019-09-12 10:52:30.702 +1000 AEST) 4       
 ereefs4-demo.ona.im recom                 master         recom                                                     success (262: 2019-09-19 13:16:29.433 +1000 AEST) 9       
 ereefs4-demo.ona.im pywps-server          recom.csiro.au ncwps                                                     failed (269: 2019-09-19 20:55:41.263 +1000 AEST)  4       
 ereefs4-demo.ona.im data-proxy            recom.csiro.au cache                                                     success (99: 2019-09-04 08:57:56.157 +1000 AEST)  4       
 ereefs4-demo.ona.im swarm-infra           recom.csiro.au seaweedfs traefik prometheus keycloak                     success (259: 2019-09-19 10:15:35.623 +1000 AEST) 4       
 idc                 dont-panic            skeleton                                                                 success (271: 2019-09-20 09:23:31.358 +1000 AEST) 6       
 idc                 marlin                master                                                                   success (234: 2019-09-16 17:14:41.036 +1000 AEST) 6       
 idc                 airchem               master                                                                   failed (272: 2019-09-20 11:00:55.553 +1000 AEST)  6       
 onaci-gitops        gitops-devops-group   master                                                                   success (244: 2019-09-18 12:29:47.131 +1000 AEST) 3       
 t.ona.im            coastal-vis-dashboard master         vis-dashboard                                             success (255: 2019-09-18 16:29:37.074 +1000 AEST) 5       
 t.ona.im            swarm-infra           master         seaweedfs traefik prometheus keycloak                     failed (261: 2019-09-19 10:36:24.665 +1000 AEST)  9       

docker ona ps <domain>

List the Swarm services running on the swarm

dow184@TOWER-SL:~/src/onaci/docker-ona$ docker ona ps ais.ona.im 
ais.ona.im. IN A 152.83.141.112
List all Services on ais.ona.im:
 NAME                                    MODE       REPLICAS IMAGE                                        
 seaweedfs_master                        replicated x/x      chrislusf/seaweedfs:latest                   
 prometheus_fluentd                      global     x/x      grafana/fluent-plugin-loki:master            
 traefik_catch-all                       replicated x/x      nginx:latest                                 
 cronicle_frontend                       replicated x/x      svendowideit/docker-cronicle:latest          
 seaweedfs_docker-volume-plugin-run-once global     x/x      docker:latest                                
 job-prune-old-files-in-log-volumes      global     x/x      onaci/logprune:latest                        
 seaweedfs_volume                        global     x/x      chrislusf/seaweedfs:latest                   
 prometheus_prometheus                   replicated x/x      stefanprodan/swarmprom-prometheus:v2.5.0     
 prometheus_alertmanager                 replicated x/x      stefanprodan/swarmprom-alertmanager:v0.15.3  
 job-docker-system-prune                 global     x/x      docker:latest                                
 keycloak_gatekeeper                     replicated x/x      onaci/keycloak-gatekeeper:onaci              
 prometheus_dockerd-exporter             global     x/x      stefanprodan/dockerd-exporter:latest         
 seaweedfs_etcd                          global     x/x      appcelerator/etcd:latest                     
 prometheus_loki                         replicated x/x      grafana/loki:master                          
 prometheus_cadvisor                     global     x/x      google/cadvisor:latest                       
 prometheus_node-exporter                global     x/x      stefanprodan/swarmprom-node-exporter:v0.16.0 
 keycloak_echo                           replicated x/x      onaci/docker-http-https-echo:onaci           
 keycloak_server                         replicated x/x      onaci/keycloak:latest                        
 traefik_server                          global     x/x      traefik:v1.7.14-alpine                       
 keycloak_postgresql                     replicated x/x      postgres:10.6-alpine                         
 gitlab_runner_ais-ona-im_swarm-infra    replicated x/x      onaci/gitlab-runner:latest                   
 prometheus_onaci-exporter               replicated x/x      onaci/prometheus-exporter:latest             
 prometheus_unsee                        replicated x/x      cloudflare/unsee:v0.8.0                      
 seaweedfs_filer                         global     x/x      chrislusf/seaweedfs:latest                   
 cronicle_userapi                        replicated x/x      nginx:latest                                 
 prometheus_grafana                      replicated x/x      onaci/swarmprom-grafana:6                    
 prometheus_blackbox                     replicated x/x      prom/blackbox-exporter:latest                
 seaweedfs_s3                            replicated x/x      chrislusf/seaweedfs:latest 

docker ona create <domain> <optional|git_repo_url and options as per create>

Creates a new gitlab group called . Sets some basic CI/CD variables from Vault.

(Doesn't add swarm-infra, that may not be the purpose)

If git_repo options are set, follow through with the docker ona add functionality

***** this is starting to feel mildly useless.

docker ona add --branch <master> --stack <stack-name> <domain> <git_repo_url>

Adds the <git_repo_url> project to the specified group using mirroring

if the domain resolves, add a runner for that repo.

• TODO: also disable shared runners if the domain resolves

docker ona trigger --mirror=true --pipeline==false <domain> <optional|repo_name>

Requests the CI/CD system to re-mirror the repo from its source, or to trigger the CI/CD pipeline.

docker ona update <domain>

Check the existing config, and reapply it if needed

initially, this will make sure:

• the swarm is up and functioning
• the gitlab runners for all projects are running, and talking to gitlab (if not, remakes them)

Details

All the non-local deployment configuration is stored in our gitlab server, and secrets are stored in our vault server.

Installation

Install by copying the appropriate platform binary to your ~/.docker/cli-plugins/ dir.

NOTE: requires Docker 19.03 or above.

test setup

docker ona create --stack traefik --stack keycloak --stack elasticsearch --stack prometheus --stack cronicle --stack gitlab  b.ona.im ssh://git@bitbucket.csiro.au:7999/sima/docker-compose-files.git
docker ona trigger b.ona.im

and then setup the extra secrets we need for all the magic

Ideas

docker ona update traefik.yml should read what stacks it depends on, and tell the user, or start them first

test if the remote docker daemon is actually a swarm - help the user if not

docker ona create local should just add the info to the .docker/config.json? thus allowing you to do local deploys

docker ona trigger <domain>

auto-add all the remote swarms it finds as remote contexts so we can just interact with them directly

hook up the configured vault as a docker secrets plugin

JSON definition for a cluster.. to be used for create and update

optionally without the global "stackdomain" setting (or over-ridden from the cmdline), so I can set up a test cluster to look like a prod one...

stackdomain: "t.ona.im"
repositories:
    - swarm-infra:
        url: https://github.com/onaci/swarm-infra
        branch: master
        deploy:
            - traefik
            - prometheus
            - seaweedfs
    - ncwps:
        url: https://anything@bitbucket.csiro.au/scm/onaci/pywps-server.git
        branch: recom.ona.im
        deploy:
            - ncwps:
                stackdomain: data.recom.ona.im
    - ....

Useful URL's to know

Get all projects:

curl --header "PRIVATE-TOKEN: GITLABTOKEN" "https://git.ona.im/api/v4/projects/"

get a project's info:

curl --header "PRIVATE-TOKEN: GITLABTOKEN" "https://git.ona.im/api/v4/projects/14"

trigger a mirror pull:

curl --header "PRIVATE-TOKEN: GITLABTOKEN" -X POST "https://git.ona.im/api/v4/projects/14/mirror/pull"