claiflow

What is claiflow

claiflow is an open-source project for managing tasks across multiple Kubernetes clusters.

claiflow provides standardized templates for Kubernetes and services running on Kubernetes to execute tasks. This helps prevent unpredictable operations and user errors.

Manipulate claiflow resources using the REST API for Kubernetes automation.

How to install claiflow?

Server

1. Requirements

   You need MariaDB version 10.6 or above.

2. Installation

   Please refer to the README.md file in the deploy package for detailed information.

Client

1. How to make own claiflow-client?

   Build the claiflow-client with the configuration that you prefer.

   You can easily build your own claiflow-client by registering predefined template-command handlers.

   Check the examples to see how to write and run claiflow-client.

How to use?

1. Kubernetes port-forward.

   kubectl -n claiflow port-forward service/claiflow-service 8099:8099
   

2. Declare cluster resource.

   curl --request POST \
       --url http://localhost:8099/api/v1/cluster \
       --header 'content-type: application/json' \
       --data '{"uuid": "00000000000000000000000000000000","name": "test cluster"}'
   

   output:

   {"uuid":"00000000000000000000000000000000","name":"test cluster","created":"2023-07-06T01:23:54.060601557Z","updated":"2023-07-06T01:23:54.060601557Z"}

   • cluster's uuid is optional.

     If it is marked as optional, it means that if no information is available, it will be generated as a random UUID.

3. Declare cluster-client-token resource.

   curl --request POST \
       --url http://localhost:8099/api/v1/cluster_token \
       --header 'content-type: application/json' \
       --data '{"cluster_uuid": "00000000000000000000000000000000","uuid": "00000000000000000000000000000001","name": "test cluster client token","summary": "test cluster client token summary","token": "CLIENT_AUTHENTICATION_TOKEN"}'
   

   output:

   {"uuid":"00000000000000000000000000000001","name":"test cluster client token","summary":"test cluster client token summary","cluster_uuid":"00000000000000000000000000000000","token":"CLIENT_AUTHENTICATION_TOKEN","issued_at_time":"2023-07-06T01:29:23.53447061Z","expiration_time":"2024-07-05T00:00:00Z","created":"2023-07-06T01:29:23.53447061Z","updated":"2023-07-06T01:29:23.53447061Z"}

   • cluster's uuid

     The cluster's uuid refers to the value generated and outputted when creating the cluster resource.

   • cluster-client-token's uuid is optional.

   • cluster-client-token's token is optional.

     The token of the cluster-client-token is the assertion information used by the claiflow-client for authentication.

4. Deploy the claiflow-client to Kubernetes cluster.

   1. Check the server url, cluster uuid, and cluster token values.

      server grpc url: localhost:18099
      cluster uuid:    00000000000000000000000000000000
      cluster token:   CLIENT_AUTHENTICATION_TOKEN
      

   1. Apply the values you found above when running claiflow-client. For more information, see the examples/README.md.

      # move examples directory
      cd examples
      # build client binary
      make go-build example=multistep_features
      # build docker image
      make docker-build example=multistep_features image=SOME_REPOSITORY_URL
      # push docker image
      make docker-push example=multistep_features image=SOME_REPOSITORY_URL
      # change your configuration in configmap: server url, cluster uuid, cluster token
      vi environment.yaml
      # create namesapce, configmap
      kubectl apply -f environment.yaml 
      # create serviceaccount, clusterrole, clusterrolebinding
      kubectl apply -f service_account.yaml
      # change image_name to your docker image
      vi application.yaml 
      # create deployment
      kubectl apply -f application.yaml
      

5. Deploy the pre-defined templates used by claiflow-client

   REPLACE INTO `template` (`uuid`, `name`, `summary`, `flow`, `inputs`, `origin`, `created`) VALUES ('example_simple', 'example_simple', '', '[{"$id":"step1","$command":"helloworld","inputs":"$inputs"}]', '{}', 'userdefined', NOW());
   REPLACE INTO `template` (`uuid`, `name`, `summary`, `flow`, `inputs`, `origin`, `created`) VALUES ('example_iter', 'example_iter', '', '[{"$id":"step1","$range":"$inputs.x_list","$steps":[{"$id":"step2","$command":"math_pow","inputs":{"x":"$step1.val","y":2}}]}]', '{}', 'userdefined', NOW());
   REPLACE INTO `template` (`uuid`, `name`, `summary`, `flow`, `inputs`, `origin`, `created`) VALUES ('example_pass_val', 'example_pass_val', '', '[{"$id":"step1","$command":"swap_command","inputs":{"param1":"$inputs.input1","param2":"$inputs.input2"}},{"$id":"step2","$command":"swap_command","inputs":{"param1":"$step1.outputs.value1","param2":"$step1.outputs.value2"}}]', '{}', 'userdefined', NOW());
   

6. Create the tasks using the template you want to execute.

   curl --request POST \
       --url http://localhost:8099/api/v1/service \
       --header 'Content-Type: application/json' \
       --data '{"cluster_uuids":["00000000000000000000000000000000"],"uuid":"c8590e761af64c6891a47b6570c0f93e","name":"example_simple","template_uuid":"example_simple","inputs":{"name":"world"}}'
   

   output:

   [{"cluster_uuid":"00000000000000000000000000000000","uuid":"c8590e761af64c6891a47b6570c0f93e","name":"example_simple","summary":"","template_uuid":"example_simple","flow":"[{"$id":"step1","$command":"helloworld","inputs":"$inputs"}]","inputs":{"name":"world"},"step_max":1,"priority":0,"created":"2023-07-06T02:08:17.580567016Z","statuses":[{"step_seq":0,"status":0,"created":"2023-07-06T02:08:17.580567016Z"}]}]

   • task's uuid is optional.

   • cluster's uuid is string or string array.

     { "cluster_uuid": "00000000000000000000000000000000"}
     { "cluster_uuid": ["00000000000000000000000000000000","00000000000000000000000000000001"]}
     

7. Check the task result.

   curl --request GET \
       --url http://localhost:8099/api/v1/cluster/00000000000000000000000000000000/service/c8590e761af64c6891a47b6570c0f93e \
       --header   "accept: application/json"
   

   output:

   {"cluster_uuid":"00000000000000000000000000000000","uuid":"c8590e761af64c6891a47b6570c0f93e","name":"example_simple","summary":"","template_uuid":"example_simple","flow":"[{"$id":"step1","$command":"helloworld","inputs":"$inputs"}]","inputs":{"name":"world"},"step_max":1,"priority":0,"created":"2023-07-06T02:08:17.580567Z","statuses":[{"step_seq":0,"status":0,"created":"2023-07-06T02:08:17.580567Z"}]}

Etc

1. More information about the APIs can be found on the Swagger page.

   http://localhost:8099/swagger/index.html

2. You can access the metric information for the server in Prometheus metric format.

   http://localhost:8099/metrics

3. Kubernetes version compatibility

   Kubernetes version v1.24.0 or later is required because claiflow-client uses storage.k8s.io/v1 CSIStorageCapacity resource information.

   claiflow-client uses the kubernetes/client-go, helm, and prometheus-operator libraries to communicate with the Kubernetes API.

   Version compatibility for each library can be found below.

   • kubernetes/client-go
   • helm
   • prometheus-operator

   Currently, claiflow-client are using the following versions

   • kubernetes/client-go: v0.26.0
   • helm: v3.11.1
   • prometheus-operator: v0.55.0