shell-operator

Shell-operator is a tool for running event-driven scripts in a Kubernetes cluster.

This operator is not an operator for a particular software product as prometheus-operator or kafka-operator. Shell-operator provides an integration layer between Kubernetes cluster events and shell scripts by treating scripts as hooks triggered by events. Think of it as an operator-sdk but for scripts.

Shell-operator is used as a base for more advanced Addon-operator that supports helm charts and value storages.

Shell-operator provides:

• Ease the management of a Kubernetes cluster: use the tools that Ops are familiar with. It can be bash, python, kubectl, etc.
• Kubernetes object events: hook can be triggered by add, update or delete events. Learn more about hooks.
• Object selector and properties filter: Shell-operator can monitor only particular objects and detect changes in their properties.
• Simple configuration: hook binding definition is a JSON structure on stdout.

Quickstart

You need to have a Kubernetes cluster, and the kubectl must be configured to communicate with your cluster.

Steps to setup Shell-operator in your cluster are:

• build an image with your hooks (scripts)
• create necessary RBAC objects (for onKubernetesEvent binding)
• run Pod with a built image

Build an image with your hooks

A hook is a script that, when executed with --config option, returns configuration in JSON. Learn more about hooks.

Let's create a small operator that will watch for all Pods in all Namespaces and simply log a name of a new Pod.

"onKubernetesEvent" binding is used to tell shell-operator about what objects we want to watch. Create the pods-hook.sh file with the following content:

#!/usr/bin/env bash

if [[ $1 == "--config" ]] ; then
  cat <<EOF
  {"onKubernetesEvent": [
    {"kind":"Pod",
     "event":["add"]
  }
  ]}
EOF
else
  podName=$(jq -r .[0].resourceName $BINDING_CONTEXT_PATH)
  echo "Pod '${podName}' added"
fi

Make the pods-hook.sh executable:

chmod +x pods-hook.sh

You can use a prebuilt image flant/shell-operator:latest with bash, kubectl, jq and shell-operator binaries to build you own image. You just need to ADD your hook into /hooks directory in the Dockerfile.

Create the following Dockerfile in the directory where you created the pods-hook.sh file:

FROM flant/shell-operator:latest
ADD pods-hook.sh /hooks

Build image (change image tag according your Docker registry):

docker build -t "registry.mycompany.com/shell-operator:monitor-pods" .

Push image to the Docker registry accessible by Kubernetes cluster:

docker push registry.mycompany.com/shell-operator:monitor-pods

Install shell-operator in a cluster

We need to watch for Pods in all Namespaces. That means that we need specific RBAC definitions for shell-operator:

kubectl create namespace example-monitor-pods &&
kubectl create serviceaccount monitor-pods-acc \
  --namespace example-monitor-pods &&
kubectl create clusterrole monitor-pods --verb=get,watch,list --resource=pods &&
kubectl create clusterrolebinding monitor-pods \
  --clusterrole=monitor-pods \
  --serviceaccount=example-monitor-pods:monitor-pods-acc

Shell-operator can be deployed as a Pod. Put this manifest into the shell-operator-pod.yaml file:

apiVersion: v1
kind: Pod
metadata:
  name: shell-operator
spec:
  containers:
  - name: shell-operator
    image: registry.mycompany.com/shell-operator:monitor-pods
    imagePullPolicy: Always
  serviceAccountName: monitor-pods-acc

Start shell-operator by applying a shell-operator-pod.yaml file:

kubectl -n example-monitor-pods apply -f shell-operator-pod.yaml

For instance, deploy kubernetes-dashboard to trigger onKuberneteEvent:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml

Run kubectl -n example-monitor-pods logs po/shell-operator and see that the hook will print dashboard pod names:

...
INFO     : QUEUE add TASK_HOOK_RUN@KUBE_EVENTS pods-hook.sh
INFO     : TASK_RUN HookRun@KUBE_EVENTS pods-hook.sh
INFO     : Running hook 'pods-hook.sh' binding 'KUBE_EVENTS' ...
Pod 'kubernetes-dashboard-769df5545f-99xsb' added
...

To delete created objects execute:

kubectl delete ns example-monitor-pods &&
kubectl delete clusterrole monitor-pods &&
kubectl delete clusterrolebinding monitor-pods

This example is also available in /examples: monitor-pods.

Hook binding types

onStartup

This binding has only one parameter: order of execution. Hooks are loaded at start and then hooks with onStartup binding are executed in order defined by parameter. Read more about onStartup bindings here.

Example hook --config:

{"onStartup":10}

schedule

This binding is for periodical running of hooks. Schedule can be defined with granularity of seconds. Read more about schedule bindings here.

Example hook --config with 2 schedules:

{
  "schedule": [
   {"name":"every 10 min",
    "schedule":"0 */10 * * * *",
    "allowFailure":true
   },
   {"name":"Every Monday at 8:05",
    "schedule":"0 5 8 * * 1"
    }
  ]
}

onKubernetesEvent

This binding defines a subset of Kubernetes objects that Shell-operator will monitor and a jq expression to filter their properties. Read more about onKubernetesEvent bindings here.

Note: No custom resources monitoring for now, see issue #14.

Example of hook --config:

{
  "onKubernetesEvent": [
  {"name":"Execute on changes of namespace labels",
   "kind": "namespace",
   "event":["update"],
   "jqFilter":".metadata.labels"
  }]
}

Prometheus target

Shell-operator provides /metrics endpoint. More on this in METRICS document.

Examples

More examples can be found in examples directory.

License

Apache License 2.0, see LICENSE.