k8s-microservice-operator

Kubernetes Microservice Operator

Microservice Operator based on Operator SDK

What is a Kubernetes Operator?

Kubernetes operators are applications that are using the Kubernetes API to create auto-updating, self-monitoring and self-healing infrastructure and applications. There are used to supplement the Kubernetes resources we use day-to-day. Operators are defining Custom Resources and work on them. As such, they use the Custom Resource Definition extension of Kubernetes. The advantage of using Kubernetes custom resources is that they can be managed in the same way as your other Kubernetes resources.

Operators usually read their custom resources and build or modify Kubernetes resources based on the state of the custom resources. One example are database operators which manage database systems in Kubernetes.

Goal of the microservice operator

This operator aims to make operations a lot easier by abstracting the usual Deployment, Service and ConfigMap definitions using a simple and unified Microservice custom resource. The operator will then manage the underlying Kubernetes resources automatically.

For the operator, we need to define the following things:

• generate the Operator skeleton as we will use the Operator SDK which generates a lot of boilerplate code around running a operator and interacting with the Kubernetes API
• define a custom resource definition which is the specification of a Microservice. For simplification, it will define only a small subset of the possible configuration. We need properties for the number of replicas, the image, which ports should be exposed and the service type. Additional properties are for now out of scope but can be added rather easily.
• implement the reconcile loop. The reconcile loop defines the actions which the operator need to do when the Microservice resource is created, modified or deleted

The minimal feature set of the current operator only does creation and deletion of a deployment and service objects and omits things like ConfigMap creation.

A sample custom resource of a microservice will look like this:

apiVersion: apps.example.com/v1
kind: Microservice
metadata:
  name: microservice-test
  labels:
    app: nginx
spec:
  replicas: 2
  image: nginx:1.17.6
  ports:
    - 80
  serviceType: LoadBalancer

Building the project

To generate the Operator Skeleton and add new things to the operator, we first need to install the Operator SDK CLI.

On Mac you can use Homebrew and install it using brew install operator-sdk. On other systems follow this guide: https://v1-2-x.sdk.operatorframework.io/docs/installation/install-operator-sdk/#install-from-github-release

For example to create a new API in the Operator you can use the following command: operator-sdk create api --group apps --version v1 --kind Microservice --resource --controller

For local testing, you can use Minikube or any other local cluster tool. To install Minikube see: https://minikube.sigs.k8s.io/docs/start/

To build and deploy the operator, we can use the makefile from the Operator SDK.
Ensure that minikube is still running.

Run the following commands for deploying the operator:

# installs custom resource definitions
make install

# to run the operator in minikube we need to configure the docker daemon to use the minikube context
eval $(minikube -p minikube docker-env)

# builds docker image and deploys operator
make IMG=controller:v1 docker-build deploy

Microservice CRD

You find the CRD specification in the api folder under microservice_types.go. In this file, only change the MicroserviceSpec type as the other types are autogenerated.

You need to modify the file with the following code:

// MicroserviceSpec defines the desired state of Microservice
type MicroserviceSpec struct {
	// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
	// Important: Run "make" to regenerate code after modifying this file

	// +kubebuilder:validation:Minimum=0
	// Replicas is the number of replicas for the microservice deployment
	Replicas int32 `json:"replicas,omitempty"`

	// Image is the Docker image and tag to use for the microservice deployment
	Image string `json:"image,omitempty"`

	// Ports is the list of HTTP container ports for the microservice deployment
	Ports []int32 `json:"ports"`

	// ServiceType is the service type to use for the microservice service
	ServiceType string `json:"serviceType,omitempty"`
}

Note the +kubebuilder annotations. These can be used to add configuration for the code generation, in our case we configured a validation that replicas should at least have 0 replicas. You can read here more about validation.

When you do changes on the CRD you must run make generate and make manifests. This is necessary to regenerate the custom resource code and deployable manifests.

Example usage

A sample microservice custom resource:

apiVersion: apps.example.com/v1
kind: Microservice
metadata:
  name: microservice-sample
  labels:
    app: nginx
spec:
  replicas: 3
  image: nginx:1.17.6
  ports:
    - 80
  serviceType: NodePort

Store the contents in microservice.yaml and deploy it with kubectl apply -f microservice.yaml.

Check the pods created by the microservice operator using kubectl get po and the service with kubectl get svc.

Resources

• Operator SDK In-depth tutorial
• Operator SDK Reference
• Kubernetes client-go kubernetes client