k8sreconcile

K8S Generic Reconciler

In a Kubernetes operator, the job of reconciling the expected state of a set of objects with the actual state is a tedious one. Whether your are using the operator-sdk or writing your own operator from scratch, you will probably end up needing to create (and then reconcile) many types of Kubernetes objects - deployments, services, configmaps, etc. The actual code to compare a set of expected objects with the actual objects running in the cluster, and then create/delete/apply changes is often copied many, many times over. The lack of Golang generics is one reason for this.

This goal of this library is to solve this duplication by providing a generic utility that:

• Takes a set of expected Kubernetes objects.
• Determines any differences from the actual Kubernetes state, and does so in a generic way that doesn't rely on comparing only selected fields, which is, unfortunately, the way it's usually done. For example: https://github.com/operator-framework/operator-sdk-samples/blob/master/go/memcached-operator/pkg/controller/memcached/memcached_controller.go#L134 Doing it this way either requires a lot of custom code to check every interesting field of the object, and/or leaves your operator unable to repair state drift for fields that the operator isn't explicitly coded to look for.
• Creates objects that don't exist.
• Updates objects whose actual state doesn't match the expected.
• Deletes objects that shouldn't exist.
• Supports Listers for fetching current object state. In practice, in a production Kubernetes cluster, a single operator may be reconciling many types of objects, and there may be many operators running in the cluster. Read calls to the API server should always go through Listers to minimize traffic to the API server.

Interfaces

To allow a Kubernetes object to be handled by the Reconcile() function, a couple of interfaces (defined in interface.go) need to be implemented. These interfaces tell the Reconciler how to get/list/update/etc the object in question.

Implementations for a number of native Kubernetes objects have been provided under ./impl. These all use Listers for get/fetch operations. Feel free to add implementations for more native types via pull request.

Implementations for your own Custom Resources (to live in your own code) is also totally possible!

Usage

import (
        "github.com/dansimone/k8sreconciler"
)

Let's say in your operator, you want to reconcile a set of Kubernetes Deployments. There is an existing impl for Deployments, so we'll use that:

import (
	"github.com/dansimone/k8sreconcile"
	reconcileimpl "github.com/dansimone/k8sreconcile/impl"
	"k8s.io/apimachinery/pkg/labels"
	"k8s.io/client-go/kubernetes"
	corelistersv1 "k8s.io/client-go/listers/core/v1"
)

...
	// Constuct your list of expected deployments
	deploymentsList := createExpectedDeployments()

 	// Construct your k8s client and service lister (you would need to actually instantiate these)
 	var kubeClientSet kubernetes.Interface
 	var serviceLister corelistersv1.ServiceLister

	// Here we'll specify to delete unexpected deployments from the following namespace that match the following selector
	deleteFromNamespace := "example"
	deleteSelector := labels.SelectorFromSet(map[string]string{"app": "test-example"})

    err = k8sreconcile.Reconcile(r, r.DeploymentsToObjects(deploymentsList), deleteFromNamespace, deleteSelector)

To skip deletion of unexpected objects, provide an empty deleteFromNamespace or a nil deleteSelector:

    err = k8sreconcile.Reconcile(r, r.DeploymentsToObjects(deploymentsList), "", nil)

Examples

An example of reconciling a set of expected Deployments:

go run ./examples/example.go --v=4 --kubeconfig <your_kubeconfig>

TODOs

• Example of reconciling a Custom Resource
• CI

Contributions

Pull requests welcome.