operator

Teleport Kubernetes Operator

This package implements an operator for Kubernetes. The Teleport Kubernetes Operator allows users deploying Teleport on Kubernetes to manage Teleport resources through Kubernetes custom resources

Currently supported resources are users and roles.

For more details, read the corresponding RFD.

Architecture

Teleport Operator is a Kubernetes (K8s) operator based on the operator-sdk. The operator lives right beside the Teleport server, using a sidecar approach. When multiple replicas are running, only the leader reconciles Kubernetes resources.

Startup

When the operator starts it:

• connects to teleport using a local admin client (like tctl on an auth node)
• uses this local connection to ensure the operator role exist
• grabs the leader lock, to ensure only one operator is acting upon the modifications
• registers a Teleport bot (see MachineID) in charge of renewing certificates

After this point, the operator will listen to modifications in the K8S cluster, and act upon. All the teleport resource changes are made using a gRPC client with certificates provided by tBot.

Reconciliation

When something changes (either by a kubectl apply or from any other source), we start the reconciliation.

First, we try to identify the operation type: deletion or creation/modification.

If it's a deletion and we have our own finalizer, we remove the object in Teleport and remove the finalizer. K8S will auto-remove the object when it gets 0 finalizers.

If it's a creation/modification:

• we add the finalizer, if it's not there already
• we lookup the object in Teleport side. If it's already here we validate it was created by the operator.
• we either create or update the resource in Teleport

Diagram

      POD
+--------------------------------------------------------+
|                                                        |         +------+
|                                                        |         |      |
|     teleport                                           |         |      |
| +---------------------------------+                    |         |      |
| |                                 |                    |         +-+----+
| |                                 |                    |           |
| |                            +----+                    |           | kubectl apply -f
| |  +-------------+           |gRPC|<--+                |           |
| |  |/etc/teleport|           +----+   |                |           |
| |  +^------------+                |   |                |           |
| |   |                             |   |                |           |
| |   |   +-----------------+       |   | Manage         |           |
| |   |   |/var/lib/teleport|       |   | Resources      |           |
| |   |   +^----------------+       |   |                |           |  kube-apiserver
| |   |    |                        |   |                |      +----v----------------+
| +---+----+------------------------+   |                |      |                     |
|     |    |                            |                |      |                     |
|     |    |                            |                |      |                     |
|     |    |   operator                 |                |      |                     |
| +---+----+----------------------------+--------+       |      |                     |
| |   |    |                            |        |       |      |                     |
| |  ++----+----+                 +-----v----+   |       |      |                     |
| |  |  tBot    |-----------------> teleport <---+-------+------>                     |
| |  +----------+ Get client      | operator |   |       |      |                     |
| |              & renew certs    +----------+   |       |      |                     |
| |                                              |       |      |                     |
| |                                              |       |      |                     |
| +----------------------------------------------+       |      +---------------------+
|                                                        |
|                                                        |
|                                                        |
+--------------------------------------------------------+

Running

Requirementes

K8S cluster

If you don't have a cluster yet, you can start one by using the minikube tool.

Operator's docker image

You can obtain the docker image by pulling from public.ecr.aws/gravitational/teleport

HELM chart from Teleport with the operator

The teleport-cluster Helm chart supports deploying the operator alongside teleport.

Other tools

We also need the following tools: helm, kubectl and docker

Running the operator

Install the helm chart:

# Run the command at the root of the teleport repo
helm upgrade --install --create-namespace -n teleport-cluster \
	--set clusterName=teleport-cluster.teleport-cluster.svc.cluster.local \
	--set teleportVersionOverride="11.0.0-dev" \
	--set operator.enabled=true \
	teleport-cluster ./examples/chart/teleport-cluster

kubectl config set-context --current --namespace teleport-cluster

Wait for the deployment to finish:

kubectl wait --for=condition=available deployment/teleport-cluster --timeout=2m

If it doesn't, check the errors.

Now, we want access to two configuration tools using a Web UI: K8S UI and Teleport UI.

If you are using minikube, you have to create a tunnel with: minikube tunnel (this command runs is foreground, open another terminal for the remaining commands).

Create a new Teleport User and login in the web UI:

PROXY_POD=$(kubectl get po -l app=teleport-cluster -o jsonpath='{.items[0].metadata.name}')
kubectl exec $PROXY_POD teleport -- tctl users add --roles=access,editor teleoperator
echo "open following url (replace the invite id) and configure the user"
TP_CLUSTER_IP=$(kubectl get service teleport-cluster -o jsonpath='{ .status.loadBalancer.ingress[0].ip }')
echo "https://${TP_CLUSTER_IP}/web/invite/<id>"

Open the Kubernetes Dashboard (minikube dashboard if your cluster was created by minikube) and switch to teleport-cluster namespace. Your resources will appear under the Custom Resources menu.

You can manage users and roles using to usual kubernetes tools, for example, kubectl.

As an example, create the following file (roles.yaml) and then apply it:

apiVersion: "resources.teleport.dev/v5"
kind: TeleportRole
metadata:
  name: myrole
spec:
  allow:
    logins: ["root"]
    kubernetes_groups: ["edit"]
    node_labels:
      dev: ["dev", "dev2"]
      type: [ "compute", "x" ]

kubcetl apply -f roles.yaml

And now check if the role was created in Teleport and K8S (teleport-cluster namespace).

PROXY_POD=$(kubectl get po -l app=teleport-cluster -o jsonpath='{.items[0].metadata.name}')
kubectl exec $PROXY_POD teleport -- tctl get roles/myrole