Admission Webhook for adding GCP credentials to pods
Goal
We need a way to inject common data (env vars, volumes) to pods (e.g. notebooks).
See issue.
How this works
An admission controller
intercepts requests to the Kubernetes API server, and can modify and/or validate the requests.
We are implementing a custom MutatingAdmissionWebhook.
Define a MutatingWebhookConfiguration,
for example:
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
name: gcp-cred-webhook
labels:
app: gcp-cred-webhook
webhooks:
- name: gcp-cred-webhook.kubeflow.org
clientConfig:
service:
name: gcp-cred-webhook
namespace: default
path: "/add-cred"
caBundle: "..."
rules:
- operations: [ "CREATE" ]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
namespaceSelector:
matchLabels:
add-gcp-cred: "true"
This specifies
- When there is a pod being created (see
rules
) in the namespace that has labels add-gcp-cred="true"
(see namespaceSelector
),
- call the webhook service
gcp-cred-webhook.default
at path /add-cred
(see clientConfig
)
Webhook implementation
The webhook should be a server that can handled request coming from the configured path (/add-cred
in the above).
The request and response types are both AdmissionReview
The webhook check if the pod has labels:
gcp-cred-secret: SOME_SECRET
gcp-cred-secret-filename: SOME_KEY.json
If yes, it will add volume, volumeMount, and environment variable to the pod.
Reference
- https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/
- https://github.com/kubernetes/kubernetes/tree/v1.13.0/test/images/webhook
- https://github.com/morvencao/kube-mutating-webhook-tutorial
- How to self sign: link
- What to put for caBundle: issue