notebook-controller

Notebook Controller

The controller allows users to create a custom resource "Notebook" (jupyter notebook).

It has been developed using Golang and Kubebuilder.

Spec

The user needs to specify the PodSpec for the Jupyter notebook. For example:

apiVersion: kubeflow.org/v1
kind: Notebook
metadata:
  name: my-notebook
spec:
  template:
    spec:
      containers:
        - name: my-notebook
          image: kubeflownotebookswg/jupyter:master
          args:
            [
              "start.sh",
              "lab",
              "--LabApp.token=''",
              "--LabApp.allow_remote_access='True'",
              "--LabApp.allow_root='True'",
              "--LabApp.ip='*'",
              "--LabApp.base_url=/test/my-notebook/",
              "--port=8888",
              "--no-browser",
            ]

The required fields are containers[0].image and (containers[0].command and/or containers[0].args). That is, the user should specify what and how to run.

All other fields will be filled in with default value if not specified.

Environment parameters

Commandline parameters

metrics-addr: The address the metric endpoint binds to. The default value is :8080.

probe-addr: The address the health endpoint binds to. The default value is :8081.

enable-leader-election: Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager. The default value is false.

Implementation detail

This part is WIP as we are still developing.

Under the hood, the controller creates a StatefulSet to run the notebook instance, and a Service for it.

Build, Run, Deploy

You’ll need a Kubernetes cluster to run against. You can use KIND to get a local cluster for testing, or run against a remote cluster.

Note: Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster kubectl cluster-info shows).

Create a Docker image and deploy it (Suggested)

The Makefile will use the notebook-controller:<commit-hash>, if you want to use a different image name of tag then simply change that line in the beginning of the makefile. Build and push your image to the location specified by IMG and TAG:

make docker-build docker-push IMG=<some-registry>/notebook-controller TAG=<some-tag>

The Makefile has a deploy rule that will build and push the Docker image, create a notebook-controller-system namespace and finally generate and apply the necessary YAMLs. Deploy the controller to the cluster with the image specified by IMG and TAG:

make deploy IMG=<some-registry>/notebook-controller TAG=<some-tag>

Verify that the controller is running in the notebook-controller-system namespace:

$ kubectl get pods -l app=notebook-controller -n notebook-controller-system
NAME                                             READY   STATUS    RESTARTS   AGE
notebook-controller-deployment-564d76877-mqsm8   1/1     Running   0          16s

Build and Run the Controller locally

In order to build the controller you will need to use Go 1.17 and up in order to have Go Modules support. You will also need to have a k8s cluster.

1. Install the CRDs into the cluster:

make install

1. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):

make run

1. To run with culling first you need:

i) Proxy requests from localhost to NotebookPod Service (this will run in the foreground, so switch to a new terminal:

kubectl proxy

ii) Apply an AuthorizationPolicy to grant appropriate access privileges for the /api/kernels endpoint:

kubectl apply -f hack/dev_culling_authorization_policy.yaml

iii) run the controller locally:

make run-culling

Clean-up

Uninstall the notebook controller manager:

make undeploy

Uninstall the notebooks.kubeflow.org CRD:

make uninstall

Contributing

https://www.kubeflow.org/docs/about/contributing/

Development Environment

To develop on notebook-controller, your environment must have the following:

• go version v1.17+.
• docker version 20.10+.
• kubectl version 1.22.0+.
• kustomize version 3.8.7+.
• kubernetes Access to a Kubernetes v1.22.0+ cluster.
• kubebuilder version 3.3.0+.

In order for the custom Notebook Controller to be functional from your local machine, the admins must:

1. Set the number of replicas to zero:

   kubectl edit deployment notebook-controller-deployment -n=kubeflow
   

2. Allow the controller to proxy the traffic to the Notebook Services by executing on your local machine:

   kubectl proxy
   

3. Start the manager locally in developer mode:

   export DEV="true"
   make run
   

Testing

Make sure all the tests are passing after you add a new feature:

make test

TODO

• e2e test (we have one testing the jsonnet-metacontroller one, we should make it run on this one)
• status field should reflect the error if there is any. See #2269.
• Istio integration (controller will generate istio resources to secure each user's notebook)
• CRD validation
• ttlSecondsAfterFinished: This is in the original jsonnet controller spec, but not being used yet. I think we want to cleanup the notebook after idle?
• Add more instructions on contributing like build,deploy and test locally.
• A script for installing all deps.