gardener-extension-shoot-fleet-agent

Gardener Extension for Fleet agent installation

Project Gardener implements the automated management and operation of Kubernetes clusters as a service. Its main principle is to leverage Kubernetes concepts for all of its tasks.

Recently, most of the vendor specific logic has been developed in-tree. However, the project has grown to a size where it is very hard to extend, maintain, and test. With GEP-1 we have proposed how the architecture can be changed in a way to support external controllers that contain their very own vendor specifics. This way, we can keep Gardener core clean and independent.

Configuration

Example configuration for this extension controller with default config and 2 project specific configs:

apiVersion: shoot-fleet-agent-service.extensions.config.gardener.cloud/v1alpha1
kind: Configuration
defaultConfig:
  kubeconfig: #base64encoded kubeconfig of cluster running Fleet manager
  labels: #extra labels to apply to Cluster registration
    env: dev
projectConfig:  
  myproject:
    kubeconfig: #base64encoded kubeconfig of cluster running Fleet manager
    labels: 
      project: myproject
      somelabel: samevalue
    namespace: clusters
  myotherproject:
    kubeconfig: #base64encoded kubeconfig of cluster running Fleet manager
    labels: 
      project: mypotherproject
    namespace: fleetclusters

Unless the shoot is in one of the myproject or myother project it will get defaultConfiguration. Configuration supplied in ControllerRegistration will be applied as default however it can be overriden via configuration in Extension for given Shoot.

Extension-Resources

Example extension resource:

apiVersion: extensions.gardener.cloud/v1alpha1
kind: Extension
metadata:
  name: "extension-shoot-fleet-agent"
  namespace: shoot--project--abc
spec:
  type: shoot-fleet-agent

When an extension resource is reconciled, the extension controller will register Shoot cluster in Fleet management cluster(configured in kubeconfig in Configuration object above.

Please note, this extension controller relies on existing properly configured Fleet multi-cluster deployment configured above. By default configuration from ControllerRegistration is used but if Extension specifies different config for particular Shoot that configuration overrides the default settings.

Shooted seeds registration in Fleet

All shooted seeds will be ignored and not registered in Fleet. If you need to register shooted seeds please do so manually.

Fleet CRD cluster default labels

By default all clusters registered in fleet will have following labels:

Any labels defined in configuration are added to those default labels.

How to start using or developing this extension controller locally

You can run the controller locally on your machine by executing make start. Please make sure to have the kubeconfig to the cluster you want to connect to ready in the ./dev/kubeconfig file. Static code checks and tests can be executed by running VERIFY=true make all. We are using Go modules for Golang package dependency management and Ginkgo/Gomega for testing.

Feedback and Support

Feedback and contributions are always welcome. Please report bugs or suggestions as GitHub issues or join our Slack channel #gardener (please invite yourself to the Kubernetes workspace here).

Learn more!

Please find further resources about out project here:

• Our landing page gardener.cloud
• "Gardener, the Kubernetes Botanist" blog on kubernetes.io
• "Gardener Project Update" blog on kubernetes.io
• Gardener Extensions Golang library
• GEP-1 (Gardener Enhancement Proposal) on extensibility
• Extensibility API documentation