bootstrapper

local-volume-provisioner-bootstrap

local-volume-provisioner-bootstrap is used to bootstrap provisioner. The main use case of bootstrapper is to make provisioner configurable. Below is a detailed flow of how the bootstrap process works:

• looks for a configmap passed in via flag -volume-config. By default it looks for the configmap named local-volume-default-config. A valid configmap must be present for the bootstrap process to run.
• reads and validates the configmap, then auto-generates missing configurations, see below
• looks for a service account passed in via flag -serviceaccount; otherwise, a default service account named local-storage-admin will be created
• creates two cluster role bindings for the service account, i.e. system:persistent-volume-provisioner and system:node. The role bindings are named local-storage:provisioner-node-binding and local-storage:provisioner-pv-binding respectively.
• creates provisioner daemonset based on the service account, volume configmap, -image option, etc
• exit

Configuration

Volume config

Volume config is a map from storage class to volume configuration, i.e. map[string]MountConfig, where MountConfig is the configuration for a single volume, see below.

type MountConfig struct {
	// The hostpath directory
	HostDir string `json:"hostDir"`
	// The mount point of the hostpath volume
	MountDir string `json:"mountDir"`
}

• HostDir is required; it points to the directory where provisioner looks for local volumes.
• MountDir is optional, it is the path inside container where hostDir is mounted to. If omitted, MountDir will be auto-generated by bootstrapper. The generation rule is to trim '/' prefix and change "/" to "" (based on kubernetes convention), then concatenate with root path. For example, suppose -mountRoot flag equals to "/mnt/local-storage" and hostDir equals to "/mnt/others", then generated MountDir will be "/mnt/local-storage/mntothers".

Below is an example configmap:

kind: ConfigMap
metadata:
  name: local-volume-config
  namespace: kube-system
data:
  storageClassMap: |
    local-fast: 
       hostDir: "/mnt/ssds"
       mountDir: "/local-ssds"
    local-slow:
       hostDir: "/mnt/hdds"
       mountDir: "/local-hdds"
    local-storage:
      hostDir: "/mnt/disks"

An example of using the bootstraper is included here.

PV Label Config (optional)

One can (optionally) also configure the provisioner to copy certain labels from the node to the PVs of the local volumes on that node. For example in the config below, the admin has chosen to copy the labels failure-domain.beta.kubernetes.io/zone and failure-domain.beta.kubernetes.io/region from the node to the PVs as those zone and region labels are quite applicable to the local volume PVs of that node.

kind: ConfigMap
metadata:
  name: local-volume-config
  namespace: kube-system
data:
  nodeLabelsForPV: |
    - failure-domain.beta.kubernetes.io/zone
    - failure-domain.beta.kubernetes.io/region
  storageClassMap: |
    local-fast: 
       hostDir: "/mnt/ssds"
       mountDir: "/local-ssds"
    local-slow:
       hostDir: "/mnt/hdds"
       mountDir: "/local-hdds"
    local-storage:
      hostDir: "/mnt/disks"

Command line options

To see all options, compile bootstrapper and use -h option, below is a curated list of important options:

• -image: Name of local volume provisioner image (default "quay.io/external_storage/local-volume-provisioner")
• -volume-config: Name of the local volume configuration configmap. The configmap must reside in the same namespace with bootstrapper. (default "local-volume-default-config")
• -serviceaccount: Name of the service accout for local volume provisioner (default "local-storage-admin")

Development

Compile the bootstrapper:

make

Make the container image and push to the registry

make push

Deploy to existing cluster:

kubectl create -f deployment/kubernetes/<version>/example-config.yaml
kubectl create -f deployment/kubernetes/<version>/admin-account.yaml
kubectl create -f deployment/kubernetes/<version>/bootstrapper.yaml

The version in the path above can be latest for the latest provsioner or one of the earlier images.

Note: The configmap (as shown in example-config.yam) is required to run the Bootstrap process.

The compatibility matrix between the bootstrap container image, local volume provisioner constainer image and Kubernetes is as follows:

Future improvements

• Right now, bootstrapper is bound to cluster-admin role and provisioner is bound to system:node, system:persistent-volume-provisioner, these roles have more more privileges than we need. As a future enhancemnet, we can look into using different roles with fewer permissions.
• Bootstrapper will update user-created configmap if MountDir is omitted from user. We choose to update the confimap instead of auto-generating a new one because 1. the config will be shared between bootstrapper and provisioner; 2. the config only has settings for volumes so it won't mis-configure any other settings. We can revisit this when we have more settings in this configmap.
• Auto-generated MountDir doesn't take path length into consideration. If user provides a very long HostPath, bootstrapper will fail to create MountDir.