sriov-controller

SRIOV controller

Note: Currently, due to CI limitations, the controller code does not get tested by CI/CD. However, the code was tested in a local test bed with Intel SRIOV NICs and proved to be functional.

SRIOV controller is a daemonset which parsing host's SRIOV VF information from sriov config map and advertises it to kubelet via Device plugin API. Once SRIOV network service information is advertised to kubelet, POD can request SRIOV network services. Configmap used by sriov-controller is generated by nsm-sriov-config-manager.

nsm-sriov-config-manager is a standalone binary which should be but not mandatory executed before sriov-controller.

nsm-sriov-manager does the following:

• Scan local host and detects ALL VFs
• Creates vfio device for every found VF
• Creates nsm-sriov-configmap.yaml which is k8s configmap file with all required information for sriov-controller to advertise VFs resources.

By default ALL VFs belong to default network service nsm-default. If VFs provide different Network Services, then for such VFs NetworkService: tag must be edited to reflect its Network Service.

Here is example of generated config map:

apiVersion: v1
data:
  0000-81-10.2: |
    NetworkService: networkservicemesh.io/sriov-nsm-default
    parentDevice: enp129s0f0
    vfLocalID: 1
    vfioDevice: /dev/vfio/93
  0000-81-10.3: |
    NetworkService: networkservicemesh.io/sriov-ns-1
    parentDevice: enp129s0f1
    vfLocalID: 1
    vfioDevice: /dev/vfio/101
  0000-81-10.4: |
    NetworkService: networkservicemesh.io/sriov-ns-1
    parentDevice: enp129s0f0
    vfLocalID: 2
    vfioDevice: /dev/vfio/94
   0000-81-10.7: |
    NetworkService: networkservicemesh.io/sriov-ns-2
    parentDevice: enp129s0f1
    vfLocalID: 3
    vfioDevice: /dev/vfio/103
  0000-81-11.0: |
    NetworkService: networkservicemesh.io/sriov-ns-2
    parentDevice: enp129s0f0
    vfLocalID: 4
    vfioDevice: /dev/vfio/96
  0000-81-11.1: |
    NetworkService: networkservicemesh.io/sriov-ns-3
    parentDevice: enp129s0f1
    vfLocalID: 4
    vfioDevice: /dev/vfio/104
  0000-81-11.4: |
    NetworkService: networkservicemesh.io/sriov-ns-4
    parentDevice: enp129s0f0
    vfLocalID: 6
    vfioDevice: /dev/vfio/98
  0000-81-11.7: |
    NetworkService: networkservicemesh.io/sriov-ns-5
    parentDevice: enp129s0f1
    vfLocalID: 7
    vfioDevice: /dev/vfio/107
kind: ConfigMap
metadata:
  labels:
    networkservicemesh.io/sriov: ""
  name: nsm-sriov-vf-list
  namespace: nsm-system

Here is example for a deployment requesting 4 SRIOV network services

apiVersion: apps/v1
kind: Deployment
spec:
  selector:
    matchLabels:
      app: nsm-sriov
  replicas: 1
  template:
    metadata:
      labels:
        app: nsm-sriov
    spec:
      containers:
        - image: networkservicemesh/sriov-client-image:latest
          name: nsm-sriov 
          command:
            - /bin/sh
            - -xec
            - |
              while true; do ( echo "Doing nothing"; sleep 30; ) done
          imagePullPolicy: Always
          resources:
            limits:
              networkservicemesh.io/sriov-ns-1: 1
              networkservicemesh.io/sriov-ns-2: 1
              networkservicemesh.io/sriov-ns-3: 1
              networkservicemesh.io/sriov-ns-4: 1    
metadata:
   name: nsm-sriov
   namespace: nsm-system