irq-smp-balance
This provides ability to isolate the Guaranteed QoS class pod cores from handling interrupts in a K8s cluster.
Quickstart Installation
Build
Here is the procedure to compile and build irq-smp-balance docker image.
$ git clone https://github.com/pperiyasamy/irq-smp-balance.git
$ cd irq-smp-balance
$ make
$ make image
Deployment
This requires a daemon and daemonset pod running in worker nodes watching for Guaranteed QoS class pods
having labels with irq-load-balancing.docker.io=true
and exclude assigned pod CPUs from IRQ balancing.
The daemon and daemonset pod share a config file, by default both chooses /etc/sysconfig/pod_irq_banned_cpus
file.
If another config file chosen, the hostPath irqbalanceconf
in ./deployments/irqsmpbalance-daemonset.yaml
has to be updated accordingly before the deployment.
It's time to deploy the daemonset on the cluster, We'll apply a daemonset which installs irq-smp-balance
using kubectl
from this repo. From the root directory of the clone, apply the daemonset YAML file:
$ cat ./deployments/auth.yaml | kubectl apply -f -
$ cat ./deployments/irqsmpbalance-daemonset.yaml | kubectl apply -f -
Now run the daemon on the worker node:
$ nohup irqsmpdaemon </dev/null >irqsmp.out 2>irqsmp.err &
The irqsmpdaemon can also be run with different config and log files, Refer the help:
$ irqsmpdaemon -h
Usage of irqsmpdaemon:
-config string
irq balance config file (default "/etc/sysconfig/irqbalance")
-log string
log file (default "/var/log/irqsmpdaemon.log")
-podfile string
pod irq banned cpus file (default "/etc/sysconfig/pod_irq_banned_cpus")
Cleanup
build clean up:
$ make clean
Undeploy the daemonset:
$ cat ./deployments/irqsmpbalance-daemonset.yaml | kubectl delete -f -
$ cat ./deployments/auth.yaml | kubectl delete -f -
Shutting down the daemon:
$ pkill irqsmpdaemon
Debug & Troubleshooting
Refer to the following logs for more information.
# Create service account, cluster role and cluster role binding
$ kubectl create -f deployments/auth.yaml
serviceaccount/irq-smp-balance-sa created
secret/irq-smp-balance-sa-secret created
clusterrole.rbac.authorization.k8s.io/irq-smp-balance created
clusterrolebinding.rbac.authorization.k8s.io/irq-smp-balance-role-binding created
# Create smp affinity daemonset
$ kubectl create -f deployments/irqsmpbalance-daemonset.yaml
daemonset.apps/kube-smp-affinity-amd64 created
$ kubectl -n kube-system get pods
NAME READY STATUS RESTARTS AGE
kube-smp-affinity-amd64-pqwq9 1/1 Running 0 47s
# Look for daemon set pod logs
$ kubectl logs kube-smp-affinity-amd64-pqwq9 -n kube-system --follow
# Start irqsmpdaemon as a background process on the host
$ nohup irqsmpdaemon </dev/null >irqsmp.out 2>irqsmp.err &
# Look for the daemon logs
$ tail -f /var/log/irqsmpdaemon.log