special-resource-operator

command module
v0.0.0-...-5e97683 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 18, 2022 License: Apache-2.0 Imports: 33 Imported by: 0

README

Special Resource Operator

The Special Resource Operator (SRO) manages the deployment of software stacks for hardware accelerators on an existing (day 2) OpenShift or Kubernetes cluster. SRO can be used for a case as simple as building and loading a single kernel module, or as complex as deploying the driver, device plugin, and monitoring stack for a hardware accelerator.

For loading kernel modules, SRO is designed around the use of "driver containers." Driver containers are increasingly being used in cloud-native environments, especially when run on pure container operating systems to deliver hardware drivers to the host.

Driver containers extend the kernel stack beyond the out-of-box software and hardware features of a specific kernel. Driver containers work on various container capable Linux distributions. With driver containers the host stays "clean" and there will not be any clash between different library versions or binaries on the host.

Installation

Note: The Special Resource operator has a dependency on the Node Feature Discovery (NFD) operator. If deploying on OpenShift from OperatorHub, NFD will be installed automatically. If deploying from the CLI, first install NFD.

From OperatorHub

The Special Resource Operator is available as a community operator on OperatorHub, and as an official Red Hat operator (tech-preview) starting in OpenShift 4.9.

From the CLI

Deploy to vanilla k8s:

$ git clone https://github.com/openshift-psap/special-resource-operator
$ cd special-resource-operator
$ make deploy TAG=master

Deploy to OCP:

$ git clone https://github.com/openshift-psap/special-resource-operator
$ cd special-resource-operator
$ make deploy TAG=master

To build and deploy using a custom operator image:

$ make local-image-build
$ make local-image-push
$ make deploy

Note: The image TAG will default to the name of the current git branch, but can be overriden by setting the TAG variable. See the Makefile for more details.

To deploy the simple-kmod example special resource on OpenShift 4.x:

$ oc apply -f charts/example/simple-kmod-0.0.1/simple-kmod.yaml

Running locally

NOTE: SRO depends on objects that are created using other deployment methods. It is recommended to first deploy SRO, scale it down and then run locally.

Special Resource Operator can be run locally against a cluster using following command:

$ make deploy
$ kubectl scale -n special-resource-operator deployment/special-resource-controller-manager --replicas=0
$ make manager helm-plugins
$ HELM_PLUGINS=$PWD/helm-plugins KUBECONFIG=$HOME/.kube/config OPERATOR_NAMESPACE=special-resource-operator ./manager

SRO manages a subdirectory inside Go's os.UserCacheDir for the Helm cache.

Creating a special resource recipe

For most updated examples and use cases please see docs/lab/README.md.

See docs/recipes.md for instructions on how to create a recipe for SRO to manage.

See charts/example for some examples. In particular:

  • The simple-kmod example shows how to build and deploy two simple kernel modules in a driver container on OpenShift.
  • The centos-simple-kmod example uses the same kernel module as simple-kmod, but is written for running on a vanilla kubernetes cluster with CentOS worker nodes.

Node Feature Discovery dependency

There is a general problem when trying to configure a cluster with a special resource. One does not know which nodes have a special resource and which do not. To address this, SRO relies on the NFD operator. NFD will label the host with node specific attributes, like PCI cards, kernel or OS version and more. The .yaml template files in a special resource recipe can use these NFD labels in their nodeSelector fields to ensure that the software stack is run only on the nodes with the hardware feature. See upstream NFD for more info.

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
api
v1beta1
Package v1beta1 contains API Schema definitions for the sro v1beta1 API group +kubebuilder:object:generate=true +groupName=sro.openshift.io
Package v1beta1 contains API Schema definitions for the sro v1beta1 API group +kubebuilder:object:generate=true +groupName=sro.openshift.io
cmd
cli
internal
controllers/state
Package state is a generated GoMock package.
Package state is a generated GoMock package.
resourcehelper
Package resourcehelper is a generated GoMock package.
Package resourcehelper is a generated GoMock package.
pkg
assets
Package assets is a generated GoMock package.
Package assets is a generated GoMock package.
clients
Package clients is a generated GoMock package.
Package clients is a generated GoMock package.
cluster
Package cluster is a generated GoMock package.
Package cluster is a generated GoMock package.
kernel
Package kernel is a generated GoMock package.
Package kernel is a generated GoMock package.
lifecycle
Package lifecycle is a generated GoMock package.
Package lifecycle is a generated GoMock package.
metrics
Package metrics is a generated GoMock package.
Package metrics is a generated GoMock package.
poll
Package poll is a generated GoMock package.
Package poll is a generated GoMock package.
proxy
Package proxy is a generated GoMock package.
Package proxy is a generated GoMock package.
registry
Package registry is a generated GoMock package.
Package registry is a generated GoMock package.
resource
Package resource is a generated GoMock package.
Package resource is a generated GoMock package.
runtime
Package runtime is a generated GoMock package.
Package runtime is a generated GoMock package.
storage
Package storage is a generated GoMock package.
Package storage is a generated GoMock package.
upgrade
Package upgrade is a generated GoMock package.
Package upgrade is a generated GoMock package.
test

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL