Installing the NVIDIA GPU Operator — NVIDIA GPU Operator 24.6.1 documentation (2024)

Specifying the Operator Namespace

Both the Operator and operands are installed in the same namespace.The namespace is configurable and is specified during installation.For example, to install the GPU Operator in the nvidia-gpu-operator namespace:

$ helm install --wait --generate-name \ -n nvidia-gpu-operator --create-namespace \ nvidia/gpu-operator

If you do not specify a namespace during installation, all GPU Operator components are installed in the default namespace.

Preventing Installation of Operands on Some Nodes

By default, the GPU Operator operands are deployed on all GPU worker nodes in the cluster.GPU worker nodes are identified by the presence of the label feature.node.kubernetes.io/pci-10de.present=true.The value 0x10de is the PCI vendor ID that is assigned to NVIDIA.

To disable operands from getting deployed on a GPU worker node, label the node with nvidia.com/gpu.deploy.operands=false.

$ kubectl label nodes $NODE nvidia.com/gpu.deploy.operands=false

Preventing Installation of NVIDIA GPU Driver on Some Nodes

By default, the GPU Operator deploys the driver on all GPU worker nodes in the cluster.To prevent installing the driver on a GPU worker node, label the node like the following sample command.

$ kubectl label nodes $NODE nvidia.com/gpu.deploy.driver=false

Installation on Red Hat Enterprise Linux

In this scenario, use the NVIDIA Container Toolkit image that is built on UBI 8:

$ helm install --wait --generate-name \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ --set toolkit.version=v1.16.1-ubi8

Replace the v1.16.1 value in the preceding command with the version that is supportedwith the NVIDIA GPU Operator.Refer to the GPU Operator Component Matrix on the platform support page.

When using RHEL8 with Kubernetes, SELinux must be enabled either in permissive or enforcing mode for use with the GPU Operator.Additionally, network restricted environments are not supported.

Pre-Installed NVIDIA GPU Drivers

In this scenario, the NVIDIA GPU driver is already installed on the worker nodes that have GPUs:

$ helm install --wait --generate-name \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ --set driver.enabled=false

Pre-Installed NVIDIA GPU Drivers and NVIDIA Container Toolkit

In this scenario, the NVIDIA GPU driver and the NVIDIA Container Toolkit are already installed onthe worker nodes that have GPUs.

Before installing the Operator, ensure that the default runtime is set to nvidia.Refer to Configuration in the NVIDIA Container Toolkit documentation for more information.

Install the Operator with the following options:

$ helm install --wait --generate-name \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ --set driver.enabled=false \ --set toolkit.enabled=false

Pre-Installed NVIDIA Container Toolkit (but no drivers)

In this scenario, the NVIDIA Container Toolkit is already installed on the worker nodes that have GPUs.

1. Configure toolkit to use the root directory of the driver installation as /run/nvidia/driver, because this is the path mounted by driver container.

   $ sudo sed -i 's/^#root/root/' /etc/nvidia-container-runtime/config.toml

1. Install the Operator with the following options (which will provision a driver):

   $ helm install --wait --generate-name \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ --set toolkit.enabled=false

Running a Custom Driver Image

If you want to use custom driver container images, such as version 465.27, thenyou can build a custom driver container image. Follow these steps:

• Rebuild the driver container by specifying the $DRIVER_VERSION argument when building the Docker image. Forreference, the driver container Dockerfiles are available on the Git repository at https://gitlab.com/nvidia/container-images/driver.

• Build the container using the appropriate Dockerfile. For example:

  $ docker build --pull -t \ --build-arg DRIVER_VERSION=455.28 \ nvidia/driver:455.28-ubuntu20.04 \ --file Dockerfile .

  Ensure that the driver container is tagged as shown in the example by using the driver:<version>-<os> schema.

• Specify the new driver image and repository by overriding the defaults inthe Helm install command. For example:

  $ helm install --wait --generate-name \ -n gpu-operator --create-namespace \ nvidia/gpu-operator \ --set driver.repository=docker.io/nvidia \ --set driver.version="465.27"

These instructions are provided for reference and evaluation purposes.Not using the standard releases of the GPU Operator from NVIDIA would mean limitedsupport for such custom configurations.

Rancher Kubernetes Engine 2

For Rancher Kubernetes Engine 2 (RKE2), set the following in the ClusterPolicy.

toolkit: env: - name: CONTAINERD_CONFIG value: /var/lib/rancher/rke2/agent/etc/containerd/config.toml.tmpl - name: CONTAINERD_SOCKET value: /run/k3s/containerd/containerd.sock - name: CONTAINERD_RUNTIME_CLASS value: nvidia - name: CONTAINERD_SET_AS_DEFAULT value: "true"

These options can be passed to GPU Operator during install time as below.

helm install gpu-operator -n gpu-operator --create-namespace \ nvidia/gpu-operator $HELM_OPTIONS \ --set toolkit.env[0].name=CONTAINERD_CONFIG \ --set toolkit.env[0].value=/var/lib/rancher/rke2/agent/etc/containerd/config.toml.tmpl \ --set toolkit.env[1].name=CONTAINERD_SOCKET \ --set toolkit.env[1].value=/run/k3s/containerd/containerd.sock \ --set toolkit.env[2].name=CONTAINERD_RUNTIME_CLASS \ --set toolkit.env[2].value=nvidia \ --set toolkit.env[3].name=CONTAINERD_SET_AS_DEFAULT \ --set-string toolkit.env[3].value=true

MicroK8s

For MicroK8s, set the following in the ClusterPolicy.

toolkit: env: - name: CONTAINERD_CONFIG value: /var/snap/microk8s/current/args/containerd-template.toml - name: CONTAINERD_SOCKET value: /var/snap/microk8s/common/run/containerd.sock - name: CONTAINERD_RUNTIME_CLASS value: nvidia - name: CONTAINERD_SET_AS_DEFAULT value: "true"

These options can be passed to GPU Operator during install time as below.

helm install gpu-operator -n gpu-operator --create-namespace \ nvidia/gpu-operator $HELM_OPTIONS \ --set toolkit.env[0].name=CONTAINERD_CONFIG \ --set toolkit.env[0].value=/var/snap/microk8s/current/args/containerd-template.toml \ --set toolkit.env[1].name=CONTAINERD_SOCKET \ --set toolkit.env[1].value=/var/snap/microk8s/common/run/containerd.sock \ --set toolkit.env[2].name=CONTAINERD_RUNTIME_CLASS \ --set toolkit.env[2].value=nvidia \ --set toolkit.env[3].name=CONTAINERD_SET_AS_DEFAULT \ --set-string toolkit.env[3].value=true

CUDA VectorAdd

In the first example, let’s run a simple CUDA sample, which adds two vectors together:

1. Create a file, such as cuda-vectoradd.yaml, with contents like the following:

   apiVersion: v1kind: Podmetadata: name: cuda-vectoraddspec: restartPolicy: OnFailure containers: - name: cuda-vectoradd image: "nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.7.1-ubuntu20.04" resources: limits: nvidia.com/gpu: 1

2. Run the pod:

   $ kubectl apply -f cuda-vectoradd.yaml

   The pod starts, runs the vectorAdd command, and then exits.

3. View the logs from the container:

   $ kubectl logs pod/cuda-vectoradd

   Example Output

   [Vector addition of 50000 elements]Copy input data from the host memory to the CUDA deviceCUDA kernel launch with 196 blocks of 256 threadsCopy output data from the CUDA device to the host memoryTest PASSEDDone

4. Removed the stopped pod:

   $ kubectl delete -f cuda-vectoradd.yaml

   Example Output

   pod "cuda-vectoradd" deleted

Jupyter Notebook

You can perform the following steps to deploy Jupyter Notebook in your cluster:

1. Create a file, such as tf-notebook.yaml, with contents like the following example:

   ---apiVersion: v1kind: Servicemetadata: name: tf-notebook labels: app: tf-notebookspec: type: NodePort ports: - port: 80 name: http targetPort: 8888 nodePort: 30001 selector: app: tf-notebook---apiVersion: v1kind: Podmetadata: name: tf-notebook labels: app: tf-notebookspec: securityContext: fsGroup: 0 containers: - name: tf-notebook image: tensorflow/tensorflow:latest-gpu-jupyter resources: limits: nvidia.com/gpu: 1 ports: - containerPort: 8888 name: notebook

2. Apply the manifest to deploy the pod and start the service:

   $ kubectl apply -f tf-notebook.yaml

3. Check the pod status:

   $ kubectl get pod tf-notebook

   Example Output

   NAMESPACE NAME READY STATUS RESTARTS AGEdefault tf-notebook 1/1 Running 0 3m45s

4. Because the manifest includes a service, get the external port for the notebook:

   $ kubectl get svc tf-notebook

   Example Output

   NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEtf-notebook NodePort 10.106.229.20 <none> 80:30001/TCP 4m41s

5. Get the token for the Jupyter notebook:

   $ kubectl logs tf-notebook

   Example Output

   [I 21:50:23.188 NotebookApp] Writing notebook server cookie secret to /root/.local/share/jupyter/runtime/notebook_cookie_secret[I 21:50:23.390 NotebookApp] Serving notebooks from local directory: /tf[I 21:50:23.391 NotebookApp] The Jupyter Notebook is running at:[I 21:50:23.391 NotebookApp] http://tf-notebook:8888/?token=3660c9ee9b225458faaf853200bc512ff2206f635ab2b1d9[I 21:50:23.391 NotebookApp] or http://127.0.0.1:8888/?token=3660c9ee9b225458faaf853200bc512ff2206f635ab2b1d9[I 21:50:23.391 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).[C 21:50:23.394 NotebookApp]To access the notebook, open this file in a browser: file:///root/.local/share/jupyter/runtime/nbserver-1-open.htmlOr copy and paste one of these URLs: http://tf-notebook:8888/?token=3660c9ee9b225458faaf853200bc512ff2206f635ab2b1d9or http://127.0.0.1:8888/?token=3660c9ee9b225458faaf853200bc512ff2206f635ab2b1d9

The notebook should now be accessible from your browser at this URL:http://your-machine-ip:30001/?token=3660c9ee9b225458faaf853200bc512ff2206f635ab2b1d9.