5

I have three Nvidia GPUs in my cluster, and so many pods are running in my cluster. How can I find which one of these pods schedule GPU and how many GPUs they schedule?

I use this link to enable the GPU schedule in my cluster for Nvidia GPU. https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/

thanks

Nader
  • 163
  • 1
  • 9

1 Answers1

6

Here's the example given from a kubectl plugin, kubectl-view-allocations:

> kubectl-view-allocations -r gpu

Resource                   Requested       Limit  Allocatable  Free
  nvidia.com/gpu           (71%) 10.0  (71%) 10.0         14.0   4.0
  ├─ node-gpu1               (0%)  __    (0%)  __          2.0   2.0
  ├─ node-gpu2               (0%)  __    (0%)  __          2.0   2.0
  ├─ node-gpu3             (100%) 2.0  (100%) 2.0          2.0    __
  │  └─ fah-gpu-cpu-d29sc         2.0         2.0           __    __
  ├─ node-gpu4             (100%) 2.0  (100%) 2.0          2.0    __
  │  └─ fah-gpu-cpu-hkg59         2.0         2.0           __    __
  ├─ node-gpu5             (100%) 2.0  (100%) 2.0          2.0    __
  │  └─ fah-gpu-cpu-nw9fc         2.0         2.0           __    __
  ├─ node-gpu6             (100%) 2.0  (100%) 2.0          2.0    __
  │  └─ fah-gpu-cpu-gtwsf         2.0         2.0           __    __
  └─ node-gpu7             (100%) 2.0  (100%) 2.0          2.0    __
     └─ fah-gpu-cpu-x7zfb         2.0         2.0           __    __

Seems to offer what you are looking for.

Furthermore apparently, sharing GPUs is possible (with Nvidia-provided software) - might be useful to you or whoever finds this in the future:

fuero
  • 9,879