ExamworthyExamworthy
Open the appStart free

NCA-AIIO - AI Operations - Section 3.2

Describe AI cluster orchestration and job scheduling essentials.

Describe how orchestration platforms such as Kubernetes with the NVIDIA GPU Operator manage containerised AI workloads, and how job schedulers such as Slurm allocate GPU resources across multi-tenant clusters. Distinguish between queue-based scheduling for batch training jobs and priority-based scheduling needed to meet latency requirements for interactive workloads.

OrchestrationJob scheduling

Practice question for this objective

Free sampleAI Operationsmedium

A platform team is migrating containerised inference services to a Kubernetes cluster that has GPUs. Multiple small inference replicas each need a fraction of a single GPU rather than an entire device. Which approach best enables multiple pods to share one physical GPU on that cluster?

  • ARequest multiple CPU cores per pod and rely on the GPU driver to time-share the device automatically across pods
  • BSet Kubernetes CPU and memory requests per pod and let the default scheduler bin-pack the inference replicas onto the GPU node with no GPU-sharing driver
  • CDeploy a fractional GPU scheduler such as Run:ai or NVIDIA MIG, which partitions the GPU and exposes sub-GPU resource fractions to the Kubernetes scheduler Correct
  • DSet resource requests to zero for the GPU field so pods schedule freely, then configure a node taint to attract only inference workloads
Describe how fractional GPU scheduling enables multiple Kubernetes pods to share a single physical GPU for inference workloads. Standard Kubernetes GPU device plugins expose whole GPUs as integer resources, so each pod receives one or more complete devices. Fractional GPU solutions such as the Run:ai scheduler or NVIDIA MIG break this constraint. Run:ai provides a virtual GPU abstraction that enforces per-pod memory and compute limits on a shared physical GPU. MIG statically partitions supported NVIDIA GPUs into isolated instances with their own memory bandwidth and compute resources, each presented to Kubernetes as a separate allocatable resource. Both approaches allow multiple smaller inference pods to coexist on one GPU, improving utilisation without interference.

Why A is wrong: Standard GPU drivers do not automatically time-share a GPU across independent processes in proportion to pod CPU requests. Without an explicit partitioning or virtualisation layer, each pod would attempt to use the full device, causing contention.

Why B is wrong: CPU and memory requests constrain only host resources; they do not partition or arbitrate the GPU. With no GPU-sharing driver the default scheduler still treats the GPU as one indivisible device, so the first pod claims it and the remaining replicas are starved or fail to schedule.

Why C is correct: Fractional GPU schedulers extend the Kubernetes device-plugin model to advertise sub-GPU resources. Run:ai intercepts scheduling and enforces GPU memory and compute quotas per pod. NVIDIA Multi-Instance GPU (MIG) partitions a physical GPU into independent instances with dedicated memory and compute slices, each schedulable as a distinct Kubernetes resource.

Why D is wrong: Setting GPU requests to zero causes the Kubernetes scheduler to ignore GPU availability entirely; pods would be placed without any GPU allocation or enforcement, leading to uncontrolled device access and potential out-of-memory failures.

See more NCA-AIIO practice questions, answers explained.

More in this domain

Back to all AI Operations objectives, or the NCA-AIIO cert hub.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.