AI Operations

A cloud provider needs to share a single A100 GPU among seven independent tenants running small inference workloads, with each tenant requiring guaranteed memory and compute bandwidth that cannot be affected by a noisy neighbour. Which virtualisation approach best meets this requirement?

• AMulti-Instance GPU partitioning, creating up to seven hardware-isolated compute and memory slices on the GPUcheck_circle Correct
• BNVIDIA vGPU in time-sliced mode, assigning each tenant a virtual GPU profile
• CFull GPU passthrough, dedicating the entire physical GPU to a single virtual machine
• DCUDA Multi-Process Service, allowing multiple CUDA processes to share GPU context simultaneously

Why A is correct: MIG uses hardware mechanisms in Ampere and later GPUs to divide a single physical GPU into isolated instances, each with its own dedicated compute engines, memory slice, and cache. This prevents cross-tenant interference and provides the guaranteed performance isolation required for multi-tenant inference.

Why B is wrong: vGPU time-slicing is tempting because it divides a GPU across many tenants, but time-sliced vGPU does not provide hardware-level memory partitioning, so one tenant's memory pressure can affect others and there is no guaranteed bandwidth isolation.

Why C is wrong: Full passthrough gives maximum performance and true isolation for one tenant, but it cannot serve multiple tenants simultaneously on the same physical GPU, making it unsuitable when seven tenants must share one device.

Why D is wrong: CUDA MPS enables concurrent GPU sharing among processes on the same machine but provides no memory isolation between clients. A misbehaving tenant can corrupt another tenant's memory, making it unsuitable where strict isolation is required.

An operations team monitors a fleet of training GPUs and wants a single early-warning signal that a GPU is being forced to reduce its clock speed because it has reached a hardware-protective limit. They already collect temperature, power draw, and utilisation. Which additional telemetry signal most directly tells them that a GPU is actively throttling rather than simply running warm or busy?

• AThe instantaneous SM (streaming multiprocessor) utilisation percentage, because a GPU that is throttling will show utilisation pinned at 100 per cent while its effective work falls.
• BThe board power draw in watts, because any value at or near the configured power limit confirms the GPU is throttling to protect itself.
• CThe clock throttle-reason flags, because they explicitly report why the current clock is being held down, including thermal, power-cap, or hardware-slowdown causes.check_circle Correct
• DThe GPU memory utilisation percentage, because memory pressure is the usual trigger that forces the scheduler to lower the core clock.

Why A is wrong: Tempting because utilisation feels like the headline activity metric, but utilisation only reports how much of the time the SMs were issuing work; it does not reveal whether the clock was lowered for protection, so a throttling GPU and a healthy busy GPU can both read high.

Why B is wrong: Tempting because hitting the power cap can trigger throttling, but power at the limit only shows the GPU is power-bound, not that the clock was actually reduced; the explicit throttle reason is what distinguishes a capped-but-fine GPU from a throttled one.

Why C is correct: Correct: the throttle-reason bitmask exposed through DCGM and NVML names the active cause of any clock reduction (thermal slowdown, software or hardware power cap, reliability slowdown), so it directly confirms throttling and its origin rather than being inferred from secondary signals.

Why D is wrong: Tempting because high memory use does correlate with heavy workloads, but memory utilisation measures occupied frame-buffer capacity and has no direct mechanism for lowering the core clock, so it cannot confirm a protective throttle event.

A cluster operator wants to qualify GPU nodes before they enter a scheduling pool by actively exercising memory, compute, and the interconnect to surface latent faults, rather than only passively reading temperature and utilisation counters during production jobs. Which DCGM capability is designed for this proactive node-qualification role?

• AContinuous polling of NVML utilisation fields exported as Prometheus metrics for dashboard alerting.
• BReading the GPU's persisted XID journal to list any fault codes recorded since the last driver reload.
• CDCGM diagnostics, which run active health checks that load the GPU's memory, compute, and links to detect faults before the node accepts jobs.check_circle Correct
• DSetting a persistence-mode flag so the driver stays resident and reduces job start-up latency on the node.

Why A is wrong: Prometheus-style polling reports passive runtime telemetry; it is tempting because it is the most familiar DCGM use, but it observes whatever the GPU is currently doing rather than actively stress-testing the GPU to expose latent faults.

Why B is wrong: Reviewing XID history surfaces faults that have already occurred, which is useful but reactive; this answer appeals because it relates to fault detection, yet it does not actively drive the hardware to provoke a latent defect before the node is used.

Why C is correct: DCGM diagnostics deliberately exercise GPU memory, compute pipelines, and interconnect with targeted test workloads, so latent defects are provoked and caught during qualification rather than during a production training run.

Why D is wrong: Persistence mode keeps the driver initialised to cut start-up overhead and performs no health testing at all; it is plausible as a node-readiness setting, but it neither tests hardware nor reports faults.

A research team submits a distributed training job that requires 16 GPUs spread across 4 nodes. The job fails to start because the scheduler allocates only 3 of the 4 nodes before a competing job claims the fourth. Which scheduling strategy is specifically designed to prevent this outcome?

• AGang scheduling, which holds all required nodes in reserve and starts all processes simultaneouslycheck_circle Correct
• BFair-share scheduling, which divides available GPU resources proportionally among all active users
• CPreemptive scheduling, which evicts lower-priority jobs to free resources for higher-priority requests
• DBackfill scheduling, which slots smaller jobs into idle windows left by reserved future allocations

Why A is correct: Gang scheduling ensures every process in a distributed job is launched at the same time across all required nodes. This atomic allocation eliminates the partial-allocation deadlock where some nodes are claimed by competing jobs before the full gang is assembled.

Why B is wrong: Fair-share scheduling governs resource equity across users over time, but it does not guarantee that all nodes for a single job are reserved simultaneously, so the same partial-allocation race can still occur.

Why C is wrong: Preemptive scheduling can reclaim resources from lower-priority jobs, but it does not inherently co-allocate all nodes at once. A preempted node may become available only after the remaining nodes are already taken by other jobs.

Why D is wrong: Backfill scheduling improves cluster utilisation by filling gaps around reserved slots, but it is not the mechanism that guarantees simultaneous allocation of all nodes for a single multi-node job.

A platform team runs a shared Kubernetes GPU cluster. They notice that small inference pods get scheduled onto whichever nodes have free GPUs, leaving GPUs scattered one-per-node. This fragmentation blocks an incoming eight-GPU training pod that needs eight free GPUs on a single node, even though the cluster has more than eight GPUs free in total. Which scheduling approach most directly reduces this fragmentation so large single-node jobs can be placed?

• ABin-packing the small pods onto as few nodes as possible so that whole nodes are left fully freecheck_circle Correct
• BSpreading the small pods evenly across all nodes to balance GPU temperature and power draw
• CEnabling cluster autoscaling so new nodes are added whenever a large job cannot be placed
• DRaising the large training pod's priority so it preempts and evicts the small inference pods

Why A is correct: Bin-packing consolidates small workloads onto fewer nodes, which preserves intact nodes with all GPUs free for large jobs and directly addresses the fragmentation described.

Why B is wrong: Spreading is tempting because it balances thermals and load, but it is the exact behaviour that causes fragmentation here, leaving one GPU free per node rather than whole free nodes.

Why C is wrong: Autoscaling can add capacity and seems to solve placement, but it adds cost and start-up delay and does not reclaim the GPUs already stranded by fragmentation across existing nodes.

Why D is wrong: Higher priority with preemption could evict small pods to make room, but it disrupts running inference and is a blunt fix rather than preventing the fragmentation that bin-packing avoids.