What you must be able to do. Place a workload, component, or NVIDIA software layer correctly, and tell training from inference and GPU from CPU by the demands each one makes.
In one sentenceThe shared vocabulary the rest of the exam assumes: how AI, machine learning, and deep learning relate, why GPUs differ from CPUs, how training and inference differ, and where each NVIDIA software layer fits.
Recall check: answer these from memory first
- State the relationship between AI, machine learning, and deep learning, from widest to narrowest.
- Give the one-line reason a GPU suits AI workloads that a CPU does not, and name the property each is built for.
- Name three ways training and inference make different demands on a system.
What it tests. The shared vocabulary and the NVIDIA software stack: how AI, machine learning, and deep learning relate, why GPU architecture differs from CPU architecture, and how training and inference place different demands on a system. It also covers what factors drove the recent surge in AI, common use cases across industries, the purpose of various NVIDIA solutions, and the software components that span the lifecycle of building and deploying AI.
How to study it. This is one of the two heavyweight areas, so do not treat it as warm-up. Get the GPU-versus-CPU contrast crisp: parallel throughput versus sequential latency, and why that matters for AI. Be able to say where the NVIDIA software stack pieces fit rather than just naming them, and learn training versus inference as a set of contrasting requirements you can apply to a scenario. Tie each use case to the capability it needs so distractors that propose the wrong tool stand out.
Easy to confuse
- Training versus inference. Training builds the model from data and is the heavier, throughput-bound job run on large GPU clusters; inference serves the finished model to make predictions and is judged on latency and cost per request. The exam picks one by describing the workload, not by naming it.
- GPU versus CPU architecture. A GPU has thousands of simple cores built for parallel throughput; a CPU has a few powerful cores built for sequential, low-latency work. If the scenario stresses many identical operations at once it wants the GPU, not raw clock speed.
- Machine learning versus deep learning. Deep learning is the subset of machine learning that uses many-layered neural networks and is the part driving the GPU demand. Calling everything deep learning is the trap; not all machine learning needs accelerators.
Worked example from the NCA-AIIO bank
lock_openFree sampleEssential AI Knowledgeeasy
Which combination of factors is most widely credited for enabling the dramatic performance improvements in deep learning models over the past decade?
- AAvailability of large datasets, GPU-based parallel compute, and advances in model architecturescheck_circle Correct
- BFaster internet connectivity, improved operating systems, and reduced hardware costs
- CAdoption of relational databases, faster CPUs, and improved compiler toolchains
- DWider use of edge devices, lower memory prices, and growth in mobile applications
Identify the three core technical pillars that enabled rapid improvement in deep learning performance and adoption. The recent acceleration in AI capability is attributed to three converging factors: the explosion of digitally available training data (big data), the availability of GPUs whose massively parallel architecture suits matrix-heavy deep learning operations, and algorithmic innovations including the transformer architecture introduced in 2017. Together these pillars allowed models to scale in a way that CPU-only, small-data, or older architectures could not support.
Why A is correct: These three pillars - big data, parallel GPU compute, and algorithmic advances such as the transformer architecture - are the foundational reasons deep learning achieved its recent breakthroughs.
Why B is wrong: Network speed and OS improvements are enabling infrastructure factors but not the core technical pillars. They do not directly drive model quality or training capability.
Why C is wrong: Relational databases and CPU speed gains did not drive deep learning progress. The shift to parallel GPU compute specifically unlocked the scale needed for modern AI workloads.
Why D is wrong: Edge devices and mobile apps are consumers of AI, not drivers of its foundational improvement. Lower memory prices alone do not account for architectural breakthroughs or training-scale gains.
What you must be able to do. Size and scale accelerated infrastructure for a stated workload, and choose the networking, interconnect, or DPU option that matches the constraint named in the scenario.
In one sentenceThe physical and architectural side: sizing hardware for training, scaling and clustering GPUs, the power, cooling, and facility load they impose, the on-premise versus cloud trade-off, and the high-speed networking that ties accelerators together.
Recall check: answer these from memory first
- Say why a high-speed, low-latency interconnect matters when many GPUs cooperate on a single training job.
- State what a DPU offloads from the host CPU and the benefit that buys.
- Give one constraint that favours on-premise infrastructure and one that favours cloud.
What it tests. The physical and architectural side: hardware requirements for training workloads, how to scale GPU infrastructure for different use cases, and the power, cooling, and facility requirements that accelerated systems impose. It also covers on-premise versus cloud trade-offs, the components of accelerated clusters, data centre networking protocols and high-speed network options, and the purpose and benefits of a DPU.
How to study it. This carries the largest single share of the exam, so spend the most time here. Treat the networking and clustering topics as the core: learn why high-speed, low-latency interconnects matter when many GPUs cooperate on one job, and what a DPU offloads and why that frees the host. Keep power, cooling, and facility points as concrete decision criteria rather than trivia. Practise on-premise-versus-cloud questions as trade-off judgements, because the exam wants the best fit for the stated constraint.
Easy to confuse
- DPU versus a standard NIC. A standard network card moves packets; a DPU offloads networking, storage, and security processing onto its own cores so the host CPU is freed for application work. If the scenario wants host overhead removed, not just connectivity, it is the DPU.
- Scaling up versus scaling out. Scaling up adds more GPUs or capacity inside one node; scaling out adds more nodes connected across the network. Scaling out is what makes the high-speed interconnect decisive, because the job now crosses the fabric.
- On-premise versus cloud. On-premise trades higher upfront capital and facility responsibility for control and steady-state cost; cloud trades elasticity and speed of start for ongoing spend and less control. The exam names the constraint that decides it, so read for that.
Worked example from the NCA-AIIO bank
lock_openFree sampleAI Infrastructuremedium
A team is preparing to train a large language model whose total memory footprint - accounting for model parameters, intermediate activations, and optimiser states - comfortably exceeds the capacity of a single GPU. Which infrastructure decision most directly addresses this constraint?
- AReplace the GPUs with DPUs to offload memory management to a dedicated data-processing unit.
- BSwitch to a higher-throughput Ethernet fabric between nodes, as network latency is the bottleneck that prevents the model fitting in memory.
- CIncrease fast NVMe storage capacity so the model can be streamed from disk into GPU memory in chunks during the forward pass.
- DDistribute the model across multiple GPUs so the combined memory across the pool can hold all training states simultaneously.check_circle Correct
Determine when and why a model's training memory footprint requires distribution across multiple GPUs. GPU memory must simultaneously hold model parameters, intermediate activations produced during the forward pass, and optimiser states such as first and second moment estimates. When this combined footprint exceeds the capacity of a single GPU, the model must be partitioned across multiple GPUs using strategies such as tensor parallelism or pipeline parallelism. DPUs and storage upgrades do not expand the GPU memory pool available to the training process.
Why A is wrong: DPUs handle networking, storage, and security offload from the CPU; they have no general-purpose tensor compute memory that substitutes for GPU VRAM in training workloads.
Why B is wrong: Network throughput affects gradient synchronisation speed but does not change how much GPU memory is available; upgrading the fabric cannot make a model fit in memory it does not have.
Why C is wrong: While CPU offload and disk-backed parameter swapping exist as workarounds, they introduce severe performance penalties; the standard architectural decision is to add GPUs until combined memory is sufficient, not to rely on disk streaming.
Why D is correct: When a model's parameters, activations, and optimiser states together exceed one GPU's memory, spreading them across multiple GPUs is the standard solution - each GPU holds a partition of the total working set.
What you must be able to do. Keep a shared GPU estate healthy, utilised, and fairly scheduled by picking the right monitoring measure, orchestration step, or virtualisation choice for the situation.
In one sentenceRunning accelerated infrastructure once it is built: monitoring data centre and GPU health, orchestrating and scheduling jobs across a shared estate, and the trade-offs of virtualising accelerated hardware.
Recall check: answer these from memory first
- Say what orchestration and job scheduling solve when many jobs compete for a limited pool of GPUs.
- Name two metrics that genuinely signal GPU health or utilisation, not just that a node is reachable.
- Give one thing virtualising accelerated hardware buys you and one thing it costs.
What it tests. Running accelerated infrastructure once it is built: data centre management and monitoring essentials, cluster orchestration and job scheduling, the measures and criteria used to monitor GPUs, and the considerations involved in virtualising accelerated infrastructure. The theme is keeping a shared, expensive GPU estate healthy, utilised, and fairly scheduled.
How to study it. This is the smallest area, but it is concentrated, so a focused pass earns reliable marks. Learn orchestration and job scheduling as the answer to sharing scarce GPUs across many jobs, and know which metrics actually signal GPU health and utilisation rather than guessing. Understand what virtualising accelerated hardware buys you and what it costs. Because the topics are few, drill them to certainty rather than skimming, since each one is more likely to appear.
Easy to confuse
- Orchestration versus monitoring. Orchestration decides what runs where and when, placing and scheduling jobs across the cluster; monitoring observes the health and utilisation of what is already running. The exam asks one or the other by whether the goal is to allocate work or to watch it.
- GPU utilisation versus availability. Availability says the GPU is up and reachable; utilisation says how much of its capacity the work is actually using. A node can be fully available and badly underused, which is the inefficiency the operations area cares about.
Worked example from the NCA-AIIO bank
lock_openFree sampleAI Operationsmedium
A research team submits a distributed training job that requires 16 GPUs spread across 4 nodes. The job fails to start because the scheduler allocates only 3 of the 4 nodes before a competing job claims the fourth. Which scheduling strategy is specifically designed to prevent this outcome?
- AGang scheduling, which holds all required nodes in reserve and starts all processes simultaneouslycheck_circle Correct
- BFair-share scheduling, which divides available GPU resources proportionally among all active users
- CPreemptive scheduling, which evicts lower-priority jobs to free resources for higher-priority requests
- DBackfill scheduling, which slots smaller jobs into idle windows left by reserved future allocations
Explain how gang scheduling prevents partial-allocation failures in multi-node distributed GPU training jobs. Gang scheduling (also called co-scheduling) treats all processes of a distributed job as a single indivisible unit. The scheduler withholds the job from running until every required slot on every required node is available at the same instant, then launches all ranks together. This is essential for tight-coupling frameworks such as MPI or PyTorch distributed training where all ranks must communicate from the first iteration. Without gang scheduling, partial allocations stall indefinitely, wasting the already-allocated GPUs and blocking other work.
Why A is correct: Gang scheduling ensures every process in a distributed job is launched at the same time across all required nodes. This atomic allocation eliminates the partial-allocation deadlock where some nodes are claimed by competing jobs before the full gang is assembled.
Why B is wrong: Fair-share scheduling governs resource equity across users over time, but it does not guarantee that all nodes for a single job are reserved simultaneously, so the same partial-allocation race can still occur.
Why C is wrong: Preemptive scheduling can reclaim resources from lower-priority jobs, but it does not inherently co-allocate all nodes at once. A preempted node may become available only after the remaining nodes are already taken by other jobs.
Why D is wrong: Backfill scheduling improves cluster utilisation by filling gaps around reserved slots, but it is not the mechanism that guarantees simultaneous allocation of all nodes for a single multi-node job.