NCA-ADS - Foundations of Accelerated Data Science (12% of the exam) - Section 5.2
Understand GPU acceleration concepts.
Describe how GPU architectures use thousands of parallel cores to accelerate data-parallel workloads compared to CPU sequential execution. Understand the roles of CUDA cores, memory bandwidth, and device memory in determining the performance of GPU-accelerated data science operations.
Practice question for this objective
lock_openFree sampleFoundations of Accelerated Data Sciencemedium
A data scientist loads a 40 GB dataset into GPU memory for columnar aggregations using cuDF, but the GPU has only 24 GB of VRAM. Which characteristic of GPU-accelerated data science most directly explains why this configuration will fail before any computation begins?
AHigh memory bandwidth between CPU and GPU becomes a bottleneck when the PCIe transfer rate is slower than the dataset size.
BGPU SIMT execution requires all threads to operate on data resident in VRAM, so the dataset must fit entirely in GPU memory before kernel dispatch.check_circle Correct
CColumnar data layouts are padded to cache-line boundaries on the GPU, causing the effective memory footprint to exceed the raw dataset size by a fixed overhead.
DcuDF requires a contiguous virtual address space spanning the entire dataset, and GPU address spaces are smaller than CPU address spaces on current hardware.
Identify GPU VRAM capacity as the primary constraint that determines whether a GPU-accelerated data science workload can execute. GPU-accelerated libraries such as cuDF operate on data that must reside in VRAM because GPU compute kernels can address VRAM directly during SIMT execution. Unlike CPU workloads that can page to disk or use unified virtual memory transparently, a cuDF operation on a dataset larger than available VRAM will raise an out-of-memory error at allocation time. Understanding this constraint is essential when sizing GPU instances or deciding whether to partition data, use Dask with multiple GPUs, or fall back to a CPU workflow.
Why A is wrong: PCIe bandwidth is a throughput concern for data transfer, not the reason the configuration fails outright. A bandwidth bottleneck slows a successful computation; it does not prevent the operation from starting when the dataset exceeds VRAM capacity.
Why B is correct: GPU memory is the hard constraint in GPU-accelerated data science. SIMT (Single Instruction, Multiple Threads) execution means all active threads reference addresses in VRAM; there is no transparent spill to host RAM as with CPU virtualisation, so exceeding VRAM capacity causes an out-of-memory error before computation starts.
Why C is wrong: While columnar padding exists, it is a minor overhead and not the reason a 40 GB dataset cannot reside on a 24 GB GPU. The fundamental issue is that the raw dataset already exceeds total VRAM, making any padding discussion secondary and misleading here.
Why D is wrong: Modern GPU virtual address spaces are large and not the practical limiting factor. The actual limit is physical VRAM capacity, not virtual addressing. Citing address space size misleads a candidate into thinking a software or architecture limit causes the failure rather than the straightforward physical memory constraint.
Exam traps in Foundations of Accelerated Data Science
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-ADS bank for this domain.
Run both stages on the GPU, because once data is resident in device memory every operation benefits equally from the higher core count regardless of its control flow.
errorWhy it is wrong: Tempting because keeping data GPU-resident avoids transfer cost, but a sequential branch-dependent parser cannot be parallelised across cores, so the GPU's thousands of lanes sit idle while one path is followed.
The GPU runs each row on a deeply pipelined out-of-order core that aggressively reorders instructions to hide branch mispredictions across the column.
errorWhy it is wrong: Out-of-order, branch-heavy execution is the strength of CPU cores, not GPUs, so this is a tempting description borrowed from CPU design; GPU cores are comparatively simple and in-order, and the speedup here comes from running many elements at once rather than from sophisticated single-thread reordering.
The depth of the instruction pipeline on each core, because a longer pipeline lets each addition retire in fewer cycles once the column is loaded.
errorWhy it is wrong: Pipeline depth affects per-instruction latency, which is tempting when arithmetic seems central, but here the cores are idle waiting on data, so retiring additions faster does not help when the bottleneck is moving the column through memory.