NCA-ADS - Foundations of Accelerated Data Science (12% of the exam) - Section 5.3
Compare CPU and GPU workloads.
Compare CPU and GPU suitability for different workload types, recognising that data-parallel operations such as matrix multiplication and large-scale aggregations benefit most from GPU execution. Identify workloads where data transfer overhead or branching logic makes CPU processing preferable.
Practice question for this objective
lock_openFree sampleFoundations of Accelerated Data Sciencemedium
A team is choosing between a CPU-based and a GPU-based pipeline for two tasks: (1) computing element-wise transformations across 200 million floating-point values, and (2) executing a decision-tree traversal where each row follows a unique branch path determined by its feature values. Which assignment best matches each task to the correct processing unit?
AGPU for the element-wise transformations and CPU for the decision-tree traversal, because uniform parallel operations over large arrays suit GPU throughput while conditional, branchy logic suits the CPU's out-of-order execution model.check_circle Correct
BGPU for both tasks, because modern GPUs have enough cores to handle branchy tree traversal in parallel across all rows simultaneously.
CCPU for both tasks, because CPUs have higher single-core clock speeds and larger caches that benefit all data science workloads regardless of data size.
DCPU for the element-wise transformations and GPU for the decision-tree traversal, because CPUs handle large vectorised operations more efficiently than GPUs do.
Identify which workload characteristics favour GPU acceleration versus CPU processing for data science tasks. GPU throughput derives from executing thousands of threads in lockstep on uniform operations (SIMD model). Element-wise floating-point transformations over large arrays have no branching and identical instruction sequences per element, making them ideal for GPU parallelism. Decision-tree traversal, by contrast, sends each row down a unique conditional path determined by its feature values; on a GPU this produces warp divergence where threads in the same warp must wait for divergent branches to serialise, negating the parallelism benefit. The CPU's deeper branch predictor, larger cache hierarchy, and out-of-order execution pipeline are well-suited to this serial, branchy control flow.
Why A is correct: Element-wise operations over 200 million values are embarrassingly parallel with no branching, exactly matching GPU strengths. Decision-tree traversal involves per-row conditional branching that causes CUDA warp divergence, degrading GPU efficiency; the CPU's branch predictor and out-of-order execution handle this pattern far better.
Why B is wrong: GPU cores execute in SIMD warps where divergent branches cause serialisation within a warp; branchy decision-tree traversal with per-row unique paths leads to warp divergence and poor GPU utilisation, making a CPU the better fit for that task.
Why C is wrong: Higher single-core clock speed and larger caches do not compensate for the lack of parallelism when processing 200 million values; the element-wise transformation is exactly the class of embarrassingly parallel, compute-heavy work where GPUs excel.
Why D is wrong: This assignment is reversed. Large-scale element-wise floating-point work is embarrassingly parallel and is precisely where GPU throughput dominates; CPU vector units cannot match thousands of GPU cores for that task at 200 million elements.
Exam traps in Foundations of Accelerated Data Science
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-ADS bank for this domain.
cuDF does not support group-by aggregations on small DataFrames and falls back silently to a CPU-based path.
errorWhy it is wrong: cuDF does support group-by aggregations regardless of DataFrame size; there is no automatic silent fallback that would explain the slowdown as a capability gap.
cuDF silently falls back to CPU execution for tables under one million rows, so the rewrite gained nothing and added import overhead.
errorWhy it is wrong: Tempting because a size-based fallback would explain the result, but cuDF does not transparently switch to CPU below a row threshold; it runs on the GPU regardless, so the cause lies elsewhere.
Place both stages on the GPU, because a unified device removes the need to coordinate two processing units and the GPU handles any workload at least as well as a CPU.
errorWhy it is wrong: Tempting because a single device simplifies coordination, but a GPU underperforms on low-volume, latency-sensitive, one-at-a-time parsing where there is little parallel work, so forcing stage one onto the GPU is a poor fit.