NCA-GENL - Data Analysis and Visualization (14% of the exam) - Section 4.3
Prepare information for machine learning tasks on the GPU.
Prepare raw data for machine learning by performing normalisation, encoding, and feature engineering directly on the GPU to minimise CPU-to-GPU transfer overhead. Understand how GPU-resident data pipelines reduce training bottlenecks, and apply tools such as RAPIDS cuML and DALI for end-to-end GPU data preparation.
Practice question for this objective
lock_openFree sampleData Analysis and Visualizationmedium
A team is building a text classification pipeline. After tokenising a large corpus on the CPU, they notice the GPU sits idle during the tokenisation and normalisation steps, then spikes briefly during the forward pass before sitting idle again. Which redesign most effectively eliminates the host-to-device transfer bottleneck and keeps the GPU busy throughout preprocessing?
AIncrease the number of CPU worker threads feeding the DataLoader so that batches arrive faster, saturating the GPU with more frequent data transfers at a higher bandwidth per second.
BPin the input tensors in CPU memory using pinned allocation so that direct memory access transfers are faster, then copy the pinned tensors to the GPU once per epoch at the start of training.
CMove the tokenisation and feature-scaling steps onto the GPU using a GPU-native library such as cuDF or RAPIDS, keeping the tensors resident on the GPU throughout the pipeline until training is complete.check_circle Correct
DPre-tokenise the entire dataset on the CPU, serialise the resulting integer tensors to disk as memory-mapped files, then load each batch lazily from disk directly into GPU memory at training time.
Understand how keeping data on the GPU throughout preprocessing eliminates host-to-device transfer overhead in ML input pipelines. Host-to-device transfer over the PCIe bus is a well-known throughput bottleneck in GPU-accelerated ML pipelines. When preprocessing steps such as tokenisation and normalisation run on the CPU, each batch must be copied from host DRAM to GPU VRAM before the forward pass can begin, leaving the GPU idle between bursts of computation. GPU-native preprocessing libraries (for example, cuDF from the RAPIDS suite) allow tokenisation, encoding, and scaling to execute as GPU kernels, so the tensors remain resident in device memory from the moment the raw data is ingested. Because no transfer occurs between preprocessing and training, the pipeline stages overlap naturally and GPU utilisation remains high throughout. Pinned memory and prefetching are complementary optimisations that reduce per-transfer latency but do not eliminate transfers; they are therefore inferior to full on-device preprocessing for removing the bottleneck.
Why A is wrong: Adding CPU workers reduces the time data sits in a queue, but every batch still crosses the PCIe bus from host to device memory. The root cause - host-to-device transfer overhead - is not addressed; the GPU can still stall between transfers, and more frequent small transfers may worsen PCIe congestion rather than improve it.
Why B is wrong: Pinned memory does accelerate individual host-to-device copies and enables asynchronous transfer, which is a genuine optimisation. However, it does not eliminate transfers; the data still moves across the PCIe bus. Preprocessing on the CPU and then copying once per epoch still leaves the GPU idle during tokenisation, so the fundamental pipeline stall persists.
Why C is correct: Performing tokenisation and scaling directly on the GPU with a library like cuDF means the data never leaves device memory between preprocessing and training. This eliminates the host-to-device transfers that stall the pipeline, and the GPU remains active across all stages rather than waiting for the CPU to finish and then copy data across the PCIe bus.
Why D is wrong: Memory-mapped files can reduce RAM pressure and allow partial loading, which is attractive for very large datasets. However, loading from disk to GPU still requires a host-staging step; data travels through the CPU memory hierarchy before reaching the GPU. Disk I/O latency is far higher than DRAM-to-GPU transfers, so this approach introduces a new bottleneck rather than removing the existing one.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENL bank for this domain.
Raise the DataLoader worker count and enable pinned host memory so the CPU tokenisation overlaps with the forward pass and copies run faster.
errorWhy it is wrong: More workers and pinned memory overlap and accelerate transfers, but the tokenisation and normalisation still run on the CPU and the per-batch host-to-device copy remains, so the bottleneck is reduced rather than removed.
Call DataFrame.to_pandas after each transformation to validate intermediate results, then push the frame back onto the GPU before the next step runs.
errorWhy it is wrong: This looks like prudent checking, but to_pandas copies the whole frame to the host and back every step, which is exactly the transfer overhead the team is trying to remove.
scikit-learn StandardScaler fitted on a CPU NumPy array, with the scaled output then copied to the GPU.
errorWhy it is wrong: Tempting because StandardScaler is the standard normalisation tool, but fitting and transforming on CPU NumPy arrays requires keeping the full dataset in host memory and then copying the result to the GPU, reintroducing host-to-device transfer overhead on a 200-million-row dataset.