ExamworthyExamworthy
Open the appStart free

NCA-GENL - Data Analysis and Visualization - Section 4.3

Prepare information for machine learning tasks on the GPU.

Prepare raw data for machine learning by performing normalisation, encoding, and feature engineering directly on the GPU to minimise CPU-to-GPU transfer overhead. Understand how GPU-resident data pipelines reduce training bottlenecks, and apply tools such as RAPIDS cuML and DALI for end-to-end GPU data preparation.

Practice question for this objective

Free sampleData Analysis and Visualizationmedium

A team is building a text classification pipeline. After tokenising a large corpus on the CPU, they notice the GPU sits idle during the tokenisation and normalisation steps, then spikes briefly during the forward pass before sitting idle again. Which redesign most effectively eliminates the host-to-device transfer bottleneck and keeps the GPU busy throughout preprocessing?

  • AIncrease the number of CPU worker threads feeding the DataLoader so that batches arrive faster, saturating the GPU with more frequent data transfers at a higher bandwidth per second.
  • BPin the input tensors in CPU memory using pinned allocation so that direct memory access transfers are faster, then copy the pinned tensors to the GPU once per epoch at the start of training.
  • CMove the tokenisation and feature-scaling steps onto the GPU using a GPU-native library such as cuDF or RAPIDS, keeping the tensors resident on the GPU throughout the pipeline until training is complete. Correct
  • DPre-tokenise the entire dataset on the CPU, serialise the resulting integer tensors to disk as memory-mapped files, then load each batch lazily from disk directly into GPU memory at training time.
Understand how keeping data on the GPU throughout preprocessing eliminates host-to-device transfer overhead in ML input pipelines. Host-to-device transfer over the PCIe bus is a well-known throughput bottleneck in GPU-accelerated ML pipelines. When preprocessing steps such as tokenisation and normalisation run on the CPU, each batch must be copied from host DRAM to GPU VRAM before the forward pass can begin, leaving the GPU idle between bursts of computation. GPU-native preprocessing libraries (for example, cuDF from the RAPIDS suite) allow tokenisation, encoding, and scaling to execute as GPU kernels, so the tensors remain resident in device memory from the moment the raw data is ingested. Because no transfer occurs between preprocessing and training, the pipeline stages overlap naturally and GPU utilisation remains high throughout. Pinned memory and prefetching are complementary optimisations that reduce per-transfer latency but do not eliminate transfers; they are therefore inferior to full on-device preprocessing for removing the bottleneck.

Why A is wrong: Adding CPU workers reduces the time data sits in a queue, but every batch still crosses the PCIe bus from host to device memory. The root cause - host-to-device transfer overhead - is not addressed; the GPU can still stall between transfers, and more frequent small transfers may worsen PCIe congestion rather than improve it.

Why B is wrong: Pinned memory does accelerate individual host-to-device copies and enables asynchronous transfer, which is a genuine optimisation. However, it does not eliminate transfers; the data still moves across the PCIe bus. Preprocessing on the CPU and then copying once per epoch still leaves the GPU idle during tokenisation, so the fundamental pipeline stall persists.

Why C is correct: Performing tokenisation and scaling directly on the GPU with a library like cuDF means the data never leaves device memory between preprocessing and training. This eliminates the host-to-device transfers that stall the pipeline, and the GPU remains active across all stages rather than waiting for the CPU to finish and then copy data across the PCIe bus.

Why D is wrong: Memory-mapped files can reduce RAM pressure and allow partial loading, which is attractive for very large datasets. However, loading from disk to GPU still requires a host-staging step; data travels through the CPU memory hierarchy before reaching the GPU. Disk I/O latency is far higher than DRAM-to-GPU transfers, so this approach introduces a new bottleneck rather than removing the existing one.

See more NCA-GENL practice questions, answers explained.

More in this domain

Back to all Data Analysis and Visualization objectives, or the NCA-GENL cert hub.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.