A data scientist builds a pipeline that reads raw sensor data with cuDF, runs feature engineering with cuDF operations, and then fits a cuML Random Forest. After profiling, they discover most wall-clock time is spent on memory transfers rather than computation. Which single architectural change is most likely to resolve this?
- AReplace cuDF with pandas for the feature-engineering stage so the data lives in familiar host memory before being passed to cuML
- BConvert the cuDF DataFrame to a NumPy array immediately after ingestion so downstream libraries can share a common in-memory format
- CWrite the feature-engineered data to a Parquet file on NVMe storage between each stage so each stage can reload only the columns it needs
- DPass the cuDF DataFrame directly into cuML using the shared device-memory pointer, keeping the data on the GPU for every stage of the pipeline Correct
Why A is wrong: Moving to pandas forces the data back to host memory, which introduces exactly the host-to-device transfer that cuML would then need to reverse. This increases the number of transfers rather than eliminating them.
Why B is wrong: Exporting to NumPy moves the data off the GPU to host memory. Any subsequent GPU operation would need to copy it back, adding at least two transfers where the goal is zero.
Why C is wrong: Materialising intermediate results to disk introduces I/O latency far greater than in-memory transfer costs and still requires re-reading and re-loading data to the GPU for subsequent stages.
Why D is correct: cuDF and cuML both operate on GPU memory and exchange data via the CUDA array interface or cuDF's native integration, so no host-device transfer is required between stages. This is the defining benefit of the RAPIDS end-to-end workflow.