A data engineer is building an ETL workflow that extracts a 30 GB Parquet file, applies several column-wise transformations, and loads the result into a downstream feature store. She wants the extract, transform, and load steps to all run on the GPU so the data is read straight into device memory and stays there until written back out. Which approach keeps the workflow GPU-resident end to end and avoids repeated host-device copies?
- ARead the Parquet file with pandas, convert the resulting frame to cuDF for the transformations, then convert the result back to pandas before writing the Parquet output.
- BRead the Parquet file with cuDF, move the frame to host memory with a to_pandas conversion for the transformations, then move it back to the GPU only for the final write.
- CRead the Parquet file with cuDF, then run the transformations through a NumPy code path by extracting the underlying arrays to host memory, and write the output with cuDF.
- DRead the Parquet file with the cuDF reader so the data lands directly in GPU memory, perform the transformations with cuDF operations, then write back with the cuDF Parquet writer, never converting to a pandas DataFrame in between. Correct
Why A is wrong: This is tempting because cuDF and pandas share an API, but reading with pandas places data on the host and each conversion forces a host-device copy, so the workflow is not GPU-resident across the extract and load steps.
Why B is wrong: Doing the transformations on the host throws away the GPU acceleration during the most compute-heavy stage and adds two transfers, defeating the goal of keeping the data on the device throughout.
Why C is wrong: NumPy operates on host arrays, so pulling the columns out to NumPy copies them off the GPU for the transform stage, reintroducing exactly the host-device movement the engineer is trying to eliminate.
Why D is correct: cuDF reads Parquet straight into device memory, its vectorised operations execute on the GPU, and its writer serialises from device memory, so the data never round-trips to host between extract, transform, and load.