ExamworthyExamworthy
Open the appStart free

NCA-GENL - Experimentation - Section 3.1

Undertake projects with modern deep learning frameworks.

Undertake end-to-end deep learning projects using frameworks such as PyTorch and TensorFlow, covering dataset loading, model definition, training loops, checkpointing, and evaluation. Recognise the practical tradeoffs between eager and graph execution modes, and use framework tooling for debugging and performance profiling.

Practice question for this objective

Free sampleExperimentationmedium

A researcher is training a transformer model in PyTorch and notices that GPU memory usage grows steadily with each batch, eventually causing an out-of-memory error. The training loop accumulates predictions for later logging but does not detach them from the computation graph. Which practice directly addresses this memory growth?

  • AEnable mixed precision training so that activations are stored in half precision, halving the memory footprint across the entire computation graph.
  • BConvert accumulated prediction tensors to plain Python scalars or detach them from the computation graph before storing them, so the autograd graph for each batch is freed after the backward pass. Correct
  • CReduce the global learning rate so that fewer gradient updates are performed per epoch, limiting the number of backward passes that accumulate memory across the graph.
  • DSwitch the data loader to use multiple worker processes so that CPU preprocessing is offloaded, reducing the volume of data resident on the GPU at any one time.
Understand how autograd graph retention causes memory growth in PyTorch training loops and apply detachment to prevent it. PyTorch's autograd engine records every operation performed on a tensor that requires gradients, building a directed acyclic graph for the backward pass. If a tensor from the forward pass is stored in a Python collection without being detached, the entire subgraph needed to differentiate it remains in memory. The fix is to call the detach operation on the tensor or extract a scalar value before accumulation, which severs the reference to the computation graph and allows PyTorch to free the associated memory once the backward pass for that batch is complete.

Why A is wrong: Mixed precision reduces the memory used by parameters and activations but does not release references that are intentionally held in Python variables. A tensor kept alive in a list still retains its full computation graph regardless of dtype, so the memory leak continues.

Why B is correct: Retaining tensors that are still attached to the computation graph prevents PyTorch from releasing the intermediate activations built during the forward pass. Detaching or converting to a scalar breaks that reference, allowing the graph to be garbage-collected after each iteration.

Why C is wrong: The learning rate controls the magnitude of parameter updates, not memory retention. Fewer backward passes would reduce training progress without fixing the root cause, which is holding live tensor references outside the training step.

Why D is wrong: Multi-worker data loading improves throughput by parallelising CPU preprocessing but does not affect memory held inside the GPU computation graph. The accumulation problem is caused by retained autograd references, not by the data-loading pipeline.

See more NCA-GENL practice questions, answers explained.

More in this domain

Back to all Experimentation objectives, or the NCA-GENL cert hub.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.