A training run shows that validation loss stops improving after epoch 10 while training loss continues to fall steeply. Which technique directly addresses this gap by penalising large weight magnitudes in the loss function itself?
- AInserting batch normalisation layers after each linear block, which normalises activations across the mini-batch during the forward pass.
- BApplying L2 weight decay, which adds the sum of squared weights to the training loss, discouraging large parameter values. Correct
- CSwitching from a cosine annealing schedule to a step decay schedule so the learning rate drops more sharply every few epochs.
- DReplacing stochastic gradient descent with Adam, which adapts per-parameter learning rates using first and second moment estimates.
Why A is wrong: Batch normalisation stabilises training and can reduce internal covariate shift, but it does not add a penalty on weight magnitudes to the loss function itself, so it addresses a different cause of the training-validation gap.
Why B is correct: L2 regularisation adds a penalty proportional to the squared magnitude of weights directly to the loss, reducing overfitting by keeping weights small and the model less sensitive to training-set noise.
Why C is wrong: Learning-rate schedules control how step size evolves during optimisation, which can help convergence but does not add a regularisation penalty on weights to the loss and will not directly close the gap described.
Why D is wrong: Adam can converge faster than plain SGD and may generalise differently, but choosing an optimiser does not intrinsically penalise weight magnitude in the loss function, so it does not directly regularise in the way described.