NCA-GENL - Core Machine Learning and AI Knowledge (30% of the exam) - Section 1.1
Understand fundamental techniques and tools for training deep learning models.
Describe the core components of a deep learning training pipeline, including forward and backward passes, loss functions, and gradient descent optimisers. Recognise how choices such as learning rate, batch size, and regularisation affect convergence and generalisation.
Deep learningTraining
Practice question for this objective
lock_openFree sampleCore Machine Learning and AI Knowledgemedium
During training, a deep learning model's validation loss stops improving after epoch 12 while training loss continues to fall. Which single adjustment is most likely to correct this behaviour?
AIncrease the learning rate to allow the optimiser to escape the current minimum faster.
BSwitch the activation function from ReLU to sigmoid in all hidden layers.
CAdd dropout layers or increase the dropout probability to regularise the network.check_circle Correct
DDouble the batch size so each gradient update is computed from more samples.
Identify regularisation techniques that prevent overfitting when validation loss diverges from training loss during deep learning model training. When training loss falls but validation loss plateaus or rises, the model is overfitting: it is memorising training data rather than learning generalisable features. Dropout is a standard regularisation technique that randomly zeros out neuron activations with a given probability at each forward pass during training, forcing the network to learn redundant representations and reducing reliance on specific neurons. This directly reduces overfitting without changing the optimiser or architecture in ways that would harm convergence.
Why A is wrong: Tempting because a low learning rate can cause slow convergence, but raising it here would worsen overfitting by letting the model fit training noise even more aggressively, widening the gap between training and validation loss.
Why B is wrong: Tempting because activation functions affect gradient flow, but sigmoid is prone to vanishing gradients in deep networks and does not address overfitting. Replacing ReLU with sigmoid would likely degrade convergence without reducing the training-validation gap.
Why C is correct: Dropout randomly deactivates units during training, preventing co-adaptation and reducing overfitting. When validation loss plateaus while training loss keeps falling, the model is memorising training data, and regularisation via dropout directly addresses that cause.
Why D is wrong: Tempting because larger batches can smooth gradient estimates, but doubling batch size without other changes often sharpens minima and can worsen generalisation. It does not introduce the regularisation effect needed to close the training-validation loss gap.
Exam traps in Core Machine Learning and AI Knowledge
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENL bank for this domain.
The batch size sets the learning rate used for each weight update, so larger batches automatically take larger optimisation steps.
errorWhy it is wrong: Tempting because batch size and learning rate interact in practice, but batch size does not set the learning rate; the two are separate hyperparameters tuned independently.
It removes the need for regularisation entirely, because shrinking each tree's contribution makes the leaf-weight and leaf-count penalties redundant.
errorWhy it is wrong: Shrinkage and the regularisation penalties address overfitting through different mechanisms and are typically used together. The claim is tempting because both fight overfitting, but neither makes the other redundant.
Increasing the learning rate by a factor of four so the size-16 updates cover the same ground as a size-64 update.
errorWhy it is wrong: Scaling the learning rate with batch size is a real heuristic, which makes this tempting, but it changes the optimisation dynamics rather than recreating the size-64 gradient estimate and does nothing to address the memory constraint the team actually faces.