During a XGBoost (GPU-accelerated) training run you plot training loss and validation loss against the number of boosting rounds. Training loss falls steadily to near zero, but validation loss decreases for the first 40 rounds, reaches a minimum, then rises for the remaining 60 rounds. What does this curve pattern indicate, and which technique directly exploits this curve to select the best model?
- AThe curve shows underfitting because the training loss is still falling; the correct fix is to raise the learning rate so both curves converge faster.
- BThe curve shows a healthy convergence; the rise in validation loss after round 40 is normal numerical noise and no intervention is needed.
- CThe curve shows overfitting; the correct fix is to add more engineered features so the model can separate the classes more easily in a higher-dimensional space.
- DThe curve shows overfitting beginning at round 40; applying early stopping at the round with minimum validation loss selects the generalisation optimum. Correct
Why A is wrong: Underfitting is indicated by high training loss, not low training loss. Here training loss reaches near zero, showing the model has ample capacity. A rising validation loss after round 40 is a signature of overfitting, and raising the learning rate would accelerate memorisation of training noise rather than preventing it.
Why B is wrong: A monotonically rising validation loss across 60 additional rounds, while training loss continues to fall, is not numerical noise. It is a systematic and reproducible signal that the model is memorising training examples rather than learning transferable patterns, and it warrants intervention.
Why C is wrong: Adding more features increases model capacity and the complexity of available patterns, which tends to worsen overfitting by giving the model additional dimensions to memorise noise. Reducing capacity or applying regularisation is the appropriate direction when validation loss rises while training loss falls.
Why D is correct: When training loss continues to fall while validation loss turns and rises, the model transitions from learning signal to memorising noise - overfitting. The minimum of the validation loss curve marks the point of best generalisation. Early stopping halts training at that round and restores the weights from there, directly using the train-vs-validation curve to prevent the model from drifting into high-variance territory.