NCA-GENL - Core Machine Learning and AI Knowledge (30% of the exam) - Section 1.4
Use diverse machine learning algorithms including XGBoost.
Compare supervised learning algorithms - decision trees, random forests, and gradient-boosted trees - and recognise the scenarios where each excels. Apply XGBoost for structured tabular data, and understand why its regularisation and tree-pruning approach frequently outperforms simpler ensembles on classification and regression benchmarks.
XGBoost
Practice question for this objective
lock_openFree sampleCore Machine Learning and AI Knowledgemedium
Two practitioners compare modelling approaches for a structured tabular dataset of 80,000 rows with mixed numeric and ordinal features, moderate noise, and clear nonlinear feature interactions. They have limited time for hyperparameter tuning and want strong baseline accuracy with low engineering effort. Why is XGBoost commonly the strong default choice over a deep neural network here?
AXGBoost is guaranteed to outperform any neural network on every dataset because boosting always reaches a lower training loss.
BGradient-boosted trees capture nonlinear feature interactions on tabular data with little tuning and no feature scaling, often matching or beating neural networks on mid-sized structured datasets.check_circle Correct
CXGBoost requires the input features to be standardised to zero mean and unit variance, which makes it more robust than neural networks that skip scaling.
DXGBoost trains without any loss function or gradients, so it avoids the optimisation difficulties that slow neural-network training on tabular data.
Understand why gradient-boosted trees are a strong low-effort default for mid-sized structured tabular data. XGBoost partitions on raw feature thresholds, so it captures nonlinear interactions without scaling and is robust with limited tuning, which makes it competitive with or superior to neural networks on mid-sized tabular problems.
Why A is wrong: Boosting can drive training loss low, making this superficially appealing, but the universal guarantee is false and lower training loss does not imply better generalisation, so the claim overreaches.
Why B is correct: Boosted trees split on raw feature thresholds, so they model interactions without scaling and reach strong accuracy with modest tuning, which is precisely why they are a robust tabular default over neural networks at this scale.
Why C is wrong: The premise is inverted: tree splits are scale-invariant, so XGBoost does not need standardisation, and the comparison wrongly attributes a scaling requirement to it that actually applies more to neural networks.
Why D is wrong: It is tempting to contrast trees with gradient-based nets, but XGBoost is itself a gradient-boosting method that uses gradients and Hessians of a loss, so the stated reason is factually wrong.
Exam traps in Core Machine Learning and AI Knowledge
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-GENL bank for this domain.
Raw image pixels, because boosted trees learn translation-invariant filters more efficiently than convolutional layers and need fewer labelled examples.
errorWhy it is wrong: Trees do not learn translation-invariant filters; convolutional networks are the standard for images. The option is tempting because XGBoost is data-efficient, but it lacks the spatial inductive bias that images require.
It increases the depth of each tree so that more interactions between features can be captured, raising training accuracy on complex datasets.
errorWhy it is wrong: This is the opposite of regularisation's effect. It is tempting because deeper trees do capture more interactions, but stronger regularisation discourages complexity rather than encouraging deeper trees.
Increase max_depth, because a smaller eta requires deeper individual trees to compensate for the reduced step size of each round.
errorWhy it is wrong: Deeper trees do add capacity and may seem to offset weaker steps, but max_depth and eta are independent levers and deepening trees raises variance rather than addressing the missing rounds, so it does not target the actual cause.