Two practitioners compare modelling approaches for a structured tabular dataset of 80,000 rows with mixed numeric and ordinal features, moderate noise, and clear nonlinear feature interactions. They have limited time for hyperparameter tuning and want strong baseline accuracy with low engineering effort. Why is XGBoost commonly the strong default choice over a deep neural network here?
- AXGBoost is guaranteed to outperform any neural network on every dataset because boosting always reaches a lower training loss.
- BGradient-boosted trees capture nonlinear feature interactions on tabular data with little tuning and no feature scaling, often matching or beating neural networks on mid-sized structured datasets. Correct
- CXGBoost requires the input features to be standardised to zero mean and unit variance, which makes it more robust than neural networks that skip scaling.
- DXGBoost trains without any loss function or gradients, so it avoids the optimisation difficulties that slow neural-network training on tabular data.
Why A is wrong: Boosting can drive training loss low, making this superficially appealing, but the universal guarantee is false and lower training loss does not imply better generalisation, so the claim overreaches.
Why B is correct: Boosted trees split on raw feature thresholds, so they model interactions without scaling and reach strong accuracy with modest tuning, which is precisely why they are a robust tabular default over neural networks at this scale.
Why C is wrong: The premise is inverted: tree splits are scale-invariant, so XGBoost does not need standardisation, and the comparison wrongly attributes a scaling requirement to it that actually applies more to neural networks.
Why D is wrong: It is tempting to contrast trees with gradient-based nets, but XGBoost is itself a gradient-boosting method that uses gradients and Hessians of a loss, so the stated reason is factually wrong.