NCA-ADS - Machine Learning With RAPIDS (16% of the exam) - Section 2.4
Tune hyperparameters.
Apply grid search, random search, and Bayesian optimisation to find hyperparameter configurations that improve model performance. Understand why tuning must be performed on validation data only, keeping the test set unseen until final evaluation.
Practice question for this objective
lock_openFree sampleMachine Learning With RAPIDSmedium
A data scientist is tuning a cuML RandomForestClassifier and wants to explore a large search space across six hyperparameters efficiently. She has GPU capacity that makes training each candidate model fast. Which search strategy best exploits that advantage while covering the space more effectively than an exhaustive approach?
ABayesian optimisation, which builds a probabilistic surrogate model to direct subsequent trials toward promising hyperparameter regionscheck_circle Correct
BGrid search, which tests every combination of the predefined values
CRandom search, which samples hyperparameter combinations from the defined distributions
DManual search, where the data scientist adjusts one hyperparameter at a time based on intuition
Compare grid, random, and Bayesian hyperparameter search strategies and identify when Bayesian optimisation is most appropriate. Bayesian optimisation maintains a probabilistic surrogate (typically a Gaussian process or tree-based model) of the objective function. After each trial it updates the surrogate and uses an acquisition function to select the next candidate, steering the search toward high-potential regions. This directed exploration finds strong configurations in far fewer trials than grid or random search. GPU acceleration makes each individual trial cheap, which amplifies the advantage of running more intelligent searches rather than exhaustive ones.
Why A is correct: Bayesian optimisation uses the results of previous trials to update a surrogate model and select the next candidates intelligently, converging on good configurations with far fewer evaluations than grid or random search - a strategy that becomes practical when GPU acceleration makes each trial fast.
Why B is wrong: Grid search is exhaustive and scales exponentially with the number of hyperparameters, making it impractical for a six-parameter space even with fast GPU training.
Why C is wrong: Random search samples the space without learning from prior trials, so it is more efficient than grid search but still does not direct subsequent trials toward promising regions.
Why D is wrong: Manual one-at-a-time tuning ignores interactions between hyperparameters and relies entirely on the practitioner's intuition, making it the least systematic and least reproducible strategy available.
Answers that look right on this material and are not. Each one is a distractor from a different question in the NCA-ADS bank for this domain.
Random search guarantees it will test the single globally optimal combination because it samples the entire space uniformly at random.
errorWhy it is wrong: Random sampling never guarantees hitting the global optimum within a finite budget; the appeal of completeness is illusory, so this overstates what random search can promise.
It evaluates every point on a fixed lattice of predefined values, guaranteeing the global optimum is among the points tested.
errorWhy it is wrong: This describes grid search, which is fixed in advance and never adapts to results; it is a familiar default, but it cannot steer later trials using earlier outcomes.
Random folding under-trains the model because each fold withholds 20 per cent of the rows; using ten folds so more data is seen per fit will close the gap with future days.
errorWhy it is wrong: The amount of data withheld per fold is not the issue and ten folds still shuffle time order, so the model would continue training on future-dated rows; this treats a leakage problem as a data-volume problem and so cannot close the deployment gap.