Introductory MLOps Practices

A data scientist trains a cuML RandomForestClassifier and a preceding cuDF-based preprocessing pipeline (label encoding and standard scaling). She serialises only the cuML model using joblib and deploys it. At inference time, raw production data is passed directly to the loaded model. What is the most likely failure mode?

• AThe joblib file becomes unreadable because cuML models must be serialised with pickle, not joblib
• BRaw features arrive on a different numeric scale and encoding than the model was trained on, causing degraded or incorrect predictions because the saved preprocessing steps are absentcheck_circle Correct
• CInference will silently succeed but return predictions for the wrong classes because joblib does not preserve feature names
• DThe model will raise an exception on load because cuML RandomForestClassifier stores GPU device state that is incompatible across environments

Why A is wrong: joblib and pickle are both compatible with cuML models; joblib is in fact the recommended approach for scikit-learn-style estimators and works correctly here, so the serialisation format is not the source of failure

Why B is correct: Saving the model without the fitted preprocessing pipeline means the transformations learned on training data (scaler means/variances, label mappings) are not applied at inference, so the model receives inputs drawn from a different distribution than it expects

Why C is wrong: Feature names are a separate concern; the more fundamental issue is that the numeric scale and encoding of input features will not match what the model learned during training, so predictions will be incorrect for a substantive statistical reason, not a metadata one

Why D is wrong: cuML estimators serialise the fitted parameters, not live GPU device handles; they can be loaded in any compatible cuML environment without an exception related to device state, so this is not the primary failure mode described

A deployed cuML credit-scoring model has served stable predictions for a year. The input feature distributions have not changed, but over the last two months the model's measured precision and recall on freshly arriving labelled outcomes have both fallen steadily. The monitoring team wants to name what they are observing so they can choose the right remedy. Which type of degradation does this pattern most directly indicate?

• AData drift, because the statistical properties of the incoming input features have shifted away from the training distribution.
• BConcept drift, because the relationship between the input features and the target outcome has changed while the inputs remain stable.check_circle Correct
• CTraining-serving skew, because the preprocessing applied at inference differs from the preprocessing used during training.
• DSampling bias, because the labelled outcomes now arriving are no longer representative of the population the model scores.

Why A is wrong: Data drift is tempting because falling metrics often accompany shifting inputs, but the scenario states the input feature distributions are unchanged, so the cause cannot be a shift in the inputs themselves.

Why B is correct: Stable inputs with falling predictive performance is the defining signature of concept drift: the mapping from features to label has changed, so the same inputs now imply different outcomes and the learned function no longer fits.

Why C is wrong: Skew is a real failure mode, but it produces degradation from the very first day of serving, not a gradual decline after a year of stable operation, so it does not match the described timeline.

Why D is wrong: Unrepresentative evaluation data can distort metrics, but the scenario gives no evidence the label sample is skewed, and a genuine real-world shift in feature-to-target behaviour is the more direct explanation of steadily falling precision and recall.

A churn model scores customers in real time, but the true churn label for each customer is only confirmed roughly 90 days after scoring. The monitoring team cannot wait three months to learn that performance has dropped, so they want an early-warning signal that does not depend on ground-truth labels. Which monitored quantity provides the earliest label-free indication that the model may be degrading?

• AThe rolling F1 score of the model computed on the most recent batch of confirmed churn outcomes.
• BThe mean prediction latency per scoring request measured at the serving endpoint.
• CThe total count of scoring requests processed by the endpoint each hour.
• DThe population stability index comparing the live input feature distributions against the training reference distribution.check_circle Correct

Why A is wrong: An F1 score is the most direct performance measure, which makes it tempting, but it requires confirmed labels and therefore cannot be computed until the 90-day delay has passed, defeating the early-warning goal.

Why B is wrong: Latency is a genuine operational metric to monitor, but it reflects serving speed, not predictive quality, so it would stay healthy even as the model's accuracy silently decays.

Why C is wrong: Request volume detects traffic anomalies and outages, which is useful, but it carries no information about whether the inputs or the feature-target relationship have shifted, so it cannot serve as a drift early warning.

Why D is correct: The population stability index quantifies how far live feature distributions have moved from the training reference using only inputs, so it flags potential data drift immediately, long before the delayed labels arrive to confirm any performance impact.

An engineer wants to benchmark only the GPU-accelerated feature-engineering step inside a longer pipeline that also reads a Parquet file from cloud storage and writes results back. The goal is to learn how fast the transformation itself runs. How should the benchmark be scoped to isolate that component?

• ATime the entire pipeline end to end including the cloud read and write, then divide the total by the number of pipeline stages to attribute an equal share to feature engineering.
• BPlace the timing boundaries around the feature-engineering step alone, with the input already resident in GPU memory and the output left on the device, so neither cloud reads nor writes are counted in the measured interval.check_circle Correct
• CTime the read plus the feature-engineering step together but exclude the write, since reads are unavoidable input preparation that should count towards the transformation.
• DRun the transformation once with the network disconnected and report whatever time results, since removing the network guarantees only compute is measured.

Why A is wrong: Tempting because dividing a total feels like a fair allocation, but pipeline stages do not take equal time, so an even split misattributes the slow cloud I/O and gives a meaningless figure for the transformation.

Why B is correct: Correct. Isolating a component means the timed interval brackets only that component: with the input already on the GPU and the output kept on the device, the recorded time reflects the transformation itself rather than network and disk costs that belong to other stages.

Why C is wrong: Tempting because the read does precede the transformation, but folding a variable cloud-read time into the measurement contaminates the figure with network latency that has nothing to do with the transformation's speed.

Why D is wrong: Tempting because disabling the network removes cloud I/O, but a single run still includes any local disk reads and warm-up costs, and one untimed-boundary measurement does not cleanly bracket the transformation step itself.

A team uses an experiment tracker to log each training run. They want future colleagues to reproduce any model exactly. Which practice most directly supports that goal?

• ALogging only the final validation loss for each run to keep the tracker lightweight
• BLinking each stored artefact to the run record that produced it, including code version and dataset versioncheck_circle Correct
• CStoring model weights in a shared folder without recording which run produced them
• DTagging each artefact file with the current calendar date as its version identifier

Why A is wrong: Recording only the final loss omits the information needed to identify which run produced a given model. Without linking the run record to the stored artefacts, a colleague cannot determine which code version, dataset version, or hyperparameters generated the result.

Why B is correct: Reproducibility requires a complete, queryable link from every artefact - model weights, preprocessors, and metrics - back to the exact run that generated it, including the code commit and the dataset version used. This linkage lets any team member reconstruct the full pipeline from a single run identifier.

Why C is wrong: Saving weights without run linkage breaks the audit trail. If multiple runs produce files with similar names, colleagues cannot determine which configuration generated a specific model, making reproducibility impossible in practice.

Why D is wrong: A calendar date is a weak version identifier: multiple runs on the same day collide, and the date carries no information about the code or data that was used. Proper artefact versioning requires linking to a specific run record rather than an imprecise timestamp.