ExamworthyExamworthy
Open the appStart free

NCA-ADS - Introductory MLOps Practices - Section 6.4

Monitor production models for drift.

Monitor production models for data drift and concept drift by comparing incoming feature distributions and prediction quality against baselines established at training time. Determine when drift is severe enough to trigger retraining rather than a simple threshold adjustment.

Data drift

Practice question for this objective

Free sampleIntroductory MLOps Practicesmedium

A fraud-detection model was trained on transaction data from 2023. In 2025, the input feature distributions remain stable, but the relationship between the features and fraudulent behaviour has shifted because attackers have changed their tactics. Which type of drift best describes this situation, and what is the primary trigger for retraining?

  • AData drift, because any change in the production environment that degrades accuracy is by definition a shift in input data.
  • BConcept drift, because the statistical relationship between the input features and the target label has changed even though the input distributions are unchanged. Correct
  • CCovariate shift, because the model was trained on a sample that did not represent the full population of future transactions.
  • DModel degradation caused by infrastructure change, because production latency or hardware differences alter effective model outputs over time.
Distinguish concept drift from data drift by identifying which statistical distribution has changed in a given production scenario. Concept drift occurs when P(y|X) changes - the same input feature vector now corresponds to a different expected output - while the input distribution P(X) remains stable. Data drift (including covariate shift) occurs when P(X) changes. Here, attacker tactic changes alter the fraud-to-feature mapping without changing raw feature distributions, making this a clear concept drift case. Detecting concept drift typically requires monitoring model performance metrics (precision, recall, F1) against labelled ground truth, whereas data drift can be detected by comparing input feature distributions alone using tests such as the Kolmogorov-Smirnov test or Population Stability Index.

Why A is wrong: Data drift specifically refers to changes in the statistical distribution of input features, not changes in how features relate to the target label. Stable input distributions with a shifting feature-to-label relationship is a different phenomenon entirely.

Why B is correct: Concept drift describes a change in the conditional distribution P(y|X) - the mapping from features to labels - independent of any change in the marginal input distribution P(X). Attackers adopting new tactics shifts this mapping, which is exactly why the model's learned decision boundary no longer reflects reality and retraining on recent labelled data is required.

Why C is wrong: Covariate shift is a form of data drift where training and production input distributions differ. The scenario explicitly states that input feature distributions have remained stable, ruling out this category as the root cause.

Why D is wrong: Infrastructure-induced degradation relates to serving environment differences, not to a change in the underlying data-generating process. The scenario describes a behavioural shift by fraudsters, which is a data-generating-process phenomenon, not an infrastructure phenomenon.

See more NCA-ADS practice questions, answers explained.

More in this domain

Back to all Introductory MLOps Practices objectives, or the NCA-ADS cert hub.

Examworthy is not affiliated with or endorsed by NVIDIA. Original, blueprint-aligned practice material only.