A demand-forecasting model on Vertex AI is losing accuracy in production, but the team has no fresh ground-truth labels for recent traffic because actual demand is only confirmed weeks later. They still need an early monitoring signal that the model's behaviour has changed, computed purely from the model's outputs on live requests rather than from any label. Which Model Monitoring signal can the team rely on under this constraint?
- AConcept-drift detection that measures the changing statistical relationship between the input features and the true labels in production.
- BPrediction drift detection that compares the distribution of the model's output predictions over time against an earlier baseline of its outputs. Correct
- CA live accuracy metric computed each hour by comparing every prediction against its confirmed actual outcome.
- DFeature attribution drift detection that recomputes per-feature attribution importances against the training baseline for each request.
Why A is wrong: Concept drift is genuinely a cause of accuracy loss, which makes it tempting, but quantifying it requires the true labels the team does not yet have, so it cannot serve as the label-free early signal needed here.
Why B is correct: Prediction drift watches only the model's output distribution against an earlier baseline, requiring no labels, so it can surface a behavioural change early while ground truth is still pending.
Why C is wrong: An hourly accuracy metric is the most direct signal in principle, but it depends on confirmed outcomes that arrive weeks late here, so it cannot be computed in time to give an early warning.
Why D is wrong: Attribution drift is label-free and useful, which makes it a near-miss, but it tracks shifts in feature importance rather than the output behaviour the team specifically wants to watch, and is heavier to compute per request.