A deployed regression model predicts daily energy demand, and the actual demand for any given day is only known once metered readings settle two days later. A monitoring engineer wants a single dashboard panel that tracks how the model's prediction error evolves day by day, accounting correctly for the fact that today's error cannot be computed until its ground truth lands. Which approach correctly tracks live prediction error under this delay?
- APlot the model's error each day against the predictions made that same day before any ground truth has arrived.
- BTrack only the distribution of the model's raw output values and treat any change in their spread as the error metric.
- CCompute error only at the end of each month once all labels are final, and leave the panel blank until then.
- DJoin each prediction to its ground truth once the metered reading settles, then plot the error on a rolling window aligned to the prediction date. Correct
Why A is wrong: Charting error on the day of prediction is tempting for immediacy, but the true demand is not yet known, so any error computed then is undefined or fabricated and cannot reflect real performance.
Why B is wrong: Watching the spread of outputs is a reasonable drift check, but the spread of predictions is not the prediction error; the model could be confidently wrong with a perfectly stable output distribution.
Why C is wrong: Waiting for a full month of settled labels gives accurate numbers but no timely signal, so a sustained mid-month degradation would stay invisible far longer than the two-day delay actually requires.
Why D is correct: Joining each prediction to its later-arriving ground truth and plotting a rolling error aligned to the prediction date correctly handles the two-day delay, giving an honest, continuously updating view of real prediction error as labels mature.