Engineering, production, governance, cost, communication · insight-1273draft — in review

Choose the drift reference window deliberately, and keep one pinned to the training data

Situation
A drift monitor compares each day with the previous seven days, and the model's inputs have been shifting slowly for three months.
Action
Run two comparisons: current versus the training snapshot of the live model, for cumulative distance, and current versus a short rolling window, for sudden breaks; alert on each with its own meaning.
Reason
The reference defines what 'different' means; a moving one detects only fast change and a fixed one detects accumulated change.
Boundary
Feature and score monitoring for models that are retrained rarely or run for long periods.

Symptom or risk

A rolling reference absorbs gradual change, so the day-over-day statistic stays small while the model operates far from the data it was trained on.

Models retrained daily on a recent window are already close to a rolling reference.

A pinned reference produces constant alerts in seasonal businesses unless it covers a full cycle.

Choosing a training reference that includes a one-off event, so ordinary days look drifted.