Feature engineering and preprocessing · insight-1129
Pair a training baseline with a same-season recent window when monitoring feature drift
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Data scientist
- Industries
- aviation, retail
Summary: situation, action, reason, boundary
Situation
A monitor compares production features with the training sample only, and the business is seasonal.
Action
Compare each feature against two references: the training baseline for cumulative change and a recent window of the same season or weekday type for abrupt change; alert on the second, review the first on a schedule.
Reason
Seasonality is expected variation, so it must be removed from the comparison before a change counts as drift; a fixed baseline cannot do that.
Boundary
Demand, traffic and sales features with weekly or annual cycles.
Symptom or risk
Every December looks like drift against a summer-heavy baseline, while a slow monthly slide against the training period is accepted once the alert is muted.
Exceptions
Stable processes with no seasonality, and models retrained often on rolling windows.
Trade-offs
A same-season window needs at least a year of history and misses a change that starts during the first window.
Pitfalls
Choosing a reference window that already contains an incident.