Feature engineering and preprocessing · insight-1124
Ship a changed feature definition under a new version instead of editing it in place
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- ML engineer, Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
An owner improves the calculation of a shared feature, such as a cleaner delay measure, and wants the fix applied under the same name.
Action
Publish the corrected definition as a new versioned feature, migrate consumers one by one with a shadow comparison, and retire the old version only after each consumer is retrained and validated.
Reason
A consuming model has adapted to the old signal's quirks; a silent update to it can have arbitrary effects that are costly to diagnose.
Boundary
Shared feature tables, feature stores and signals produced by another team or model.
Symptom or risk
Consumer models fitted to the old behaviour change silently, even for a correct improvement, and the cause is hard to find.
Exceptions
A bug fix whose old values were corrupt, where pinning the wrong values has no value.
Trade-offs
Versioning costs storage, staleness and effort to maintain multiple copies.
Pitfalls
Versioning the code but not the stored historical values, so training sets rebuilt later mix both definitions.
Lessons and concepts
Claims and their sources
Sculley et al. describe unstable data dependencies, where an input signal changes behaviour over time, and name a versioned copy of the signal as a common mitigation with its own staleness and maintenance costs.