Engineering, production, governance, cost, communication · insight-1258
Key cached predictions by entity, model version and feature version
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer
- Industries
- e-commerce, aviation, saas
Summary: situation, action, reason, boundary
Situation
A service caches scores per customer for 24 hours to save compute, and a new model version ships at noon.
Action
Include the model version and feature-definition version in the cache key or invalidate the cache on release, set the time-to-live from how fast the inputs change, and log whether each response was a cache hit.
Reason
A cached value is only valid for the code and inputs that produced it; without versions in the key, a release does not take effect where the cache is warm.
Boundary
Precomputed score tables, per-entity caches, recommendation and pricing lookups.
Symptom or risk
Half of the traffic still gets scores from the old model, so an A/B comparison, a canary or a rollback does not behave as designed, and the cache hides the regression for hours.
Exceptions
Where a score is deliberately frozen for a period, such as a quoted price, store it as a decision record and not as a cache.
Trade-offs
Versioned keys empty the cache at each release and cause a compute spike; warm it gradually.
Pitfalls
Measuring canary quality on traffic that is mostly cache hits from the incumbent.