LLMs, GenAI, RAG, agents, evaluation · insight-1542
Key answer caches on index version or document version, so a document update invalidates stale answers
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A response cache keyed on the normalised question makes a popular FAQ cheap, and the underlying policy document then changes.
Action
Include the index snapshot id or the versions of the documents used in the cache key, or store those ids with each entry and invalidate on change events, and clear entries when documents are deleted.
Reason
A cached answer is a derived copy of the retrieved evidence; its validity depends on that evidence, not on the question string.
Boundary
Exact-match and semantic response caches in RAG applications over changing sources.
Symptom or risk
The cache keeps serving the answer generated from the old text until its time-to-live expires, bypassing every freshness and deletion control in the index.
Exceptions
Answers that do not depend on retrieved content, such as a greeting or a static help message.
Trade-offs
More invalidation lowers the hit rate and the savings.
Pitfalls
Versioning documents but not the prompt or model, so a prompt change still serves old answers.