LLMs, GenAI, RAG, agents, evaluation · insight-1558
Re-verify remembered facts against the system of record before acting on them
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation
Summary: situation, action, reason, boundary
Situation
An assistant recalls from memory that a passenger prefers a window seat, or that a rebooking is pending, and uses it to act.
Action
Store each memory with its source and timestamp, mark facts that can change as volatile, and before any write that depends on one, read the current value from the system of record and prefer it over memory.
Reason
Memory holds a snapshot of the past, while the system of record holds the present, so write decisions should rest on the second.
Boundary
Assistants with long-term memory that also have write tools.
Symptom or risk
The remembered fact is stale, so the agent changes a booking based on a state that no longer holds.
Exceptions
Stable preferences, such as language, may be used without a re-read.
Trade-offs
Extra reads add latency and load.
Pitfalls
Letting the agent quote memory as current fact in its answer.