LLMs, GenAI, RAG, agents, evaluation · insight-0503

Supply volatile facts at request time because parametric knowledge is frozen at training

Situation
Agents ask a chat model about this season's baggage allowance or a fare rule that changed last month.
Action
Classify every fact the feature relies on as stable or volatile; deliver volatile facts (fares, policies, schedules) through retrieval or tool lookups that carry an effective date, and include recently changed rules in the eval set.
Reason
Knowledge stored in weights reflects the training corpus and can only be updated by further training, whereas an external index can be refreshed and cited.
Boundary
Prices, policies, inventories, regulations, product catalogues.

Symptom or risk

The model answers fluently with the rule as it appeared in its training data, and nothing in the output marks the answer as outdated.

Stable general knowledge and language skills do not need retrieval.

Retrieval adds latency, indexing work and a new failure mode (the wrong document is retrieved).

Asking the model for its own knowledge cutoff and treating the reply as a guarantee of what it knows.

Lewis et al. note that updating the world knowledge stored in a pretrained model's parameters and providing provenance remain open problems, and combine parametric memory with a replaceable non-parametric index.
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks