LLMs, GenAI, RAG, agents, evaluation · insight-0540
Do not fine-tune to inject facts that change; retrieve them instead
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation, retail
Summary: situation, action, reason, boundary
Situation
A team fine-tunes a model on this quarter's fare rules so agents can ask about them without retrieval.
Action
Keep changing facts in an index or database queried at request time; reserve fine-tuning for behaviour and format, and test any knowledge-injection idea against retrieval on the same eval set.
Reason
Studies found new knowledge in fine-tuning data is learned slowly and increases the tendency to hallucinate, and retrieval outperformed unsupervised fine-tuning for knowledge injection in the tests reported.
Boundary
Policies, prices, catalogues, regulations, internal documentation.
Symptom or risk
The model learns the rules slowly and incompletely, answers confidently about rules it never absorbed, and every rule change requires another training run.
Exceptions
Stable domain vocabulary and style can benefit from continued pretraining or fine-tuning.
Trade-offs
Retrieval adds infrastructure and its own failure modes.
Pitfalls
Judging the fine-tune only on questions seen in training.
Lessons and concepts
Claims and their sources
Gekhman et al. found fine-tuning examples with new knowledge were learned much more slowly than examples consistent with the model's knowledge, and that learning them increased hallucination.
Ovadia et al. found retrieval-augmented generation consistently outperformed unsupervised fine-tuning for injecting both previously seen and entirely new knowledge.