LLMs, GenAI, RAG, agents, evaluation · insight-0542
Curate a small, consistent fine-tuning set before collecting a large noisy one
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- aviation, telecom
Summary: situation, action, reason, boundary
Situation
A team plans to fine-tune on 200,000 historical agent replies exported without review.
Action
Start with a few hundred to a few thousand reviewed, diverse, policy-correct examples, measure on the eval set, and grow the data only where error analysis shows gaps.
Reason
LIMA reported strong results from fine-tuning a large model on 1,000 curated examples and argued most knowledge comes from pretraining, so fine-tuning data mainly shapes style and format.
Boundary
Instruction tuning for style, format and response behaviour.
Symptom or risk
The model learns outdated policies, inconsistent tone and agents' mistakes along with the good examples.
Exceptions
Tasks that need broad coverage of rare cases, or a smaller base model, may need more data.
Trade-offs
Expert review is slow and expensive per example.
Pitfalls
Reading LIMA as proof that small data teaches new knowledge; the finding is about alignment style on a strong base model.
Lessons and concepts
Claims and their sources
Zhou et al. fine-tuned a 65B LLaMA on 1,000 curated prompt-response pairs without RLHF, reported strong results, and concluded almost all knowledge is learned during pretraining.