LLMs, GenAI, RAG, agents, evaluation · insight-0550
Evaluate the exact artefact you will serve, not the training checkpoint
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A LoRA adapter scores well during training, then is merged into the base weights and quantized for serving without re-evaluation.
Action
Run the frozen eval set on the final serving artefact (merged or adapter-loaded, quantized or not, with the serving template and decoding settings) and gate release on that run.
Reason
PEFT tooling supports both merging adapters into base weights and loading them separately, and each serving path, combined with quantization, is a different numerical model.
Boundary
Any fine-tuned model moving from training to serving.
Symptom or risk
Merging, quantization or a different chat template changes outputs, and the served model is worse than the evaluated one.
Exceptions
None; a short smoke eval is the minimum when time is tight.
Trade-offs
An extra evaluation run per release.
Pitfalls
Reusing training-time metrics in the release note.
Lessons and concepts
Claims and their sources
The Hugging Face PEFT documentation covers merging LoRA adapter weights into the base model as well as loading adapters separately.