LLMs, GenAI, RAG, agents, evaluation · insight-0543

Rerun safety and refusal tests after every fine-tune, even on benign data

Situation
A team fine-tunes an aligned model on harmless customer-service transcripts and evaluates only task quality.
Action
Keep a safety and refusal regression suite (harmful requests, policy boundaries, data-leak probes) and make it a release gate for every fine-tuned version alongside the task eval.
Reason
Research showed fine-tuning on a few adversarial examples broke safety guardrails and that fine-tuning on benign, common datasets also degraded safety to a lesser degree.
Boundary
Any fine-tuning or preference tuning of an aligned model.

Symptom or risk

The tuned model now complies with requests the base model refused, and the regression is discovered by users.

None for customer-facing systems; internal prototypes still benefit from a small suite.

Safety suites need upkeep and expert review.

Assuming the base model's safety evaluation transfers to the fine-tuned model.

Qi et al. found that fine-tuning an aligned model on a handful of adversarial examples broke its safety guardrails and that fine-tuning on benign, commonly used datasets also degraded safety.
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!