Neural networks, deep learning, NLP · insight-0444
Keep a retention test set to measure what fine-tuning made the model forget
- Evidence type
- empirical finding
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- manufacturing
Summary: situation, action, reason, boundary
Situation
You fine-tune a pretrained defect-detection or document model on a new product line and it improves on that line.
Action
Freeze a retention test set from the original task and run it before and after every fine-tune. If it drops, replay a share of original training data, lower the learning rate, or limit which layers change, then compare.
Reason
Networks trained on tasks in sequence tend to overwrite weights that earlier tasks relied on; only a test on the old task shows it.
Boundary
Fine-tuning pretrained models and periodic retraining on recent data.
Symptom or risk
Sequential training can erase earlier capabilities, so accuracy on the original product lines degrades and nobody notices because only the new test set is run.
Exceptions
A model retired from its earlier tasks, where forgetting does not matter.
Trade-offs
Replay data and extra evaluations add cost; restricting updates can cap gains on the new task.
Pitfalls
Building the retention set from data that was also used for fine-tuning.
Lessons and concepts
Claims and their sources
Kirkpatrick et al. note that neural networks are generally not capable of learning tasks sequentially without catastrophic forgetting, and propose slowing learning on weights important to old tasks.