Neural networks, deep learning, NLP · insight-0440
Reproduce the pretrained model's preprocessing exactly before fine-tuning
- Evidence type
- practitioner heuristic
- Difficulty
- easy
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
You load a pretrained image or text backbone and feed it data prepared by your own pipeline.
Action
Read the model card or config and copy its preprocessing (resize, crop, normalization, color order, tokenizer and casing). Assert on a few samples that your pipeline matches the reference preprocessing outputs.
Reason
Pretrained features are tuned to a specific input distribution; a mismatch is a domain shift you created yourself.
Boundary
Any transfer learning from published checkpoints.
Symptom or risk
Different normalization constants, channel order, resolution, casing or tokenizer settings shift inputs away from what the backbone learned, and transfer looks weaker than it is.
Exceptions
Deliberate domain adaptation, where you retrain early layers anyway.
Trade-offs
The reference preprocessing may be slower or awkward for your serving stack; port it, do not approximate it.
Pitfalls
Applying the correct preprocessing in training but a simplified version in serving.