Neural networks, deep learning, NLP · insight-0433
Compute input normalization statistics on the training split only
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, ML engineer
- Industries
- energy
Summary: situation, action, reason, boundary
Situation
You standardize tabular inputs for a neural net with a mean and standard deviation computed on the full dataset before splitting.
Action
Fit the scaler on the training split, store it with the model artefact, and apply the same stored transform to validation, test and serving inputs.
Reason
This is the Day 9 rule applied to neural pipelines: any fitted preprocessing is part of the model and must not see evaluation data.
Boundary
Feature scaling, vocabulary building, target encoding and image mean or std computed from your own data.
Symptom or risk
Validation information enters training through the statistics; with time-ordered data, future level shifts leak into the past and backtests look better than live performance.
Exceptions
Using published normalization constants of a pretrained model, which were fixed before your data existed.
Trade-offs
Statistics from a short training window may be noisy; that is the honest version of what serving sees.
Pitfalls
Recomputing statistics at serving time on the incoming batch, which creates training-serving skew.
Lessons and concepts
Claims and their sources
scikit-learn's common-pitfalls guide warns against fitting preprocessing on the full dataset and recommends fitting on training data only, for example inside a pipeline.