Feature engineering and preprocessing · insight-0133
Standardize features before PCA unless they share a meaningful common unit
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, ML engineer
- Industries
- manufacturing, cross-industry
Summary: situation, action, reason, boundary
Situation
You run PCA on features measured in different units, such as revenue, seat counts and percentages.
Action
Standardize inside the pipeline before PCA, and inspect explained variance and loadings both with and without scaling before choosing.
Reason
PCA finds directions of maximum variance, so features with numerically larger variance dominate unless rescaled.
Boundary
PCA for compression, visualization, denoising or as input to clustering and regression.
Symptom or risk
The first components simply align with the largest-variance units; the reduction reflects measurement scale, not structure.
Exceptions
When all variables share a unit and their variance differences matter (sensor readings in the same unit), leaving them unscaled can be correct.
Trade-offs
Scaling gives each feature equal prior weight, which may overweight noisy low-variance features.
Pitfalls
Fitting PCA on the full dataset before cross-validation, which leaks test-set structure.
Lessons and concepts
Claims and their sources
PCA results depend on whether variables are individually scaled; variables in different units are usually scaled to standard deviation one (ISL, PCA chapter, scaling the variables).