Feature engineering and preprocessing · insight-0105
Put every fitted preprocessing step inside the cross-validated pipeline, not before it
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
Your notebook imputes, scales or selects features on the whole dataframe, then calls cross_val_score on the result.
Action
Wrap imputation, scaling, encoding and selection in a Pipeline or ColumnTransformer and pass the whole pipeline to the cross-validation or search object.
Reason
Each fold then fits preprocessing on its own training part only, which mimics deployment where future rows are unseen.
Boundary
Any estimator whose preprocessing learns parameters from data: imputers, scalers, encoders, selectors, PCA, resamplers.
Symptom or risk
Validation folds have already influenced imputation medians, scaler statistics or selected columns; cross-validation scores are optimistic.
Exceptions
Stateless transforms, such as a fixed log or a hard-coded unit conversion, learn nothing and can run before splitting.
Trade-offs
Refitting preprocessing per fold costs compute; caching transformers helps when steps are expensive.
Pitfalls
Assuming unsupervised steps are safe; a scaler or PCA fitted on all rows still uses test-row statistics.
Lessons and concepts
Claims and their sources
Splitting before preprocessing, never fitting on test data and using Pipeline prevents preprocessing leakage; selecting features on all data inflated accuracy on random data in the docs' example.
Pipeline and ColumnTransformer compose preprocessing and estimators so they are fitted together.