Feature engineering and preprocessing · insight-0131

Ship the fitted preprocessing with the model so training and serving transform rows identically

Situation
Preprocessing lives in a notebook while serving engineers reimplement it in another service or language.
Action
Serialize one fitted pipeline containing preprocessing and model, version it with its data, and add a parity test that compares offline and online feature values for sampled rows.
Reason
Any transformation fitted on training data must be applied unchanged to test and production data, or the model sees a different feature space.
Boundary
Batch and online serving, especially when teams or languages differ between training and inference.

Symptom or risk

Small differences in scaling, encoding order or missing handling shift inputs; predictions degrade with no error raised.

Features computed in a shared feature store with one definition for both paths already give parity; still test it.

Bundling preprocessing ties serving to the training stack; a cross-language format needs its own parity checks.

Column order changes between training and serving that silently swap features of the same type.

Transformations fitted on training data must be applied to test and production data; in the docs' example, skipping the scaler on test data raised MSE from about 0.9 to about 62.8.
Common pitfalls and recommended practices (scikit-learn User Guide, section 12)