Feature engineering and preprocessing · insight-0106

Make target encoding fold-safe with cross fitting; fit().transform() on training rows leaks

Situation
You replace a categorical feature such as route or sales channel with the mean target per category.
Action
Encode training rows out of fold (scikit-learn TargetEncoder.fit_transform cross-fits), use the full-data mapping only for new rows, and keep the encoder inside the pipeline.
Reason
Out-of-fold encodings are computed without the row's own label, so the feature carries only information available for unseen rows.
Boundary
Mean, smoothed or Bayesian target encodings for regression and classification, especially with many small categories.

Symptom or risk

Each training row's encoding includes its own label, so the model learns a near-copy of the target; validation looks strong, production disappoints.

With very few, large categories leakage is small; one-hot may be simpler and equally good.

Cross fitting adds noise to training encodings and costs compute; the alternative is optimistic validation.

Calling fit(X, y).transform(X) on training data, which does not equal fit_transform(X, y).

TargetEncoder.fit_transform uses cross fitting to prevent target leakage; fit(X, y).transform(X) does not and is discouraged for training data.
scikit-learn User Guide: Target Encoder (Preprocessing data, 8.3.4.2)