Feature engineering and preprocessing · insight-1111

Use a quantile transform for heavy tails only after checking that distances and correlations may be distorted

Situation
A heavy-tailed feature feeds a distance-based or linear model, and robust scaling still leaves a long skewed tail.
Action
Fit a QuantileTransformer (uniform or normal output) on training folds, compare against a log or Yeo-Johnson transform on validation, and check which relationships between features it changes.
Reason
The quantile map depends only on ranks, so extremes cannot dominate, but spacing between values and cross-feature correlations are reshaped.
Boundary
k-NN, SVMs, clustering and linear models over skewed positive inputs such as revenue or counts.

Symptom or risk

A few extreme values still dominate distances, or a rank-based fix is applied blindly and removes the magnitude that carried the signal.

Features whose extremes are the signal, such as one very large refund, which rank mapping flattens.

Interpretability falls because a one-unit change no longer has a fixed meaning.

Fitting the transformer on all rows before splitting, or using it on a feature with few distinct values where many ties collapse.

The quantile transform is less influenced by outliers than scaling methods but distorts correlations and distances within and across features (scikit-learn section 8.3.2).
Preprocessing data (scikit-learn User Guide 8.3)