Feature engineering and preprocessing · insight-0122

Log-transform skewed inputs for linear and distance models, not for trees

Situation
A numeric feature spans orders of magnitude, such as corporate account spend or group size.
Action
Apply a log (with an offset for zeros) or another monotone transform in the linear or distance branch, and leave the raw feature for tree models.
Reason
Many effects are multiplicative, so a log makes them closer to linear; trees are unaffected by monotone transforms anyway.
Boundary
Linear and logistic regression, k-NN, k-means and neural networks with heavy-tailed positive inputs.

Symptom or risk

Linear models fit the extreme rows and underfit the bulk; distance models treat a 10x difference as huge whatever the base.

Features with negative values or meaningful zeros need a different transform or a separate zero flag.

Log scales help fit but make coefficients describe percentage changes, which you must explain.

Choosing the offset after looking at test performance, or applying the transform differently in serving.

Tree-based models are invariant under strictly monotone transformations of individual predictors (ESL section 10.7).
The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd edition)