Feature engineering and preprocessing · insight-0112

Do not give a linear model ordinal codes for unordered categories

Situation
A nominal feature such as market or payment method is label-encoded as 0, 1, 2 and fed to a linear, kernel or distance model.
Action
Use one-hot (or target encoding) for nominal features in linear and distance models; reserve ordinal codes for truly ordered levels such as fare levels L < M < H.
Reason
A linear term imposes equal spacing and a monotone effect on the code values, which only make sense when order and spacing are real.
Boundary
Linear and logistic regression, SVMs, k-NN, k-means and neural networks without embeddings.

Symptom or risk

The model assumes intl-long is twice intl-short and forces one slope across arbitrary codes; reordering the codes changes predictions.

Tree models can split ordinal codes repeatedly, so arbitrary codes are often tolerable there, though native categorical support is usually better.

One-hot adds columns; ordinal codes are compact but encode a false structure.

Ordinal levels with unequal gaps; even true orders may need one-hot if spacing matters.

Native categorical support in HistGradientBoosting is usually better than one-hot encoding or treating categories as ordinal integers.
Ensembles: Gradient boosting, random forests, bagging, voting, stacking (scikit-learn User Guide 1.11)