Feature engineering and preprocessing · insight-0114

Let gradient boosting route missing values natively before building an imputer

Situation
You plan an imputation step in front of HistGradientBoosting, LightGBM or XGBoost.
Action
Start with native missing-value handling as the baseline; add imputation only if it beats native handling on the same folds or is required for serving consistency.
Reason
These boosters learn, per split, whether missing rows go left or right based on gain, so missingness becomes a learnable signal.
Boundary
HistGradientBoosting, LightGBM and XGBoost on tabular data with NaN values.

Symptom or risk

Imputation adds code and can hide informative missingness, while the booster could have learned where missing rows belong.

If missingness at serving time will differ from training (a field becomes mandatory), learned routing goes stale; fix the data contract first.

Native handling is simple but less transparent; explicit imputation is auditable but adds assumptions.

Encoding missing as zero; LightGBM treats zeros as ordinary values unless zero_as_missing is set, and zero may be a real value.

HistGradientBoosting handles NaN natively; each split learns where missing samples go, and unseen-missing features send NaN to the child with more samples.
Ensembles: Gradient boosting, random forests, bagging, voting, stacking (scikit-learn User Guide 1.11)
LightGBM handles missing values by default (use_missing) and only treats zeros as missing with zero_as_missing=true.
Advanced Topics: Categorical Feature Support (LightGBM documentation)