Feature engineering and preprocessing · insight-1118
Skip hand-made bins before histogram boosting, which already bins every feature
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
Continuous features are bucketed into deciles before LightGBM or HistGradientBoosting in the name of robustness.
Action
Feed raw numeric features to histogram boosting and tune max_bins (max_bin in LightGBM and XGBoost) if resolution matters; bin only for reporting or for a model that needs it, such as a scorecard.
Reason
Histogram-based boosting first maps each feature to at most about 256 integer bins and searches splits over those, so extra pre-binning can only remove candidate thresholds.
Boundary
HistGradientBoosting, LightGBM and XGBoost with hist tree method.
Symptom or risk
Two coarsening steps stack, candidate thresholds are limited to decile edges, accuracy near decision boundaries drops, and the pipeline gains an extra fitted component.
Exceptions
Regulatory scorecards, monotone weight-of-evidence binning, or deliberate coarsening to stabilise a noisy feature.
Trade-offs
Raw features expose model sensitivity to measurement noise that bins would hide.
Pitfalls
Assuming fewer bins always generalise better; max_bins is itself a regulariser to tune on validation data.
Lessons and concepts
Claims and their sources
HistGradientBoosting first bins input samples into integer-valued bins (typically 256), max_bins defaults to 255, and fewer bins act as a form of regularization.