Feature engineering and preprocessing · insight-1117
Check the realised bin count when quantile-binning a zero-inflated feature
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, Data analyst
- Industries
- banking, aviation
Summary: situation, action, reason, boundary
Situation
KBinsDiscretizer with strategy quantile and ten bins is applied to a feature with a large spike, for example 60% zeros in days since last cancellation.
Action
Inspect n_bins_ after fitting, handle the spike with a separate is-zero flag and bin only the positive part, or set custom edges; assert the expected bin count in a pipeline test.
Reason
Equal-frequency bins need distinct quantiles; ties collapse neighbouring edges, and bins whose width is at most 1e-8 are removed.
Boundary
Counts, durations and amounts with a mass point, in scorecards and linear models.
Symptom or risk
Several quantile edges coincide at zero, bins are dropped with a warning, the result has fewer columns than configured, and the zero group is not isolated cleanly.
Exceptions
Continuous features without ties, where requested and realised bins agree.
Trade-offs
Custom edges are less automatic and need maintenance when the feature's distribution moves.
Pitfalls
Ignoring the warning in a batch job and discovering the width change when a downstream column index breaks.
Lessons and concepts
Claims and their sources
KBinsDiscretizer's n_bins_ attribute reports bins per feature, and bins whose width is too small (at most 1e-8) are removed with a warning.
The quantile strategy builds equally populated bins per feature.