Feature engineering and preprocessing · insight-1106
Count populated cells before adding a crossed categorical feature
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, Data analyst
- Industries
- aviation, retail
Summary: situation, action, reason, boundary
Situation
You cross two or three categoricals, such as route by weekday by fare class, to give a linear model an interaction or to build a composite key.
Action
Count rows per cell in the training fold, keep crosses where most rows sit in cells above a minimum count, and send the rest to an other bucket or cross coarser parents instead.
Reason
The variance of a cell's estimate scales with one over its row count, so many thin cells add variance without adding signal.
Boundary
Logistic and linear models with explicit crosses, and target encoding of composite keys.
Symptom or risk
Possible cells multiply while data covers a small share, so most cells hold a few rows; one-hot width balloons and encodings of thin cells are mostly noise.
Exceptions
Tree ensembles find interactions with a few splits, so a manual cross is often unnecessary there.
Trade-offs
A minimum count drops genuinely rare but important combinations.
Pitfalls
Counting cells on the full dataset, holdout rows included.
Example
Hypothetical: 11 routes by 7 weekdays by 4 fare classes make 308 cells; with 5,000 rows the mean is 16 rows per cell, and the median can be far lower.
Lessons and concepts
Claims and their sources
OneHotEncoder can group infrequent categories with min_frequency or max_categories into a single infrequent bucket.