Feature engineering and preprocessing · insight-0110
Decide in advance how a model treats categories that first appear after deployment
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- ML engineer, Data scientist
- Industries
- aviation, retail
Summary: situation, action, reason, boundary
Situation
A categorical feature grows over time: new routes, new products, new store codes.
Action
Choose an explicit unknown policy (infrequent bucket, global-mean encoding, or a route to a fallback model), test it with a held-out set of new levels, and monitor the unseen-level share.
Reason
Encoders learn a vocabulary at fit time; whatever happens to new values is a design decision, not an accident to discover in production.
Boundary
One-hot, ordinal, target and native categorical encodings in batch or online serving.
Symptom or risk
The serving transform crashes on an unseen level, or silently maps it to the dropped baseline level and gives a confident wrong score.
Exceptions
Closed vocabularies (fare class L/M/H) rarely change; a hard failure may be the right alarm there.
Trade-offs
A silent fallback keeps serving available but can hide drift; a hard failure is safer but needs an on-call response.
Pitfalls
Combining drop='first' with handle_unknown='ignore', which makes unseen levels indistinguishable from the dropped level.
Example
Synthetic airline data: a new IST route launched mid-season should get a cold-start estimate from similar routes, not the baseline route's coefficients.
Lessons and concepts
Claims and their sources
With drop set and handle_unknown='ignore', OneHotEncoder encodes unknown categories as all zeros, the same as the dropped category.
LightGBM treats categories unseen in training as missing at prediction time.