Feature engineering and preprocessing · insight-1104
Hash open-ended categorical fields into a fixed table and size it for the collisions you can accept
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- ML engineer, Data scientist
- Industries
- e-commerce, saas
Summary: situation, action, reason, boundary
Situation
A free-form field such as a user agent, query string or promo code has an unbounded vocabulary and one-hot width changes at every retrain.
Action
Use FeatureHasher or HashingVectorizer with n_features a power of two, keep signed hashing on for small tables, and choose the size by comparing validation scores across 2^16 to 2^20.
Reason
A fixed hash function maps every token to a column without storing a vocabulary, so unseen tokens still land somewhere; signs let collisions cancel on average rather than accumulate.
Boundary
Linear and other sparse-input models over tokens, URLs, identifiers and cross features.
Symptom or risk
The feature matrix width drifts between runs, memory spikes, and values first seen at serving have no column.
Exceptions
When individual levels must be explained or counted exactly: the hasher cannot be inverted.
Trade-offs
Rare levels blur together, and debugging needs the raw tokens kept elsewhere.
Pitfalls
A table size that is not a power of two maps unevenly, and disabling signs on a small table lets collision errors add up.
Lessons and concepts
Claims and their sources
FeatureHasher uses a signed hash so collisions tend to cancel, advises a power-of-two n_features because a modulo maps hashes to columns, and has no inverse_transform.