Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Standardize predictors before ridge, lasso or elastic net so the penalty is unit-free
You fit a penalized linear or logistic model where features are measured in different units.
mediumdocumented fact - Make target encoding fold-safe with cross fitting; fit().transform() on training rows leaks
You replace a categorical feature such as route or sales channel with the mean target per category.
mediumdocumented fact - Shrink target encodings of rare categories toward the global mean
Many categories have only a handful of rows, for example new routes or long-tail products.
mediumdocumented fact - Consider CatBoost's ordered target statistics when categorical encoding is the main leakage risk
A tabular problem is dominated by high-cardinality categoricals and your hand-built target encoding keeps leaking or overfitting.
mediumdocumented fact - Decide in advance how a model treats categories that first appear after deployment
A categorical feature grows over time: new routes, new products, new store codes.
mediumdocumented fact - Regularize LightGBM's native categorical splits when categories are many and small
You pass a high-cardinality integer-coded categorical to LightGBM as categoricalfeature.
mediumdocumented fact - Add a missing indicator when the fact of missingness may carry signal
A feature is missing more often for one segment or outcome, such as revenue missing more on intl-long flights in the synthetic raw table.
mediumdocumented fact - Monitor missing-rate shifts because a changed form or feed silently rewrites learned missingness
A model uses missing indicators or native NaN routing, and an upstream form, API or ETL job changes.
mediumpractitioner heuristic - Cap polynomial expansion degree and regularize, because generated features grow combinatorially
You apply PolynomialFeatures of degree 3 or more to dozens of inputs to capture nonlinearity.
mediumpractitioner heuristic - When you log-transform a skewed target, correct the back-transformed predictions toward the mean
You model log(revenue) or log(demand) and exponentiate predictions to report totals.
mediumdocumented fact - Express time features relative to the decision moment, not as absolute dates
A model uses raw dates or timestamps as features, for example booking date or departure date as ordinal numbers.
mediumpractitioner heuristic - Compute entity history features as of the prediction timestamp with point-in-time joins
You add aggregates such as a customer's booking count, a route's average load factor or an account's late payments to each training row.
mediumdocumented fact - Run feature selection inside each training fold, never on the full dataset first
You screen hundreds of candidate features by correlation with the target or by model importance, then cross-validate the shortlist.
mediumdocumented fact - Normalize count and frequency encodings by window length so they survive changes in data volume
You encode a category by how often it appears, or an entity by its raw event count.
mediumpractitioner heuristic - Ship the fitted preprocessing with the model so training and serving transform rows identically
Preprocessing lives in a notebook while serving engineers reimplement it in another service or language.
mediumdocumented fact - Do not treat leave-one-out target encoding as a leakage fix; it still exposes each row's own label
You replace a category with the mean target of all other rows in that category (leave-one-out) and feed it to a tree model or boosted ensemble.
mediumdocumented fact - Hash open-ended categorical fields into a fixed table and size it for the collisions you can accept
A free-form field such as a user agent, query string or promo code has an unbounded vocabulary and one-hot width changes at every retrain.
mediumdocumented fact - Fix the missing-indicator column list, because the default mode ties the schema to the training extract
MissingIndicator runs with its default features='missing-only', and a column that was complete in the training extract develops gaps in a later feed.
mediumdocumented fact - Fit depth-one boosting first to learn whether interactions matter before hand-crafting crosses
A team plans many hand-built interaction features for a gradient-boosted model without evidence that interactions carry signal.
mediumpractitioner heuristic - Freeze and version bin edges so models and monitors compare like with like
A feature is binned by quantiles for the model and again for a drift monitor, and the edges are recomputed on each retrain or on each incoming batch.
mediumpractitioner heuristic - Ship a changed feature definition under a new version instead of editing it in place
An owner improves the calculation of a shared feature, such as a cleaner delay measure, and wants the fix applied under the same name.
mediumdocumented fact - Test the serialization boundary where offline and online features meet for type and null coercion
Training features come from warehouse or parquet files while serving reads the same features through JSON, protobuf or a key-value store.
mediumpractitioner heuristic - Judge feature drift by effect size, because p-values fire on negligible shifts at production volumes
A daily drift job runs significance tests on millions of rows per feature and marks most features as drifted most days.
mediumpractitioner heuristic - Pair a training baseline with a same-season recent window when monitoring feature drift
A monitor compares production features with the training sample only, and the business is seasonal.
mediumpractitioner heuristic - Run a leave-one-feature-out audit on a schedule to remove features that no longer earn their dependency
A production model has accumulated hundreds of features over several releases, many added in bundles or kept after a replacement arrived.
mediumdocumented fact - Re-express money features in a stable unit when the currency's purchasing power moves between training and serving
A model uses nominal fares or spend in a currency with sustained inflation or a drifting exchange rate, and is trained on past months and scored on later ones.
mediumpractitioner heuristic