Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Standardize predictors before ridge, lasso or elastic net so the penalty is unit-free
You fit a penalized linear or logistic model where features are measured in different units.
mediumdocumented fact - Make target encoding fold-safe with cross fitting; fit().transform() on training rows leaks
You replace a categorical feature such as route or sales channel with the mean target per category.
mediumdocumented fact - Shrink target encodings of rare categories toward the global mean
Many categories have only a handful of rows, for example new routes or long-tail products.
mediumdocumented fact - Consider CatBoost's ordered target statistics when categorical encoding is the main leakage risk
A tabular problem is dominated by high-cardinality categoricals and your hand-built target encoding keeps leaking or overfitting.
mediumdocumented fact - Decide in advance how a model treats categories that first appear after deployment
A categorical feature grows over time: new routes, new products, new store codes.
mediumdocumented fact - Regularize LightGBM's native categorical splits when categories are many and small
You pass a high-cardinality integer-coded categorical to LightGBM as categoricalfeature.
mediumdocumented fact - Add a missing indicator when the fact of missingness may carry signal
A feature is missing more often for one segment or outcome, such as revenue missing more on intl-long flights in the synthetic raw table.
mediumdocumented fact - Avoid single mean imputation when the analysis is inferential rather than predictive
A stakeholder wants coefficients, confidence intervals or group comparisons from data with substantial missingness.
mediumdocumented fact - Monitor missing-rate shifts because a changed form or feed silently rewrites learned missingness
A model uses missing indicators or native NaN routing, and an upstream form, API or ETL job changes.
mediumpractitioner heuristic - Add explicit interaction terms to a linear model when the effect of one driver depends on another
Domain knowledge says an effect varies by context, for example price sensitivity that differs between short-haul and long-haul markets.
mediumpractitioner heuristic - Cap polynomial expansion degree and regularize, because generated features grow combinatorially
You apply PolynomialFeatures of degree 3 or more to dozens of inputs to capture nonlinearity.
mediumpractitioner heuristic - When you log-transform a skewed target, correct the back-transformed predictions toward the mean
You model log(revenue) or log(demand) and exponentiate predictions to report totals.
mediumdocumented fact - Prefer splines to coarse bins when a linear model needs a nonlinear single-feature effect
You plan to cut days to departure or customer age into a few buckets so a linear model can bend.
mediumpractitioner heuristic - Express time features relative to the decision moment, not as absolute dates
A model uses raw dates or timestamps as features, for example booking date or departure date as ordinal numbers.
mediumpractitioner heuristic - Compute entity history features as of the prediction timestamp with point-in-time joins
You add aggregates such as a customer's booking count, a route's average load factor or an account's late payments to each training row.
mediumdocumented fact - Run feature selection inside each training fold, never on the full dataset first
You screen hundreds of candidate features by correlation with the target or by model importance, then cross-validate the shortlist.
mediumdocumented fact - Normalize count and frequency encodings by window length so they survive changes in data volume
You encode a category by how often it appears, or an entity by its raw event count.
mediumpractitioner heuristic - Ship the fitted preprocessing with the model so training and serving transform rows identically
Preprocessing lives in a notebook while serving engineers reimplement it in another service or language.
mediumdocumented fact - Drop one dummy level for unpenalized linear regression, keep all levels for penalized models and trees
You one-hot encode categoricals for a linear model and must decide whether to set drop='first'.
mediumdocumented fact - Do not treat leave-one-out target encoding as a leakage fix; it still exposes each row's own label
You replace a category with the mean target of all other rows in that category (leave-one-out) and feed it to a tree model or boosted ensemble.
mediumdocumented fact - Hash open-ended categorical fields into a fixed table and size it for the collisions you can accept
A free-form field such as a user agent, query string or promo code has an unbounded vocabulary and one-hot width changes at every retrain.
mediumdocumented fact - Replace a high-cardinality code with its descriptive attributes so new codes inherit meaning
A model uses airport, aircraft-type or merchant codes with hundreds of levels, and new codes arrive with no history.
mediumpractitioner heuristic - Count populated cells before adding a crossed categorical feature
You cross two or three categoricals, such as route by weekday by fare class, to give a linear model an interaction or to build a composite key.
mediumpractitioner heuristic - Fix the missing-indicator column list, because the default mode ties the schema to the training extract
MissingIndicator runs with its default features='missing-only', and a column that was complete in the training extract develops gaps in a later feed.
mediumdocumented fact - Use a quantile transform for heavy tails only after checking that distances and correlations may be distorted
A heavy-tailed feature feeds a distance-based or linear model, and robust scaling still leaves a long skewed tail.
mediumdocumented fact - Hand a boosted tree the ratio it cannot build cheaply, such as load factor or price per unit
The target depends on a ratio, bookings over capacity or fare over distance, yet the model receives only numerator and denominator.
mediumpractitioner heuristic - Replace two near-duplicate prices with one level and one gap before interpreting a linear model
Own fare and competitor fare move together, both sit in a linear or logistic model, and their coefficients flip sign between retrains.
mediumpractitioner heuristic - Fit depth-one boosting first to learn whether interactions matter before hand-crafting crosses
A team plans many hand-built interaction features for a gradient-boosted model without evidence that interactions carry signal.
mediumpractitioner heuristic - Freeze and version bin edges so models and monitors compare like with like
A feature is binned by quantiles for the model and again for a drift monitor, and the edges are recomputed on each retrain or on each incoming batch.
mediumpractitioner heuristic - Lowercase Turkish text with language-aware rules, because default lowercasing splits dotted and dotless I
Turkish free text, such as complaint notes or city names, is lowercased by the default step of a TF-IDF vectorizer.
mediumdocumented fact - Build calendar features for each timestamp that drives the outcome, such as booking date and travel date
A booking table has both booking date and departure date, but only departure-date calendar features exist because it is the row key.
mediumpractitioner heuristic - Ship a changed feature definition under a new version instead of editing it in place
An owner improves the calculation of a shared feature, such as a cleaner delay measure, and wants the fix applied under the same name.
mediumdocumented fact - Judge feature drift by effect size, because p-values fire on negligible shifts at production volumes
A daily drift job runs significance tests on millions of rows per feature and marks most features as drifted most days.
mediumpractitioner heuristic - Pair a training baseline with a same-season recent window when monitoring feature drift
A monitor compares production features with the training sample only, and the business is seasonal.
mediumpractitioner heuristic - Report selection stability across folds as an overlap statistic next to the accuracy
Two selection methods reach the same cross-validated score, and you must choose one to document as the feature recipe.
mediumdocumented fact - Declare which features are discrete before ranking them by mutual information
You rank features with mutualinfoclassif on a dense matrix that contains integer-coded categories next to continuous measurements.
mediumdocumented fact - Run a leave-one-feature-out audit on a schedule to remove features that no longer earn their dependency
A production model has accumulated hundreds of features over several releases, many added in bundles or kept after a replacement arrived.
mediumdocumented fact - Re-express money features in a stable unit when the currency's purchasing power moves between training and serving
A model uses nominal fares or spend in a currency with sustained inflation or a drifting exchange rate, and is trained on past months and scored on later ones.
mediumpractitioner heuristic