Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Skip feature scaling for tree ensembles and spend the effort on features that carry signal
You are about to add StandardScaler or MinMaxScaler in front of a random forest, gradient boosting model or single decision tree.
easydocumented fact - Scale features before k-NN, SVMs and k-means or the largest unit decides the distance
A distance- or kernel-based model uses features in mixed units, such as average fare in currency, load factor in 0-1 and days to departure in days.
easydocumented fact - Standardize predictors before ridge, lasso or elastic net so the penalty is unit-free
You fit a penalized linear or logistic model where features are measured in different units.
mediumdocumented fact - Use RobustScaler when a few extreme values distort mean-and-variance scaling, then decide about the extremes separately
A scaled feature has a handful of huge values, such as a 10x passenger outlier or a corporate group booking.
easydocumented fact - Put every fitted preprocessing step inside the cross-validated pipeline, not before it
Your notebook imputes, scales or selects features on the whole dataframe, then calls crossvalscore on the result.
easydocumented fact - Make target encoding fold-safe with cross fitting; fit().transform() on training rows leaks
You replace a categorical feature such as route or sales channel with the mean target per category.
mediumdocumented fact - Shrink target encodings of rare categories toward the global mean
Many categories have only a handful of rows, for example new routes or long-tail products.
mediumdocumented fact - Consider CatBoost's ordered target statistics when categorical encoding is the main leakage risk
A tabular problem is dominated by high-cardinality categoricals and your hand-built target encoding keeps leaking or overfitting.
mediumdocumented fact - Group infrequent levels before one-hot encoding a high-cardinality categorical
One-hot encoding a feature such as travel agency or product SKU would create thousands of sparse columns.
easydocumented fact - Decide in advance how a model treats categories that first appear after deployment
A categorical feature grows over time: new routes, new products, new store codes.
mediumdocumented fact - Regularize LightGBM's native categorical splits when categories are many and small
You pass a high-cardinality integer-coded categorical to LightGBM as categoricalfeature.
mediumdocumented fact - Do not give a linear model ordinal codes for unordered categories
A nominal feature such as market or payment method is label-encoded as 0, 1, 2 and fed to a linear, kernel or distance model.
easypractitioner heuristic - Add a missing indicator when the fact of missingness may carry signal
A feature is missing more often for one segment or outcome, such as revenue missing more on intl-long flights in the synthetic raw table.
mediumdocumented fact - Let gradient boosting route missing values natively before building an imputer
You plan an imputation step in front of HistGradientBoosting, LightGBM or XGBoost.
easydocumented fact - Replace sentinel values like -999 before they reach linear or distance models
A source system encodes missing or not-applicable as -999, 9999 or 0.
easypractitioner heuristic - Monitor missing-rate shifts because a changed form or feed silently rewrites learned missingness
A model uses missing indicators or native NaN routing, and an upstream form, API or ETL job changes.
mediumpractitioner heuristic - Cap polynomial expansion degree and regularize, because generated features grow combinatorially
You apply PolynomialFeatures of degree 3 or more to dozens of inputs to capture nonlinearity.
mediumpractitioner heuristic - Guard ratio features against zero and tiny denominators
You build ratios such as revenue per seat, cancellations per booking or spend per visit.
easypractitioner heuristic - When you log-transform a skewed target, correct the back-transformed predictions toward the mean
You model log(revenue) or log(demand) and exponentiate predictions to report totals.
mediumdocumented fact - Encode cyclical calendar features so December sits next to January for linear and distance models
Month, weekday or hour enters a linear, kernel or distance model as an integer.
easypractitioner heuristic - Express time features relative to the decision moment, not as absolute dates
A model uses raw dates or timestamps as features, for example booking date or departure date as ordinal numbers.
mediumpractitioner heuristic - Compute entity history features as of the prediction timestamp with point-in-time joins
You add aggregates such as a customer's booking count, a route's average load factor or an account's late payments to each training row.
mediumdocumented fact - Run feature selection inside each training fold, never on the full dataset first
You screen hundreds of candidate features by correlation with the target or by model importance, then cross-validate the shortlist.
mediumdocumented fact - Remove features that are recorded because of the outcome, not before it
A candidate feature is filled in by a process that happens after or because of the target event, such as a cancellation reason, refund amount or collections flag.
easydocumented fact - Keep raw identifiers out of the feature set and use them for grouping instead
Customer id, flight number, device id or row index ends up as a model input.
easypractitioner heuristic - Normalize count and frequency encodings by window length so they survive changes in data volume
You encode a category by how often it appears, or an entity by its raw event count.
mediumpractitioner heuristic - Ship the fitted preprocessing with the model so training and serving transform rows identically
Preprocessing lives in a notebook while serving engineers reimplement it in another service or language.
mediumdocumented fact - Take clipping limits from training-fold quantiles and record them as model parameters
You winsorize heavy-tailed features or targets to limit the influence of extremes.
easypractitioner heuristic - Standardize features before PCA unless they share a meaningful common unit
You run PCA on features measured in different units, such as revenue, seat counts and percentages.
easydocumented fact - Scale sparse matrices without centering them
Your features are a sparse matrix, for example one-hot route-by-channel indicators or TF-IDF text features, and you add StandardScaler.
easydocumented fact - Do not treat leave-one-out target encoding as a leakage fix; it still exposes each row's own label
You replace a category with the mean target of all other rows in that category (leave-one-out) and feed it to a tree model or boosted ensemble.
mediumdocumented fact - Build target encodings in time order on time-stamped data, because random cross fitting borrows from the future
A table of time-stamped outcomes, such as no-shows by departure date, is target encoded with a standard k-fold cross fit and then judged on a later period.
hardpractitioner heuristic - Hash open-ended categorical fields into a fixed table and size it for the collisions you can accept
A free-form field such as a user agent, query string or promo code has an unbounded vocabulary and one-hot width changes at every retrain.
mediumdocumented fact - Fix the missing-indicator column list, because the default mode ties the schema to the training extract
MissingIndicator runs with its default features='missing-only', and a column that was complete in the training extract develops gaps in a later feed.
mediumdocumented fact - Replay logged arrival times to mask training features the way serving will see them
A feature comes from a feed that lands hours after the event; the warehouse history is complete, but a live scoring request often lacks the value.
hardpractitioner heuristic - Choose Yeo-Johnson over Box-Cox when a feature can be zero or negative
A PowerTransformer with method box-cox was fitted on fare or revenue columns, and later extracts contain zero fares, refunds or negative adjustments.
easydocumented fact - Expect out-of-range values after min-max scaling and decide between clipping and alerting
Inputs are min-max scaled for a neural network or linear model, and serving data includes values beyond the training minimum or maximum.
easydocumented fact - Fit depth-one boosting first to learn whether interactions matter before hand-crafting crosses
A team plans many hand-built interaction features for a gradient-boosted model without evidence that interactions carry signal.
mediumpractitioner heuristic - Skip hand-made bins before histogram boosting, which already bins every feature
Continuous features are bucketed into deciles before LightGBM or HistGradientBoosting in the name of robustness.
easydocumented fact - Freeze and version bin edges so models and monitors compare like with like
A feature is binned by quantiles for the model and again for a drift monitor, and the edges are recomputed on each retrain or on each incoming batch.
mediumpractitioner heuristic - Ship a changed feature definition under a new version instead of editing it in place
An owner improves the calculation of a shared feature, such as a cleaner delay measure, and wants the fix applied under the same name.
mediumdocumented fact - Treat a feature's time-to-live as a modelling choice, since it decides which training rows receive nulls
A feature store applies a time-to-live to a feature view, and training sets are built with point-in-time joins over entities with irregular update gaps.
harddocumented fact - Test the serialization boundary where offline and online features meet for type and null coercion
Training features come from warehouse or parquet files while serving reads the same features through JSON, protobuf or a key-value store.
mediumpractitioner heuristic - Add a train-versus-live classifier to catch feature drift that per-feature tests miss
Every per-feature drift test is quiet, yet model quality is slipping, and the features are strongly related to one another.
hardpractitioner heuristic - Judge feature drift by effect size, because p-values fire on negligible shifts at production volumes
A daily drift job runs significance tests on millions of rows per feature and marks most features as drifted most days.
mediumpractitioner heuristic - Pair a training baseline with a same-season recent window when monitoring feature drift
A monitor compares production features with the training sample only, and the business is seasonal.
mediumpractitioner heuristic - Run a leave-one-feature-out audit on a schedule to remove features that no longer earn their dependency
A production model has accumulated hundreds of features over several releases, many added in bundles or kept after a replacement arrived.
mediumdocumented fact - Re-derive constraint signs whenever a feature is redefined, reversed or re-encoded
A model has a monotone constraint vector, and a refactor replaces a feature with its reversed definition, such as discount percent for fare or days left for days elapsed.
easypractitioner heuristic - Re-express money features in a stable unit when the currency's purchasing power moves between training and serving
A model uses nominal fares or spend in a currency with sustained inflation or a drifting exchange rate, and is trained on past months and scored on later ones.
mediumpractitioner heuristic