Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Skip feature scaling for tree ensembles and spend the effort on features that carry signal
You are about to add StandardScaler or MinMaxScaler in front of a random forest, gradient boosting model or single decision tree.
easydocumented fact - Scale features before k-NN, SVMs and k-means or the largest unit decides the distance
A distance- or kernel-based model uses features in mixed units, such as average fare in currency, load factor in 0-1 and days to departure in days.
easydocumented fact - Standardize predictors before ridge, lasso or elastic net so the penalty is unit-free
You fit a penalized linear or logistic model where features are measured in different units.
mediumdocumented fact - Use RobustScaler when a few extreme values distort mean-and-variance scaling, then decide about the extremes separately
A scaled feature has a handful of huge values, such as a 10x passenger outlier or a corporate group booking.
easydocumented fact - Put every fitted preprocessing step inside the cross-validated pipeline, not before it
Your notebook imputes, scales or selects features on the whole dataframe, then calls crossvalscore on the result.
easydocumented fact - Make target encoding fold-safe with cross fitting; fit().transform() on training rows leaks
You replace a categorical feature such as route or sales channel with the mean target per category.
mediumdocumented fact - Shrink target encodings of rare categories toward the global mean
Many categories have only a handful of rows, for example new routes or long-tail products.
mediumdocumented fact - Consider CatBoost's ordered target statistics when categorical encoding is the main leakage risk
A tabular problem is dominated by high-cardinality categoricals and your hand-built target encoding keeps leaking or overfitting.
mediumdocumented fact - Group infrequent levels before one-hot encoding a high-cardinality categorical
One-hot encoding a feature such as travel agency or product SKU would create thousands of sparse columns.
easydocumented fact - Decide in advance how a model treats categories that first appear after deployment
A categorical feature grows over time: new routes, new products, new store codes.
mediumdocumented fact - Regularize LightGBM's native categorical splits when categories are many and small
You pass a high-cardinality integer-coded categorical to LightGBM as categoricalfeature.
mediumdocumented fact - Do not give a linear model ordinal codes for unordered categories
A nominal feature such as market or payment method is label-encoded as 0, 1, 2 and fed to a linear, kernel or distance model.
easypractitioner heuristic - Add a missing indicator when the fact of missingness may carry signal
A feature is missing more often for one segment or outcome, such as revenue missing more on intl-long flights in the synthetic raw table.
mediumdocumented fact - Let gradient boosting route missing values natively before building an imputer
You plan an imputation step in front of HistGradientBoosting, LightGBM or XGBoost.
easydocumented fact - Avoid single mean imputation when the analysis is inferential rather than predictive
A stakeholder wants coefficients, confidence intervals or group comparisons from data with substantial missingness.
mediumdocumented fact - Replace sentinel values like -999 before they reach linear or distance models
A source system encodes missing or not-applicable as -999, 9999 or 0.
easypractitioner heuristic - Monitor missing-rate shifts because a changed form or feed silently rewrites learned missingness
A model uses missing indicators or native NaN routing, and an upstream form, API or ETL job changes.
mediumpractitioner heuristic - Add explicit interaction terms to a linear model when the effect of one driver depends on another
Domain knowledge says an effect varies by context, for example price sensitivity that differs between short-haul and long-haul markets.
mediumpractitioner heuristic - Cap polynomial expansion degree and regularize, because generated features grow combinatorially
You apply PolynomialFeatures of degree 3 or more to dozens of inputs to capture nonlinearity.
mediumpractitioner heuristic - Guard ratio features against zero and tiny denominators
You build ratios such as revenue per seat, cancellations per booking or spend per visit.
easypractitioner heuristic - When you log-transform a skewed target, correct the back-transformed predictions toward the mean
You model log(revenue) or log(demand) and exponentiate predictions to report totals.
mediumdocumented fact - Log-transform skewed inputs for linear and distance models, not for trees
A numeric feature spans orders of magnitude, such as corporate account spend or group size.
easypractitioner heuristic - Prefer splines to coarse bins when a linear model needs a nonlinear single-feature effect
You plan to cut days to departure or customer age into a few buckets so a linear model can bend.
mediumpractitioner heuristic - Encode cyclical calendar features so December sits next to January for linear and distance models
Month, weekday or hour enters a linear, kernel or distance model as an integer.
easypractitioner heuristic - Express time features relative to the decision moment, not as absolute dates
A model uses raw dates or timestamps as features, for example booking date or departure date as ordinal numbers.
mediumpractitioner heuristic - Compute entity history features as of the prediction timestamp with point-in-time joins
You add aggregates such as a customer's booking count, a route's average load factor or an account's late payments to each training row.
mediumdocumented fact - Run feature selection inside each training fold, never on the full dataset first
You screen hundreds of candidate features by correlation with the target or by model importance, then cross-validate the shortlist.
mediumdocumented fact - Remove features that are recorded because of the outcome, not before it
A candidate feature is filled in by a process that happens after or because of the target event, such as a cancellation reason, refund amount or collections flag.
easydocumented fact - Keep raw identifiers out of the feature set and use them for grouping instead
Customer id, flight number, device id or row index ends up as a model input.
easypractitioner heuristic - Normalize count and frequency encodings by window length so they survive changes in data volume
You encode a category by how often it appears, or an entity by its raw event count.
mediumpractitioner heuristic - Ship the fitted preprocessing with the model so training and serving transform rows identically
Preprocessing lives in a notebook while serving engineers reimplement it in another service or language.
mediumdocumented fact - Take clipping limits from training-fold quantiles and record them as model parameters
You winsorize heavy-tailed features or targets to limit the influence of extremes.
easypractitioner heuristic - Standardize features before PCA unless they share a meaningful common unit
You run PCA on features measured in different units, such as revenue, seat counts and percentages.
easydocumented fact - Drop one dummy level for unpenalized linear regression, keep all levels for penalized models and trees
You one-hot encode categoricals for a linear model and must decide whether to set drop='first'.
mediumdocumented fact - Scale sparse matrices without centering them
Your features are a sparse matrix, for example one-hot route-by-channel indicators or TF-IDF text features, and you add StandardScaler.
easydocumented fact - Do not treat leave-one-out target encoding as a leakage fix; it still exposes each row's own label
You replace a category with the mean target of all other rows in that category (leave-one-out) and feed it to a tree model or boosted ensemble.
mediumdocumented fact - Build target encodings in time order on time-stamped data, because random cross fitting borrows from the future
A table of time-stamped outcomes, such as no-shows by departure date, is target encoded with a standard k-fold cross fit and then judged on a later period.
hardpractitioner heuristic - Shrink a thin route's target encoding toward its origin mean before the global mean
Route-level target encoding is shrunk toward the overall rate, yet routes sharing an origin behave alike and new routes keep appearing with a handful of flights.
hardpractitioner heuristic - Hash open-ended categorical fields into a fixed table and size it for the collisions you can accept
A free-form field such as a user agent, query string or promo code has an unbounded vocabulary and one-hot width changes at every retrain.
mediumdocumented fact - Replace a high-cardinality code with its descriptive attributes so new codes inherit meaning
A model uses airport, aircraft-type or merchant codes with hundreds of levels, and new codes arrive with no history.
mediumpractitioner heuristic - Count populated cells before adding a crossed categorical feature
You cross two or three categoricals, such as route by weekday by fare class, to give a linear model an interaction or to build a composite key.
mediumpractitioner heuristic - Bracket a suspected MNAR column with a delta adjustment instead of arguing about MAR versus MNAR
A decision-critical column, such as stated income or a survey score, may be missing because of its own value, and the team debates the mechanism from missingness plots.
harddocumented fact - Fix the missing-indicator column list, because the default mode ties the schema to the training extract
MissingIndicator runs with its default features='missing-only', and a column that was complete in the training extract develops gaps in a later feed.
mediumdocumented fact - Replay logged arrival times to mask training features the way serving will see them
A feature comes from a feed that lands hours after the event; the warehouse history is complete, but a live scoring request often lacks the value.
hardpractitioner heuristic - Include the outcome when imputing for an inferential model and leave it out of a deployed imputer
You must impute predictors, and the same dataset serves both a coefficient-interpretation study and a model to be deployed for scoring.
harddocumented fact - Use a quantile transform for heavy tails only after checking that distances and correlations may be distorted
A heavy-tailed feature feeds a distance-based or linear model, and robust scaling still leaves a long skewed tail.
mediumdocumented fact - Choose Yeo-Johnson over Box-Cox when a feature can be zero or negative
A PowerTransformer with method box-cox was fitted on fare or revenue columns, and later extracts contain zero fares, refunds or negative adjustments.
easydocumented fact - Expect out-of-range values after min-max scaling and decide between clipping and alerting
Inputs are min-max scaled for a neural network or linear model, and serving data includes values beyond the training minimum or maximum.
easydocumented fact - Hand a boosted tree the ratio it cannot build cheaply, such as load factor or price per unit
The target depends on a ratio, bookings over capacity or fare over distance, yet the model receives only numerator and denominator.
mediumpractitioner heuristic - Replace two near-duplicate prices with one level and one gap before interpreting a linear model
Own fare and competitor fare move together, both sit in a linear or logistic model, and their coefficients flip sign between retrains.
mediumpractitioner heuristic - Fit depth-one boosting first to learn whether interactions matter before hand-crafting crosses
A team plans many hand-built interaction features for a gradient-boosted model without evidence that interactions carry signal.
mediumpractitioner heuristic - Check the realised bin count when quantile-binning a zero-inflated feature
KBinsDiscretizer with strategy quantile and ten bins is applied to a feature with a large spike, for example 60% zeros in days since last cancellation.
easydocumented fact - Skip hand-made bins before histogram boosting, which already bins every feature
Continuous features are bucketed into deciles before LightGBM or HistGradientBoosting in the name of robustness.
easydocumented fact - Freeze and version bin edges so models and monitors compare like with like
A feature is binned by quantiles for the model and again for a drift monitor, and the edges are recomputed on each retrain or on each incoming batch.
mediumpractitioner heuristic - Lowercase Turkish text with language-aware rules, because default lowercasing splits dotted and dotless I
Turkish free text, such as complaint notes or city names, is lowercased by the default step of a TF-IDF vectorizer.
mediumdocumented fact - Use character n-grams for short, noisy strings such as place names, codes and product titles
You classify or match short strings with typos and spelling variants, such as 'Antalya' against 'Antlya' or truncated product names.
easydocumented fact - Build calendar features for each timestamp that drives the outcome, such as booking date and travel date
A booking table has both booking date and departure date, but only departure-date calendar features exist because it is the row key.
mediumpractitioner heuristic - Ship a changed feature definition under a new version instead of editing it in place
An owner improves the calculation of a shared feature, such as a cleaner delay measure, and wants the fix applied under the same name.
mediumdocumented fact - Treat a feature's time-to-live as a modelling choice, since it decides which training rows receive nulls
A feature store applies a time-to-live to a feature view, and training sets are built with point-in-time joins over entities with irregular update gaps.
harddocumented fact - Test the serialization boundary where offline and online features meet for type and null coercion
Training features come from warehouse or parquet files while serving reads the same features through JSON, protobuf or a key-value store.
mediumpractitioner heuristic - Add a train-versus-live classifier to catch feature drift that per-feature tests miss
Every per-feature drift test is quiet, yet model quality is slipping, and the features are strongly related to one another.
hardpractitioner heuristic - Judge feature drift by effect size, because p-values fire on negligible shifts at production volumes
A daily drift job runs significance tests on millions of rows per feature and marks most features as drifted most days.
mediumpractitioner heuristic - Pair a training baseline with a same-season recent window when monitoring feature drift
A monitor compares production features with the training sample only, and the business is seasonal.
mediumpractitioner heuristic - Keep only features that survive repeated subsampling when selection is noisy
A lasso or importance-based selector returns a different shortlist when a few hundred rows change, and the team has to freeze a feature set.
harddocumented fact - Report selection stability across folds as an overlap statistic next to the accuracy
Two selection methods reach the same cross-validated score, and you must choose one to document as the feature recipe.
mediumdocumented fact - Declare which features are discrete before ranking them by mutual information
You rank features with mutualinfoclassif on a dense matrix that contains integer-coded categories next to continuous measurements.
mediumdocumented fact - Run a leave-one-feature-out audit on a schedule to remove features that no longer earn their dependency
A production model has accumulated hundreds of features over several releases, many added in bundles or kept after a replacement arrived.
mediumdocumented fact - Re-derive constraint signs whenever a feature is redefined, reversed or re-encoded
A model has a monotone constraint vector, and a refactor replaces a feature with its reversed definition, such as discount percent for fare or days left for days elapsed.
easypractitioner heuristic - Compute distances from coordinates with a great-circle formula in radians, not Euclidean on degrees
A model needs distance between origins and destinations, or a nearest-station feature, from latitude and longitude columns, and the step runs on raw degree values.
easypractitioner heuristic - Re-express money features in a stable unit when the currency's purchasing power moves between training and serving
A model uses nominal fares or spend in a currency with sustained inflation or a drifting exchange rate, and is trained on past months and scored on later ones.
mediumpractitioner heuristic