Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Skip feature scaling for tree ensembles and spend the effort on features that carry signal
You are about to add StandardScaler or MinMaxScaler in front of a random forest, gradient boosting model or single decision tree.
easydocumented fact - Scale features before k-NN, SVMs and k-means or the largest unit decides the distance
A distance- or kernel-based model uses features in mixed units, such as average fare in currency, load factor in 0-1 and days to departure in days.
easydocumented fact - Use RobustScaler when a few extreme values distort mean-and-variance scaling, then decide about the extremes separately
A scaled feature has a handful of huge values, such as a 10x passenger outlier or a corporate group booking.
easydocumented fact - Put every fitted preprocessing step inside the cross-validated pipeline, not before it
Your notebook imputes, scales or selects features on the whole dataframe, then calls crossvalscore on the result.
easydocumented fact - Group infrequent levels before one-hot encoding a high-cardinality categorical
One-hot encoding a feature such as travel agency or product SKU would create thousands of sparse columns.
easydocumented fact - Do not give a linear model ordinal codes for unordered categories
A nominal feature such as market or payment method is label-encoded as 0, 1, 2 and fed to a linear, kernel or distance model.
easypractitioner heuristic - Let gradient boosting route missing values natively before building an imputer
You plan an imputation step in front of HistGradientBoosting, LightGBM or XGBoost.
easydocumented fact - Replace sentinel values like -999 before they reach linear or distance models
A source system encodes missing or not-applicable as -999, 9999 or 0.
easypractitioner heuristic - Guard ratio features against zero and tiny denominators
You build ratios such as revenue per seat, cancellations per booking or spend per visit.
easypractitioner heuristic - Log-transform skewed inputs for linear and distance models, not for trees
A numeric feature spans orders of magnitude, such as corporate account spend or group size.
easypractitioner heuristic - Encode cyclical calendar features so December sits next to January for linear and distance models
Month, weekday or hour enters a linear, kernel or distance model as an integer.
easypractitioner heuristic - Remove features that are recorded because of the outcome, not before it
A candidate feature is filled in by a process that happens after or because of the target event, such as a cancellation reason, refund amount or collections flag.
easydocumented fact - Keep raw identifiers out of the feature set and use them for grouping instead
Customer id, flight number, device id or row index ends up as a model input.
easypractitioner heuristic - Take clipping limits from training-fold quantiles and record them as model parameters
You winsorize heavy-tailed features or targets to limit the influence of extremes.
easypractitioner heuristic - Standardize features before PCA unless they share a meaningful common unit
You run PCA on features measured in different units, such as revenue, seat counts and percentages.
easydocumented fact - Scale sparse matrices without centering them
Your features are a sparse matrix, for example one-hot route-by-channel indicators or TF-IDF text features, and you add StandardScaler.
easydocumented fact - Choose Yeo-Johnson over Box-Cox when a feature can be zero or negative
A PowerTransformer with method box-cox was fitted on fare or revenue columns, and later extracts contain zero fares, refunds or negative adjustments.
easydocumented fact - Expect out-of-range values after min-max scaling and decide between clipping and alerting
Inputs are min-max scaled for a neural network or linear model, and serving data includes values beyond the training minimum or maximum.
easydocumented fact - Check the realised bin count when quantile-binning a zero-inflated feature
KBinsDiscretizer with strategy quantile and ten bins is applied to a feature with a large spike, for example 60% zeros in days since last cancellation.
easydocumented fact - Skip hand-made bins before histogram boosting, which already bins every feature
Continuous features are bucketed into deciles before LightGBM or HistGradientBoosting in the name of robustness.
easydocumented fact - Use character n-grams for short, noisy strings such as place names, codes and product titles
You classify or match short strings with typos and spelling variants, such as 'Antalya' against 'Antlya' or truncated product names.
easydocumented fact - Re-derive constraint signs whenever a feature is redefined, reversed or re-encoded
A model has a monotone constraint vector, and a refactor replaces a feature with its reversed definition, such as discount percent for fare or days left for days elapsed.
easypractitioner heuristic - Compute distances from coordinates with a great-circle formula in radians, not Euclidean on degrees
A model needs distance between origins and destinations, or a nearest-station feature, from latitude and longitude columns, and the step runs on raw degree values.
easypractitioner heuristic