Practical insights

Results

19 results (published only)

• Feature engineering and preprocessing

  Skip feature scaling for tree ensembles and spend the effort on features that carry signal

  You are about to add StandardScaler or MinMaxScaler in front of a random forest, gradient boosting model or single decision tree.

  easydocumented fact

• Feature engineering and preprocessing

  Scale features before k-NN, SVMs and k-means or the largest unit decides the distance

  A distance- or kernel-based model uses features in mixed units, such as average fare in currency, load factor in 0-1 and days to departure in days.

  easydocumented fact

• Feature engineering and preprocessing

  Use RobustScaler when a few extreme values distort mean-and-variance scaling, then decide about the extremes separately

  A scaled feature has a handful of huge values, such as a 10x passenger outlier or a corporate group booking.

  easydocumented fact

• Feature engineering and preprocessing

  Put every fitted preprocessing step inside the cross-validated pipeline, not before it

  Your notebook imputes, scales or selects features on the whole dataframe, then calls crossvalscore on the result.

  easydocumented fact

• Feature engineering and preprocessing

  Group infrequent levels before one-hot encoding a high-cardinality categorical

  One-hot encoding a feature such as travel agency or product SKU would create thousands of sparse columns.

  easydocumented fact

• Feature engineering and preprocessing

  Do not give a linear model ordinal codes for unordered categories

  A nominal feature such as market or payment method is label-encoded as 0, 1, 2 and fed to a linear, kernel or distance model.

  easypractitioner heuristic

• Feature engineering and preprocessing

  Let gradient boosting route missing values natively before building an imputer

  You plan an imputation step in front of HistGradientBoosting, LightGBM or XGBoost.

  easydocumented fact

• Feature engineering and preprocessing

  Replace sentinel values like -999 before they reach linear or distance models

  A source system encodes missing or not-applicable as -999, 9999 or 0.

  easypractitioner heuristic

• Feature engineering and preprocessing

  Guard ratio features against zero and tiny denominators

  You build ratios such as revenue per seat, cancellations per booking or spend per visit.

  easypractitioner heuristic

• Feature engineering and preprocessing

  Encode cyclical calendar features so December sits next to January for linear and distance models

  Month, weekday or hour enters a linear, kernel or distance model as an integer.

  easypractitioner heuristic

• Feature engineering and preprocessing

  Remove features that are recorded because of the outcome, not before it

  A candidate feature is filled in by a process that happens after or because of the target event, such as a cancellation reason, refund amount or collections flag.

  easydocumented fact

• Feature engineering and preprocessing

  Keep raw identifiers out of the feature set and use them for grouping instead

  Customer id, flight number, device id or row index ends up as a model input.

  easypractitioner heuristic

• Feature engineering and preprocessing

  Take clipping limits from training-fold quantiles and record them as model parameters

  You winsorize heavy-tailed features or targets to limit the influence of extremes.

  easypractitioner heuristic

• Feature engineering and preprocessing

  Standardize features before PCA unless they share a meaningful common unit

  You run PCA on features measured in different units, such as revenue, seat counts and percentages.

  easydocumented fact

• Feature engineering and preprocessing

  Scale sparse matrices without centering them

  Your features are a sparse matrix, for example one-hot route-by-channel indicators or TF-IDF text features, and you add StandardScaler.

  easydocumented fact

• Feature engineering and preprocessing

  Choose Yeo-Johnson over Box-Cox when a feature can be zero or negative

  A PowerTransformer with method box-cox was fitted on fare or revenue columns, and later extracts contain zero fares, refunds or negative adjustments.

  easydocumented fact

• Feature engineering and preprocessing

  Expect out-of-range values after min-max scaling and decide between clipping and alerting

  Inputs are min-max scaled for a neural network or linear model, and serving data includes values beyond the training minimum or maximum.

  easydocumented fact

• Feature engineering and preprocessing

  Skip hand-made bins before histogram boosting, which already bins every feature

  Continuous features are bucketed into deciles before LightGBM or HistGradientBoosting in the name of robustness.

  easydocumented fact

• Feature engineering and preprocessing

  Re-derive constraint signs whenever a feature is redefined, reversed or re-encoded

  A model has a monotone constraint vector, and a refactor replaces a feature with its reversed definition, such as discount percent for fare or days left for days elapsed.

  easypractitioner heuristic