Feature engineering and preprocessing · insight-1116

Fit depth-one boosting first to learn whether interactions matter before hand-crafting crosses

Situation
A team plans many hand-built interaction features for a gradient-boosted model without evidence that interactions carry signal.
Action
Fit the same boosting setup with depth one (one feature per tree) and with depth two to four on identical folds; if the gap is negligible, ship the additive model, otherwise restrict interactions with interaction_cst to approved groups.
Reason
A single-split tree uses one feature, so the sum is additive in the features; the gap to deeper trees measures what interactions add at that capacity.
Boundary
HistGradientBoosting, LightGBM and XGBoost on tabular data.

Symptom or risk

Weeks go into crosses that add serving complexity and leakage surface for no accuracy gain, or a real interaction is missed.

Few rows, where deeper trees lose to noise and mask a real interaction.

Extra tuning runs; additive models are also easier to explain and constrain.

Comparing at different learning rates or tree counts, so the depth effect is confounded with capacity.

HistGradientBoosting's interaction_cst restricts which features may be used together along a branch; one singleton set per feature forbids all interactions.
Ensembles: Gradient boosting, random forests, bagging, voting, stacking (scikit-learn User Guide 1.11)