Classical ML, validation, metrics, interpretation, debugging · insight-0171

Do not read lasso's choice among correlated features as evidence that the others do not matter

Situation
Lasso keeps one of several correlated features (competitor price, fare level, season) and zeroes the rest; someone proposes dropping them from the data.
Action
Check selection stability across resamples, consider elastic net to keep correlated groups together, and describe selected features as one of a correlated set.
Reason
With correlated predictors, lasso tends to pick one somewhat arbitrarily; the elastic-net penalty encourages a grouping effect.
Boundary
Sparse linear models used for feature screening or explanation.

Symptom or risk

The selection flips between bootstrap samples or retrains; stakeholders get contradictory stories about which driver matters.

For pure prediction with stable data, lasso's arbitrary pick may predict just as well.

Elastic net adds a mixing hyperparameter and selects less aggressively.

Running selection on all data before cross-validation, which also inflates the score.

When predictors are highly correlated, the lasso tends to select only one from the group, while the elastic net exhibits a grouping effect.
Regularization and Variable Selection Via the Elastic Net