Classical ML, validation, metrics, interpretation, debugging · insight-0172
Interpret predictions, not individual coefficients, when features are strongly collinear
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist, Data analyst
- Industries
- aviation, retail
Summary: situation, action, reason, boundary
Situation
Coefficients change sign or size when a correlated feature is added, removed or the data period shifts.
Action
Check variance inflation factors and coefficient stability across resamples, report combined effects of correlated groups, and explain the model through predictions or grouped contributions.
Reason
Collinearity inflates the variance of individual coefficients without much effect on predictions within the data's range.
Boundary
Linear and logistic models with related drivers such as price, fare level and season.
Symptom or risk
Business explanations swing between versions although predictions are nearly identical; trust erodes.
Exceptions
If the decision requires an individual effect (pricing elasticity), you need a design or data that separates the drivers, not a reinterpretation.
Trade-offs
Dropping features stabilizes coefficients but may move omitted-variable bias elsewhere.
Pitfalls
Reading coefficients as causal effects; in the synthetic airline data fares are set higher when demand is expected to be high, so the fare coefficient is confounded.
Lessons and concepts
Claims and their sources
Collinearity reduces the accuracy of coefficient estimates and inflates their standard errors; the variance inflation factor assesses it (ISL section 3.3.3).