Statistics, experimentation, causal reasoning · insight-0023
Leave mediators out of the model when you want the total effect of a decision
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Data scientist, Data analyst
- Industries
- retail, e-commerce, banking
Summary: situation, action, reason, boundary
Situation
To estimate the effect of a loyalty email on spend, the analyst adds "sessions after the email" as a control variable.
Action
Classify each candidate covariate as pre-treatment (possible confounder), post-treatment mediator, or outcome-related collider. Adjust only for pre-treatment confounders when estimating the total effect; analyse mediation separately with its own assumptions.
Reason
Adjustment answers the question "holding this fixed". Holding the mechanism fixed asks a different, usually less useful, question.
Boundary
Marketing, pricing, operations and product analyses using regression or matching.
Symptom or risk
Controlling for a variable on the causal path removes part of the effect being measured, so the email looks useless even if it works through more visits.
Exceptions
Direct-effect questions (does the email work other than through visits) intentionally condition on the mediator and need stronger assumptions.
Trade-offs
Fewer covariates can look less rigorous to reviewers; explain the DAG.
Pitfalls
Using variables measured after treatment as "controls" because they are available in the same table.