Statistics, experimentation, causal reasoning · insight-0024
Treat a clean pre-trend plot as weak support for parallel trends, not proof
- Evidence type
- documented fact
- Difficulty
- hard
- Roles
- Data scientist
- Industries
- aviation, public-sector, retail
Summary: situation, action, reason, boundary
Situation
A difference-in-differences analysis of a new airport lounge on one hub shows no significant pre-period difference in trends against other hubs, and the report calls parallel trends "confirmed".
Action
Show the event-study plot with intervals, argue parallel trends from domain knowledge about why units were treated, and run a sensitivity analysis that shows how large a trend violation would change the conclusion.
Reason
Parallel trends concerns the unobserved post-period counterfactual; pre-period data can only partly speak to it, and pre-tests are often underpowered.
Boundary
Policy, pricing and product rollouts evaluated with DiD or event studies.
Symptom or risk
The pre-trend test may simply lack power, and analyses that proceed only after passing it are biased, so the estimate inherits an untested assumption.
Exceptions
Randomized rollouts across units, where parallel trends holds by design.
Trade-offs
Sensitivity analysis gives a range rather than one number, which some audiences find less satisfying.
Pitfalls
Choosing the control group after seeing which one produces the cleanest pre-trend.
Lessons and concepts
Claims and their sources
A non-significant pre-trend does not imply parallel trends holds; pre-trend tests often have low power, and conditioning on passing them introduces pre-test bias.