Statistics, experimentation, causal reasoning · insight-0028
Show the synthetic control's pre-period fit and a backdated placebo before showing the effect
- Evidence type
- documented fact
- Difficulty
- hard
- Roles
- Data scientist
- Industries
- aviation, public-sector, retail
Summary: situation, action, reason, boundary
Situation
A synthetic control built from other airports estimates that a new hub incentive raised passenger numbers, and the deck shows only the post-launch gap.
Action
Plot treated and synthetic series over the whole pre-period with a fit metric, then rerun with the intervention backdated into the pre-period and confirm no spurious effect appears. If the fit is poor, do not report a synthetic-control estimate.
Reason
The estimator's bias depends on how well the weighted donors reproduce the treated unit; a long pre-period cannot rescue a bad fit.
Boundary
Single treated units such as one market, one airport, one region or one product line.
Symptom or risk
If the synthetic unit never tracked the treated unit before launch, the post-period gap measures misfit, not the incentive.
Exceptions
When the treated unit lies outside the range of all donors (largest hub, unique market), the method cannot fit and another design is needed.
Trade-offs
Strict fit requirements rule out some appealing case studies.
Pitfalls
Tuning predictors and the pre-period window until the post-period gap looks largest.
Lessons and concepts
Claims and their sources
When the synthetic control fits the treated unit poorly before the intervention, its use is not recommended because of potential bias; a long pre-intervention period cannot drive down bias if the fit is bad, and backdating the intervention gives an in-time placebo check.