Statistics, experimentation, causal reasoning · insight-0029

Curate a small, relevant donor pool for synthetic control instead of including every available unit

Situation
A synthetic control for one route uses all 200 other routes as donors and only 12 months of pre-period data.
Action
Keep donors that are structurally similar and unaffected by the intervention or spillovers, document exclusions before estimation, and check that results survive leaving out each major donor.
Reason
The risk of over-fitting grows with the donor pool when the pre-period is short, and dissimilar donors can create interpolation bias even with a close fit.
Boundary
Market-level, route-level or regional evaluations with many potential comparison units.

Symptom or risk

With many donors and a short history, weights can fit pre-period noise, producing a near-perfect fit and an unreliable counterfactual.

Long pre-periods and genuinely homogeneous donors reduce the cost of a larger pool.

A smaller pool may fit less tightly; it is easier to defend.

Including donors that absorb the treated unit's lost demand, which inflates the estimated effect.

Over-fitting risk increases with donor pool size, especially with few pre-intervention periods; donor units must be chosen judiciously, and dissimilar donors can cause interpolation bias.
Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects