Statistics, experimentation, causal reasoning · insight-1018
Fit regression discontinuity locally and show how the estimate moves with the bandwidth
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist
- Industries
- aviation, banking, public-sector
Summary: situation, action, reason, boundary
Situation
A global fourth-degree polynomial in loyalty points gives a significant jump at the cutoff, while the fitted curve swings wildly at the edges.
Action
Fit local linear or quadratic models in a window around the cutoff, plot binned means with the fitted lines, and report estimates for several bandwidths. State whether the conclusion changes across them.
Reason
High-order polynomials of the running variable give heavy weight to distant points, and estimators based on them can mislead.
Boundary
Any regression discontinuity analysis with a continuous running variable.
Symptom or risk
Observations far from the threshold drive the fit, so the jump can be an artefact of the polynomial shape and can be significant by chance.
Exceptions
Very few observations near the cutoff, where a wider window is unavoidable and the weakness must be stated.
Trade-offs
Narrow windows have less data and wider intervals; wide windows have more bias risk.
Pitfalls
Choosing the bandwidth that gives the best p-value.
Lessons and concepts
Claims and their sources
Controlling for high-order (third, fourth or higher) polynomials of the forcing variable in regression discontinuity can give misleading causal estimates, and local linear or quadratic estimators are recommended.