Time series and forecasting · insight-0231
Difference only as much as a unit-root test and backtest support
- Evidence type
- documented fact
- Difficulty
- medium
- Roles
- Data scientist
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A model is fitted on second differences because the first-differenced plot 'still looked wavy'.
Action
Use a stationarity test such as a sequence of KPSS tests to pick the number of differences, inspect the ACF after each step, and confirm the choice by comparing backtest error and interval width.
Reason
Applying more differences than required induces false dynamics or autocorrelations, so the guidance is to difference as little as necessary.
Boundary
ARIMA modelling and any pipeline that differences targets before ML.
Symptom or risk
Extra differencing induces spurious autocorrelation, inflates forecast variance and widens intervals for no gain.
Exceptions
Tests have low power on short series; when tests disagree, let backtest accuracy decide.
Trade-offs
Under-differencing leaves a trend the model cannot handle; over-differencing adds noise.
Pitfalls
Differencing seasonal data at lag 1 and lag m and then also adding seasonal terms.
Lessons and concepts
Claims and their sources
Applying more differences than required induces false dynamics or autocorrelations; unitroot_ndiffs uses a sequence of KPSS tests to choose the number of differences.