Time series and forecasting · insight-1242
Pick the MinT covariance estimator by the number of series relative to the history length
- Evidence type
- documented fact
- Difficulty
- hard
- Roles
- Data scientist, ML engineer
- Industries
- retail, aviation, telecom
Summary: situation, action, reason, boundary
Situation
A retail group reconciles forecasts for 4,000 item-store series with 104 weeks of history, using MinT with the sample covariance of one-step forecast errors.
Action
Use a shrinkage estimator, or a variance-scaled or structure-based weighting, when the number of series is large relative to the history. Keep OLS as a floor and compare reconciled and base forecasts at every level over the same rolling origins.
Reason
FPP3 names the sample covariance as not a good estimator when the number of bottom series is large relative to the series length, and recommends shrinkage towards a diagonal matrix. With 104 error vectors, a 4,000-dimensional sample covariance built from 104 error vectors has rank at most 104.
Boundary
Large retail, network or telecom hierarchies reconciled with covariance-based methods.
Symptom or risk
With far more series than observations the sample covariance is poorly estimated and cannot even have full rank, so reconciled forecasts can end up worse than the base forecasts.
Exceptions
Small hierarchies with long histories can use the sample covariance.
Trade-offs
Diagonal-leaning estimators ignore correlations between series that may carry real accuracy gains.
Pitfalls
Reporting reconciliation gains only at the top of the hierarchy while the bottom level, where decisions are made, gets worse.
Lessons and concepts
Claims and their sources
FPP3 lists OLS, variance-scaled WLS, structural WLS and sample-covariance or shrinkage options for MinT, calls the sample covariance a poor estimator when the number of bottom-level series is large relative to the series length, and uses a shrinkage estimator (towards a diagonal matrix) instead.