Core day 23 of 60 · Week 5 · Part I · application depth

Classical Forecasting

Planning wants to replace the seasonal-naive forecast of monthly passengers with ETS or ARIMA. What do the simple baselines assume, what do the classical models add, and what comparison on one valid cutoff would show whether the extra machinery earns its place?

By the end: On one historical cutoff you fit naive, seasonal naive, drift, ETS and ARIMA using only earlier data, report error against the seasonal-naive baseline in business and scaled units, and explain why a lower AIC does not settle the choice.

At a glance

Simple baselines set the bar; ETS and ARIMA must clear it on a held-out cutoff, and neither residual checks nor AIC can show that they do.

Baselines first
Naive, seasonal naive and drift cost nothing and cannot overfit.
ETS
Level, trend and season updated with fading weights; damping flattens long horizons.
ARIMA
Difference, then regress on own lags (AR) and past errors (MA).
AIC is not accuracy
In-sample fit plus a penalty, within one model class; never ETS versus ARIMA.
One valid cutoff
Fit before it, score after it, same months for all; Day 25 repeats it.

Recall · 5 min

Without looking back: which difference orders gave the smallest standard deviation for the logged monthly series on Day 22, and what is the sign of the lag-1 ACF of an over-differenced series?

Open due reviews

Concept · 10 min

Every forecasting method is a claim about the world. Seasonal naive says "next July looks like last July"; naive says "tomorrow looks like today"; drift adds the average change. They cost nothing and set the bar (Day 1). ETS and ARIMA add machinery, but both fit only the series' own past: neither knows about holidays, prices or a competitor unless you tell it.

Different baselines win on different series, and every error grows with the horizon

Predict first

You select the "Trend" series with a horizon of 12. Which baseline has the lowest average error?

Text description of the figure

A selector for a seeded series (seasonal, trend, random walk, noisy level, trend plus season) and sliders for the forecast horizon and the moving-average window. One chart shows the series with the four baseline forecasts made at the last origin; another shows rolling-origin mean absolute error at each horizon step; a table ranks the baselines by average error and gives each one's ratio to the naive method.

Five methods, five claims about the world Overview

Mean
The average of all history. Wins on a stable level with only noise.
Naive
Tomorrow looks like today. Wins when the series wanders like a random walk, no season.
Seasonal naive
Next July looks like last July: the usual bar for a strong, stable season.
Drift
Last value plus h times the average change so far. Wins on a steady trend.
ETS
A level, a trend and a season that update as data arrive.
ARIMA
The dependence on own lags and past errors, after differencing.

The first four fit no parameters, so they cannot overfit. Any model has to beat the matching one at the horizon you use.

The baseline race, refitted at every origin Illustration

One origin: naive and drift ignore the season, seasonal naive copies last year without its growth, Holt-Winters carries level, trend and season.

Over 25 rolling origins: Holt-Winters 2.3, Seasonal naive 6.2, Drift 9.0, Naive 9.4 (mean MAE over 12 steps). Holt-Winters beats seasonal naive at 12 of 12 steps.

Naive
Drift
Seasonal naive
Holt-Winters

Synthetic monthly series (seeded): 8 years with trend 0.5 per month, a 12-month season and autocorrelated noise. Every model is refitted at each origin from month 60 on, using only earlier months.

Four methods on one seeded monthly series. Left, one origin's forecasts; right, error by horizon pooled over rolling origins. The method that matches the structure wins.

Exponential smoothing weights by age of the observation Chart

alpha = 0.2 (slow, smooth)
alpha = 0.5
alpha = 0.8 (fast, close to naive)

Weights are alpha times (1 - alpha) to the power of the age: geometric, summing to one. A high alpha forgets quickly.

What phi does to a fitted Holt forecast Illustration

phi = 1 (alpha 0.8, beta 0.3 by in-sample SSE)
phi = 0.9 (alpha 0.8, beta 0.2 by in-sample SSE)
phi = 0.8 (alpha 0.6, beta 0.3 by in-sample SSE)

At 3 months ahead the three forecasts differ by 5.6; at 36 months by 93.5. With phi below 1 the added trend shrinks by a factor phi every step, so the forecast levels off at a ceiling (dotted) instead of growing for ever.

Synthetic trending series (seeded): slope 1.6 per month plus noise sd 3. Fitted with the course's Holt code; the ceiling is the forecast 600 steps ahead. Which phi forecasts better is a backtest question, not something this picture can show.

Same history, three damping values. alpha and beta are fitted by in-sample SSE and phi is fixed at 1, 0.9 and 0.8 so its effect shows alone (AutoETS fits phi too). The forecasts agree for a few months, then part ways; a damped trend levels off at a ceiling.

What ARIMA(p,d,q)(P,D,Q)m does Diagram

Seriesregular, positive or logged
Difference: d, Dremoves level and season (Day 22)
AR terms: p, Pregress on own lags
MA terms: q, Qregress on past forecast errors
Undo differencesforecast on the original scale

Capital letters are the seasonal parts at lags m, 2m; Day 22's airline model is ARIMA(0,1,1)(0,1,1)12. AutoARIMA chooses d by repeated KPSS tests and D by a seasonal-strength measure first, then p, q, P and Q by AICc within ARIMA.

Twelve held-out months, one cutoff (today's run) Chart

Naiveignores the season38.5 MAE
Drift38.8 MAE
Seasonal naivethe bar: MASE 0.829.2 MAE
AutoETS, ETS(M,Ad,M)MASE 0.525.8 MAE
AutoARIMAMASE 0.48; its lead over ETS is one draw5.4 MAE

One valid cutoff: every model is fitted on months 1-120 only, forecasts the same 12 months, and is scored on months 121-132, never seen in fitting, in MAE and MASE against seasonal naive. Synthetic monthly passengers.

AIC is not accuracy: what the number can and cannot tell you Overview

Ranks one class on one dataset
In-sample fit plus a parameter penalty: an estimate of out-of-sample log-likelihood loss if the family is right.
Differences, not levels
A gap below about 2 is weak evidence (rule of thumb); a value such as 148 means nothing on its own.
Not across classes or scales
ETS vs ARIMA, d = 1 vs d = 0, log vs raw: the likelihood itself changes.
Not forecast error
Silent about MAE at h = 12, about a break after the window, about data the model has not seen.

Formally, AIC estimates the expected Kullback-Leibler loss on a fresh sample from the same process. Choosing between classes, or proving accuracy, is the backtest's job.

Exponential smoothing (ETS): level, trend and season, each updated with weights that fade with age; a damped trend flattens long horizons.
ARIMA: differencing (Day 22) plus regression on the series' own lags (AR) and past errors (MA).
Diagnostics: residuals should be white noise; passing is necessary, not proof of accuracy.
AICc ranks models of one class on the same data; it is not forecast error. Judge on a held-out cutoff.

Assumptions

The series is regularly spaced and has at least two to three full seasonal cycles before the cutoff; with less, seasonal parameters are poorly identified.
The structure is stable enough that a model fitted on the past describes the test year; a break inside the training window is absorbed into the level or the order.
Multiplicative components need strictly positive data; otherwise use additive terms or a transformation.

Application and workplace decision · 10 min

The code simulates 11 years of monthly passengers whose growth fades as the market saturates, with a season whose swings scale with the level. Before you run it, rank the five methods and say which ones will beat seasonal naive.

Fit naive, seasonal naive, drift, AutoETS and AutoARIMA with statsforecast on the first 120 months only.
Score all five on the final 12 months: MAE, and MASE scaled by the in-sample seasonal-naive error.
Print the selected ETS and ARIMA forms and their AICc values.

Your answer to the task (decision, method, baseline). Write it before reading the interpretation.

Python

import numpy as np
import pandas as pd
from statsforecast import StatsForecast
from statsforecast.models import AutoARIMA, AutoETS, Naive, RandomWalkWithDrift, SeasonalNaive

rng = np.random.default_rng(23)
n, h, m = 132, 12, 12                                  # 11 years of synthetic monthly passengers; 12-month horizon
t = np.arange(n)
level = 200 + 90 * (1 - np.exp(-t / 45))               # growth that fades as the market saturates
y = level * (1 + 0.18 * np.sin(2 * np.pi * t / m) + 0.06 * np.cos(4 * np.pi * t / m)) + rng.normal(0, 6, n)
df = pd.DataFrame({"unique_id": "route_A", "ds": pd.date_range("2014-01-01", periods=n, freq="MS"), "y": y})

train, test = df.iloc[:-h], df.iloc[-h:]               # ONE cutoff: the last 12 months are never seen in fitting
sf = StatsForecast(models=[Naive(), SeasonalNaive(season_length=m), RandomWalkWithDrift(),
                           AutoETS(season_length=m), AutoARIMA(season_length=m)], freq="MS", n_jobs=1)
sf.fit(train)
fc = sf.predict(h=h).merge(test[["ds", "y"]], on="ds")

scale = np.abs(train["y"].values[m:] - train["y"].values[:-m]).mean()   # in-sample seasonal-naive MAE (MASE scale)
for model in ["Naive", "SeasonalNaive", "RWD", "AutoETS", "AutoARIMA"]:
    mae = np.abs(fc["y"] - fc[model]).mean()
    print(f"{model:14s} MAE {mae:6.2f}  MASE {mae / scale:4.2f}")
ets, arima = sf.fitted_[0, 3].model_, sf.fitted_[0, 4].model_
print("selected ETS:", ets["method"], "| ARIMA (p,q,P,Q,m,d,D):", arima["arma"])
print(f"AICc: ETS {ets['aicc']:.0f}, ARIMA {arima['aicc']:.0f} (different model classes: not comparable)")

This code is executed in the repository's checks (pnpm lesson:py / lesson:sql). It is not run on this site; copy it into your notebook.

Technical depth · optional · 32 min in total

Open only what you need. Nothing here is required to complete the day; the assessment covers the core above.

Exponential smoothing: the recursion, unrolled by hand5 minThe one-line update, four clinic weeks pushed through it on paper, the derivation that turns it into geometric weights, and when it is enough.

Simple exponential smoothing Formula

\ell_t = \alpha\, y_t + (1-\alpha)\,\ell_{t-1}, \qquad \hat y_{T+h} = \ell_T

$\ell_t$: the level after seeing observation t
$\alpha$: smoothing parameter between 0 and 1: how fast the level may move
$\hat y_{T+h}$: a flat forecast at the last level for every horizon

alpha near 1 is naive, near 0 the long-run mean. Neither alpha nor l_0 is set by hand: they minimise the squared one-step errors or maximise the state-space likelihood (Hyndman et al., 2002), which also yields AICc and intervals. Holt adds a trend with its own beta; Holt-Winters a season with gamma.

Worked by hand: the SES recursion on four clinic weeks (alpha = 0.5) Table

Week t	Visits y_t	Level update	Level l_t
0	-	starting level (given)	10
1	12	0.5 x 12 + 0.5 x 10	11
2	8	0.5 x 8 + 0.5 x 11	9.5
3	14	0.5 x 14 + 0.5 x 9.5	11.75
4	10	0.5 x 10 + 0.5 x 11.75	10.875
forecast	-	flat at the last level	10.875 for every week ahead

Illustrative flu-clinic visits. Unrolling the same recursion gives the weights: 0.5 x 10 + 0.25 x 14 + 0.125 x 8 + 0.0625 x 12 + 0.0625 x 10 = 10.875 - geometric, summing to one. statsmodels returns exactly this forecast.

Unrolling the recursion: SES is a weighted average of the past Formula

\ell_T = \sum_{j=0}^{T-1} \alpha(1-\alpha)^{j}\, y_{T-j} \;+\; (1-\alpha)^{T}\,\ell_0

$\alpha(1-\alpha)^{j}$: the weight on the observation j steps old: substitute l_(t-1) into l_t again and again
$(1-\alpha)^{T}\ell_0$: what is left of the starting level; it fades geometrically and is why the initial state matters only on short series
$\sum$: the weights plus the leftover sum to one, so the forecast is a proper weighted average

Derivation: l_T = alpha y_T + (1 - alpha) l_(T-1) = alpha y_T + alpha (1 - alpha) y_(T-1) + (1 - alpha)^2 l_(T-2) = ... Each substitution adds one older observation with one more factor (1 - alpha).

Alpha trades reaction speed against noise chasing Illustration

Before the shift: one-step error 1.95 on a flat level. After it: 22 steps to cover 90% of the jump.

Before the shift: one-step error 1.90 on a flat level. After it: 7 steps to cover 90% of the jump.

Before the shift: one-step error 2.31 on a flat level. After it: 2 steps to cover 90% of the jump.

Synthetic data (seeded): level 50 for 30 months, then 65, noise sd 2.5; dashed line = true level. Lines are one-step-ahead SES forecasts; the shaded span is the time to cover 90% of the jump. Steps to 90% are counted on the noise-free copy, where they equal ln 0.1 / ln(1 - alpha) rounded up.

Simple exponential smoothing on a series with one level shift. Small alpha ignores noise and lags the shift; large alpha catches it in two steps and jumps with every point.

SES fits a wandering level; its residuals say which component to add Diagram

Fit SESflat forecast at the last level
Residual mean far from 0?it lags a trend
- yes: to Add a trend: Holt
no
Pattern at lag m?residual ACF
- yes: to Add a season: Holt-Winters
no
SES is enoughmature product demand, stable clinic visits
Add a trend: Holtits own weight beta
Add a season: Holt-Wintersits own weight gamma

After adding a component, ask the next question down. A trend makes SES lag behind by a roughly constant amount, so the residual mean moves away from zero; a season leaves a pattern at lag m. Each failure is the cue for the next component.

Holt, damping and the ETS taxonomy5 minThe three component recursions, damping worked by hand, the loop behind every ETS form, and how the taxonomy names every combination.

Holt-Winters (additive): three recursions, one forecast Formula

\begin{aligned} \ell_t &= \alpha(y_t - s_{t-m}) + (1-\alpha)(\ell_{t-1} + b_{t-1}) \\ b_t &= \beta^{*}(\ell_t - \ell_{t-1}) + (1-\beta^{*}) b_{t-1} \\ s_t &= \gamma(y_t - \ell_{t-1} - b_{t-1}) + (1-\gamma) s_{t-m} \end{aligned}

$\ell_t$: level: SES applied to the deseasonalised value, with last step's trend added to the old level
$b_t$: trend: SES applied to the change in level, with its own weight beta*
$s_t$: season: SES applied to what the level and trend do not explain, updated once per period m
$\hat y_{T+h}$: forecast l_T + h b_T + s_(T+h-m(k+1)) with k = floor((h - 1) / m): the latest estimate of the same season

Every line is the same move - new evidence times a weight plus the old state times one minus it. Holt drops the season; the multiplicative season divides instead of subtracting.

Damped trend forecast Formula

\hat y_{T+h} = \ell_T + (\phi + \phi^{2} + \dots + \phi^{h})\, b_T

$b_T$: the trend (slope) at the forecast origin
$\phi$: damping parameter, usually between 0.8 and 0.98; 1 means no damping

The sum converges, so the forecast levels off at a finite value instead of growing for ever.

Damping by hand: l_T = 100, b_T = 5, phi = 0.9 (illustrative) Table

Horizon h	Damped forecast	Trend added at step h	Undamped Holt
1	100 + 0.9 x 5 = 104.5	4.5	105
2	104.5 + 0.81 x 5 = 108.55	4.05	110
3	108.55 + 0.729 x 5 = 112.2	3.65	115
12	132.3	1.41	160
24	141.4	0.40	220
Ceiling	l_T + b_T phi / (1 - phi) = 145	0	none: keeps climbing

Each step adds the trend shrunk by one more factor phi, so the increments form a geometric series and the forecast approaches 145. A saturating subscription market has this shape; whether damping forecasts better is still a backtest question.

Every ETS form is one loop: the one-step error drives every state update Loop

States l, b, slevel, trend, season at t-1forecast ↓
One-step forecastl + b + s from one season agoobserve y_t ↓
Innovation e_ty_t minus the forecastweight it ↓
Update each statestate + alpha, beta or gamma x errornext period ↓
↺ back to 1 · States l, b, s

This innovations (state-space) form is why the likelihood, AICc and the interval formulas follow from one equation (FPP3, 8.5). Multiplicative errors change the likelihood, not the point forecasts of the same components.

The ETS taxonomy: error, trend, season Table

Component	Options	Letters
Error	Additive or multiplicative	A, M
Trend	None, additive, additive damped	N, A, Ad
Season	None, additive, multiplicative	N, A, M
Example	Multiplicative error, damped trend, multiplicative season	ETS(M,Ad,M)

Automatic selection fits the admissible combinations and keeps the lowest AICc. Multiplicative parts need strictly positive data.

Which ETS form to try first Diagram

Seasonal pattern?
- yes: to Season: M if swings grow, else A
no
Trend present?
- yes: to Damped trend (Ad)
no
No trend (N)simple or seasonal smoothing
- to Race against seasonal naive
Season: M if swings grow, else A
Damped trend (Ad)safer at long horizons
Race against seasonal naivesame origins and horizon

After choosing the season, continue with the trend question. A starting point for judgement; automatic selection by AICc within ETS does the same search.

AR, MA and ARIMA: the equations, and a forecast by hand5 minThe backshift form, an AR(1) power-price forecast computed on paper, the two-line proof that ARIMA(0,1,1) is SES, and where estimates go wrong.

AR, MA and ARIMA in one line with the backshift operator Formula

\underbrace{(1-\phi_1 B - \dots - \phi_p B^{p})}_{\text{AR}(p)}\;\underbrace{(1-B)^{d}}_{\text{I}(d)}\, y_t \;=\; c + \underbrace{(1+\theta_1 B + \dots + \theta_q B^{q})}_{\text{MA}(q)}\,\varepsilon_t

$B$: the backshift: B y_t = y_(t-1), so (1 - B) y_t is the first difference of Day 22
$(1-B)^{d}\,y_t$: y'_t, the series after d ordinary (and D seasonal) differences: the AR and MA terms act on it
$\text{AR}(1)$: y_t = c + phi y_(t-1) + e_t: today is a shrunk copy of yesterday plus a shock; stationary when |phi| < 1
$\text{MA}(1)$: y_t = mu + e_t + theta e_(t-1): a shock echoes for exactly one more step, then is gone
$\mu = c/(1-\phi)$: the long-run mean of a stationary AR(1); forecasts decay toward it
$\varepsilon_t$: white-noise innovations, estimated by maximum likelihood together with phi and theta

Seasonal ARIMA multiplies in the same polynomials in B^m (orders P, D, Q). Every forecast is computed by running this recursion forward with future shocks set to their mean, zero.

Worked by hand: an AR(1) forecast of a power price (illustrative) Table

Horizon h	Forecast c + phi y_hat	Gap to mean 50	Error variance	95% half-width
0 (last price)	60	10	-	-
1	20 + 0.6 x 60 = 56	6 = 10 x 0.6	16	7.84
2	20 + 0.6 x 56 = 53.6	3.6 = 10 x 0.6^2	16 (1 + 0.36) = 21.76	9.14
3	20 + 0.6 x 53.6 = 52.16	2.16	23.83	9.57
4	51.30	1.30	24.58	9.72
infinity	50 = c / (1 - phi)	0	sigma^2 / (1 - phi^2) = 25	9.80

Hypothetical day-ahead price deviation model with c = 20, phi = 0.6, sigma = 4 (EUR/MWh). The gap to the mean shrinks by phi each step; the variance adds sigma^2 phi^(2j) per step and levels off. Recomputed in Python with the same recursion.

Why ARIMA(0,1,1) is simple exponential smoothing, derived Formula

y_{t+1} = y_t + \varepsilon_{t+1} + \theta\varepsilon_t \;\Rightarrow\; \hat y_{t+1} = y_t + \theta\,(y_t - \hat y_t) = (1+\theta)\, y_t - \theta\, \hat y_t

$\varepsilon_t = y_t - \hat y_t$: the last shock is known once y_t is observed: it is the one-step forecast error
$\hat y_{t+1}$: set the future shock to zero; what remains is the last value corrected by theta times the last error
$\alpha = 1 + \theta$: rename and the line reads alpha y_t + (1 - alpha) y_hat_t: the SES update, so theta = alpha - 1

The same algebra gives the other rows of the equivalence table. A random walk (theta = 0) is SES with alpha = 1, which is naive.

Where ARIMA estimates go wrong Overview

Estimation
Maximum likelihood, computed recursively. MA terms make it non-linear: iterative, and it can stop at a local optimum.
AR root near the unit circle
Slow decay and huge long-horizon uncertainty; the limit is Day 21's random walk.
MA coefficient near -1
A unit root in the MA polynomial: the over-differencing signature of Day 22.
Fitted across a break
An AR(1) absorbs the step into a phi near 1 (Day 21).

AICc: what it computes, worked, and why it is not accuracy4 minThe formula term by term, two models scored by hand from their log-likelihoods, and an AICc ranking that disagrees with test error.

AIC and AICc: what the number is made of Formula

\text{AIC} = -2\log L(\hat\theta) + 2k, \qquad \text{AICc} = \text{AIC} + \frac{2k(k+1)}{T-k-1}

$\log L(\hat\theta)$: the maximised log-likelihood of the training data: in-sample fit only
$k$: every estimated quantity: smoothing or ARMA parameters, initial states and the error variance
$2k$: the penalty that makes AIC an estimate of out-of-sample log-likelihood loss under the model's own assumptions
$\frac{2k(k+1)}{T-k-1}$: small-sample correction; it matters when k is not tiny next to T and vanishes as T grows
$\text{BIC}$: -2 log L + k log T: a heavier penalty once T > 7, so it picks smaller models

The likelihood must be of the same data on the same scale. A differenced series, a log transform or a different error form changes L itself, which is why AICc never compares ETS with ARIMA or a model with d = 1 against d = 0.

Worked AICc: SES against Holt on 48 weeks of sign-ups (synthetic) Table

Quantity	SES, ETS(A,N,N)	Holt, ETS(A,A,N)
Estimated: parameters + variance	alpha, l_0 + sigma^2: k = 3	alpha, beta, l_0, b_0 + sigma^2: k = 5
Log-likelihood	-70.87	-70.83 (the trend adds almost nothing)
AIC = -2 log L + 2k	141.74 + 6 = 147.74	141.66 + 10 = 151.66
Correction 2k(k+1) / (T - k - 1)	24 / 44 = 0.55	60 / 42 = 1.43
AICc	148.29 (lower: preferred)	153.09
BIC = -2 log L + k log 48	153.36	161.02

A seeded random walk (numpy default_rng(3), shock sd 1) standing in for a SaaS product's weekly sign-ups (T = 48). statsmodels 0.15 ETSModel reports exactly these AIC, AICc and BIC values. Two extra parameters bought 0.04 of log-likelihood, so every criterion prefers SES.

AICc ranks ETS variants differently from test error (hands-on lab) Table

ETS (error / trend / season)	AICc	Test MAE
add / add-damped / mul	827.2 (best)	5.7
mul / add-damped / mul	832.3	6.0
add / add / mul	844.2	4.4 (best)
mul / add / mul	846.7	4.7

Four lowest-AICc variants from statsmodels on the training window. Spearman correlation of AICc with test MAE over all variants is +0.42, and a 12-month test is itself noisy.

ETS or ARIMA?2 minWhat each family describes, and why they are compared on a backtest only.

ETS and ARIMA side by side Comparison

Criterion	ETS	ARIMA
Describes	Components: level, trend, season	Dependence on lags and past errors
Needs stationarity	No	Yes, after differencing
Multiplicative season	Directly, with the M option	Through a log transform
Compare by AICc with the other	No: different likelihood	No: compare on a backtest

Linear ETS models are special cases of ARIMA; multiplicative ETS models have no ARIMA equivalent. A backtest compares them; AICc does not.

Where ETS and ARIMA describe the same model Table

ETS	Equivalent ARIMA	Parameter link
ETS(A,N,N)	ARIMA(0,1,1)	theta = alpha - 1
ETS(A,A,N)	ARIMA(0,2,2)	theta1 = alpha + beta - 2, theta2 = 1 - alpha
ETS(A,Ad,N)	ARIMA(1,1,2)	phi1 = phi (the damping parameter)
ETS(M,., .)	none	Multiplicative errors have no ARIMA form

From FPP3 section 9.10. The overlap is why ARIMA(0,1,1) behaves like SES, and why the two families can tie on a backtest.

Residual checks before the backtest2 minFour checks on the innovation residuals, in order, and what each one can and cannot tell you.

Residual checks, in order Sequence

Plot the innovation residuals
No trend or seasonal pattern. A change in spread after a date is often a break (Day 21).
Mean near zero
A non-zero mean is bias, the cheapest failure to fix: add it back or fix the model.
ACF inside the band
Isolated bars are expected; a pattern at lag 1 or m is not (at m: a missed season).
Ljung-Box with model df
Joint test of the first h autocorrelations (Day 22).
Then the backtest
Passing all four is necessary; beating seasonal naive is the claim.

In a claims or call-volume series, the dated break behind a spread change is often a new product or policy.

One cutoff versus many3 minFive origins showing how much the single-cutoff result of the concept section can move.

One cutoff is one draw: MAE at five origins (hands-on lab) Grid

Method →
Cutoff (12 months ahead) ↓	Seasonal naive	AutoETS	AutoARIMA
2021-12	5.50	4.20	4.50
2022-06	7.50	4.40	7.10
2022-12	7.30	4.70	7.20
2023-06	8.20	5.10	5.30
2023-12	9.20	5.80	5.40
Mean	7.53	4.85	5.90

Twelve months ahead from each origin, refit every time; a stronger shade is a larger error. ARIMA beats ETS only at the last origin, the one the application used. Keep the simplest method whose lead over seasonal naive is larger than its spread across origins.

Inside the automatic searchesadvanced3 minWhat AutoETS and AutoARIMA try by default, and the one argument that silently switches seasonality off.

What the automatic searches do by default (statsforecast 2.0.1) Table

Knob	Default	What it means
AutoETS model	'ZZZ'	Error, trend and season each chosen by AICc among admissible forms
AutoETS damped	None	Damped and undamped trends are both tried
AutoARIMA d	test='kpss', max_d=2	Repeated KPSS tests pick the ordinary differences
AutoARIMA D	seasonal_test='seas', max_D=1	A seasonal-strength measure decides one seasonal difference
AutoARIMA orders	stepwise=True, ic='aicc'	Greedy search over p, q (max 5) and P, Q (max 2); not exhaustive
season_length	1	Must be set (12 for monthly); left at 1 every seasonal term is off

Read from the installed constructor signatures. The search follows the Hyndman-Khandakar algorithm of R's forecast package. Stepwise search can miss the best order; stepwise=False is slower and still ranks by AICc, not forecast error. Pin the library version before reading fitted internals such as model_['arma']: they are not documented API.

Interval widths, and when not to use classical models3 minHow forecast variance grows with the horizon for each method, and five situations where an expert reaches for something else.

How the h-step forecast variance grows, by method Table

Method	Variance of the h-step error	Grows like
Naive	sigma^2 h	Linearly in h
Seasonal naive	sigma^2 (k + 1), k = floor((h - 1) / m)	In steps, once per season
Drift	sigma^2 h (1 + h / (T - 1))	Faster than naive
SES, ETS(A,N,N)	sigma^2 (1 + alpha^2 (h - 1))	Linearly, slower when alpha is small

Residual-based formulas assuming uncorrelated, normal errors (FPP3 sections 5.5 and 8.7). Check coverage on a backtest (Day 26) before quoting them.

When an expert would not reach for ETS or ARIMA Overview

Known future drivers
Holidays, prices, schedules: a regression or ML model can use them (Day 24).
Thousands of related series
One global model shares strength; per-series fits are slow and noisy.
Several seasonal periods
Hourly or daily data: MSTL, TBATS or Fourier terms instead.
Intermittent counts
Mostly zeros: Croston-type or count models (Day 26).
Very short history
Under two seasons: seasonal naive or a pooled model beats a fitted one.

Memory anchor

Baselines first, classes never compared on AIC, and every claim stated against seasonal naive on held-out months.

Useful for

Setting the benchmark that every later model on Days 24 to 28 has to beat, with a scale-free error.
Short, regular, well-behaved business series (monthly retail sales, insurance claim counts, call-centre volumes, energy use) where ETS or ARIMA are strong and cheap.
Producing coherent multi-step forecasts and prediction intervals from one fitted model per series.

Not suitable for

Series driven by holidays, prices or events that are known in advance but not in the series' own history; use regressors (Day 24).
Choosing between model classes on AIC, or claiming forecast accuracy from in-sample fit or residual tests alone.

Traps

Comparing AIC across ETS and ARIMA

The two classes compute likelihoods differently and ARIMA is fitted on a differenced series, so AICc values are on different scales. The model with the lower number looks better and is chosen for the wrong reason.

How to avoid it

Compare classes on held-out error from the same origins and horizons; use AICc only inside one class.

Beating the wrong baseline

A model that beats naive on a seasonal series proves little because naive ignores the season. Reports cite the easy comparison and the real bar, seasonal naive, goes unmentioned.

How to avoid it

Report the model against the baseline that matches the structure, at each horizon, as a ratio.

Senior reasoning

A senior fits baselines first and keeps them in every report. They use ETS and ARIMA as strong, cheap, auditable candidates, read diagnostics as prompts rather than verdicts, and refuse to compare information criteria across classes. They state the cutoff, the horizon and the scale, and they say how many origins a claim rests on.

Interview question

"How do you decide between ETS and ARIMA for a monthly demand series?" Structure the answer.

Outline your answer first.

Recall cards scheduled after completion

Name the four classic baselines and the ETS ingredients, including what a damped trend does.
Why is AIC not accuracy, and what do you compare instead?

Reviewed on a 1 · 3 · 7 · 14 · 30-day schedule; a miss returns the card to 1 day.

Prerequisites and related concepts

References

Forecasting: Principles and Practice (3rd edition) — Rob J Hyndman, George Athanasopoulos, 2021core
The four benchmark methods; exponential smoothing with trend, damped trend and the ETS taxonomy; AIC, AICc and BIC for model selection within the ETS class; ARIMA and seasonal ARIMA; the statement that AICc cannot compare ETS with ARIMA; residual requirements; MASE scaling. (Sections 5.2, 5.4, 5.8, 8.2, 8.4, 8.6, 9.9 and 9.10)
A state space framework for automatic forecasting using exponential smoothing methods — Rob J. Hyndman, Anne B. Koehler, Ralph D. Snyder, Simone Grose, 2002optional
Exponential smoothing methods as innovations state space models, which gives likelihood-based selection and prediction intervals for ETS.
Another look at measures of forecast accuracy — Rob J. Hyndman, Anne B. Koehler, 2006optional
The MASE definition - errors scaled by the in-sample mean absolute error of the (seasonal) naive method.
statsmodels ETSModel and ETSResults (documentation of the tsa.exponential_smoothing.ets module) — statsmodels developersoptional
ETSModel arguments (error, trend, damped_trend, seasonal) and the AICc, aic and forecast members of the results used in the lab.
StatsForecast models reference (Nixtla documentation) — Nixtlaoptional
Constructor arguments used in the code (season_length for SeasonalNaive, AutoETS and AutoARIMA; RandomWalkWithDrift alias RWD).
Automatic Time Series Forecasting: The forecast Package for R — Rob J. Hyndman, Yeasmin Khandakar, 2008optional
The automatic ARIMA algorithm (unit-root tests for d, a seasonal test for D, stepwise AICc search over orders) that AutoARIMA follows, and automatic ETS selection.
StatsForecast core class (Nixtla documentation) — Nixtlaoptional
StatsForecast(models, freq, n_jobs), fit, predict, forecast and cross_validation, and the required unique_id, ds and y columns.

Optional sections

Deep dive · 10 min

Weights. Unrolling simple exponential smoothing gives forecasts as a weighted average of past values with weights $\alpha(1-\alpha)^j$ : geometric, summing to one. A high $\alpha$ forgets quickly and tends toward naive; a low $\alpha$ approaches a long-run mean.

Taxonomy. An ETS model has an error type (additive or multiplicative), a trend (none, additive, damped) and a season (none, additive, multiplicative). Multiplicative components need positive data. The state-space form gives likelihood, AICc and prediction intervals.

ETS and ARIMA overlap, not nest. Linear exponential smoothing models are special cases of ARIMA: for example ETS(A,N,N) is ARIMA(0,1,1) with moving-average coefficient alpha minus 1, and ETS(A,A,N) is ARIMA(0,2,2). Nonlinear ETS models have no ARIMA equivalent and many ARIMA models have no ETS counterpart, so neither class includes the other.

Reading statsforecast output. sf.fit(train) stores one fitted object per series and model in fitted_; for AutoETS the model_ dictionary has method, aic, aicc and residuals, and for AutoARIMA arma lists the orders as (p, q, P, Q, m, d, D). These keys were read from statsforecast 2.0.1 by running it; they are fitted-model internals, not part of the documented API, so pin the version if you depend on them. sf.forecast(df, h) fits and predicts in one call, and cross_validation returns forecasts for each window with the cutoff column (the last training date).

Drift and damping on a fading trend. The simulated series has growth that decays. Drift keeps the old slope and over-forecasts (MAE 38.8). A damped additive trend has the slope decay with factor phi per step, which matches how saturation looks; the cost is one more parameter that can be poorly identified on short data.

When classical models are not enough. Holidays and price changes known in advance, many related series, hundreds of thousands of rows or non-linear structure call for regression on features (Day 24); sparse series for intermittent-demand methods (Day 26).

Hands-on lab: Five origins, AIC versus test error, and the residuals of the chosen ARIMA · 25 min

Run the code and answer: (1) At how many of the five origins does each model beat seasonal naive, and what is the spread of each model's MAE across origins? (2) Does the ETS variant with the lowest AICc also have the lowest test MAE, and what does Spearman correlation between the two columns say? A 12-point test is itself noisy: how does that limit the conclusion? (These AICc values come from statsmodels and are not comparable with the statsforecast AICc of the application.) (3) The ARIMA residuals fail Ljung-Box at 5%: name two structural reasons you would check before changing the model. (4) Change the simulation so that growth does not fade and see which model ranking changes.

import itertools
import warnings

import numpy as np
import pandas as pd
from scipy.stats import spearmanr
from statsforecast import StatsForecast
from statsforecast.models import AutoARIMA, AutoETS, SeasonalNaive
from statsmodels.stats.diagnostic import acorr_ljungbox
from statsmodels.tsa.exponential_smoothing.ets import ETSModel

warnings.filterwarnings("ignore")
n, h, m = 132, 12, 12
t = np.arange(n)

def make(seed):
    rng = np.random.default_rng(seed)
    level = 200 + 90 * (1 - np.exp(-t / 45))
    return level * (1 + 0.18 * np.sin(2 * np.pi * t / m) + 0.06 * np.cos(4 * np.pi * t / m)) + rng.normal(0, 6, n)

df = pd.DataFrame({"unique_id": "A", "ds": pd.date_range("2014-01-01", periods=n, freq="MS"), "y": make(23)})
sf = StatsForecast(models=[SeasonalNaive(season_length=m), AutoETS(season_length=m), AutoARIMA(season_length=m)],
                   freq="MS", n_jobs=1)

# 1) one cutoff is one draw: five rolling origins, 12 months ahead each, refit at every origin
cv = sf.cross_validation(df=df, h=h, n_windows=5, step_size=6)
err = cv.assign(**{k: (cv["y"] - cv[k]).abs() for k in ["SeasonalNaive", "AutoETS", "AutoARIMA"]})
by_origin = err.groupby("cutoff")[["SeasonalNaive", "AutoETS", "AutoARIMA"]].mean()
print(by_origin.round(1).to_string())
print("mean MAE:", by_origin.mean().round(2).to_dict())

# 2) AIC chooses among ETS variants on the training window; test MAE ranks them differently
train, test = df["y"].values[:-h], df["y"].values[-h:]
rows = []
for err_t, trend, damped, seas in itertools.product(["add", "mul"], [None, "add"], [False, True], ["add", "mul"]):
    if trend is None and damped:
        continue
    r = ETSModel(train, error=err_t, trend=trend, damped_trend=damped, seasonal=seas, seasonal_periods=m).fit(disp=False)
    rows.append((f"{err_t}/{trend}{'-damped' if damped else ''}/{seas}", r.aicc, np.abs(test - r.forecast(h)).mean()))
tab = pd.DataFrame(rows, columns=["ETS(error/trend/season)", "AICc", "test MAE"]).sort_values("AICc")
print(tab.head(4).round(1).to_string(index=False))
print(f"best by AICc: test MAE {tab.iloc[0, 2]:.1f}; best test MAE of all {tab['test MAE'].min():.1f}; "
      f"Spearman(AICc, test MAE) = {spearmanr(tab['AICc'], tab['test MAE'])[0]:+.2f}")

# 3) residual check of the ARIMA model: model_df = number of estimated ARMA terms
sf.fit(df.iloc[:-h])
arma = sf.fitted_[0, 2].model_["arma"]
res = sf.fitted_[0, 2].model_["residuals"][m + 1:]
p = acorr_ljungbox(res, lags=[2 * m], model_df=arma[0] + arma[1] + arma[2] + arma[3]).iloc[0, 1]
print(f"ARIMA (p,q,P,Q) = {arma[:4]}: residual Ljung-Box({2 * m}) p = {p:.2f}")

Interview practice · 10 min

A retailer's monthly demand forecast uses seasonal naive. A vendor offers an "AI forecaster" and shows it has a lower error on last year's data. What do you ask before agreeing to switch?

Answer as you would in the interview.

Content version 6 · last verified 2026-10-18 · depends on statsforecast, statsmodels, numpy, pandas

1 Recall · 5 min

2 Concept · 10 min