Skip to content
ML Academy

project-018 · Forecasting · e-commerce

Intermittent weekly demand for a gift-ware wholesaler: beating the zero forecast honestly

For each active product of a UK online gift-ware retailer with many wholesale customers, how many units should be available each week so that the chosen service level is met without building dead stock? Most product-weeks sell nothing, and a few sell hundreds of units to one customer.

Licence
CC BY 4.0 (as stated on the UCI page)
Reference run
none

At a glance

  • Data

    Online Retail II (UCI) (public) · 1,067,371 transaction lines, 2009-12-01 to 2011-12-09 (page states 01/12/2009-09/12/2011).

    Details on data

  • Baseline

    Zero forecast (to expose the MAE trap), 52-week mean, and seasonal naive (same week last year) for products with at least one year of history.

    Details on baseline

  • Validation

    Weekly rolling origins over the final 26 weeks; horizons 1-4 weeks (replenishment lead time assumed 2-4 weeks, a stated assumption).

    Details on validation

  • Metric

    MASE and RMSSE (scaled by in-sample naive error), cumulative bias, pinball loss at 0.9 scaled by in-sample mean absolute difference;

    Details on metric

Success criterion

On a weekly rolling-origin backtest over the last 26 weeks of the data, the selected method beats both the zero forecast and the 52-week mean on scaled pinball loss at the 0.9 quantile (the stocking quantile) and on cumulative bias, for intermittent and lumpy products separately. Winning on MAE alone does not count.

Context

Intermittent series break familiar habits: MAPE is undefined on zero weeks, and the zero forecast often minimises MAE because the median weekly demand is zero, while it would never be stocked. Croston-type methods smooth demand sizes and intervals separately; the SBA correction reduces Croston's bias; TSB lets the forecast decay when a product stops selling (obsolescence).

Data

Dataset
Online Retail II (UCI) (public)
Licence
CC BY 4.0 (as stated on the UCI page)
Size
1,067,371 transaction lines, 2009-12-01 to 2011-12-09 (page states 01/12/2009-09/12/2011).
Verification
identity verified

Access

Direct download from the UCI page, no login. Cache the file and record its checksum. Fallback if UCI is unavailable: none with the same licence; pause the project rather than using an unverified mirror.

Grain

Raw: one row per invoice line. Modelling grain after aggregation: StockCode x ISO week, units = sum of Quantity on non-cancelled invoices. Weeks before a product's first sale are not zeros (not yet listed).

ColumnMeaning
InvoiceNoInvoice id; a leading 'c' marks a cancellation (page definition)
StockCodeProduct code, documented as 5-digit; inspect codes that do not fit (non-product lines)
QuantityUnits per line; negative values accompany cancellations or returns
InvoiceDateTransaction timestamp
UnitPricePrice per unit in sterling
CustomerIDCustomer id; missing on some lines
CountryCustomer's country of residence

EDA and data-quality rules

  • Exclude cancellation invoices and negative quantities from demand; report their share and whether they cluster on a few products (returns are a separate series, not negative demand).

  • Remove lines whose StockCode does not match a product pattern (postage, fees, manual adjustments) after listing them; never drop them silently.

  • Start each product's series at its first sale; zeros before that are listing artefacts, not demand.

  • Check for multiple descriptions per StockCode and multiple codes per description before aggregating.

  • Flag weeks where one customer accounts for more than half of a product's units (wholesale lumps).

Baseline

Zero forecast (to expose the MAE trap), 52-week mean, and seasonal naive (same week last year) for products with at least one year of history.

Features

Per product: weekly units, inter-demand intervals, demand sizes; ADI (average inter-demand interval) and CV² of nonzero sizes to classify products as smooth, erratic, intermittent or lumpy. No cross-sectional features are needed for the statistical candidates. For slices: product age, price band, share of top customer.

Candidate methods

  • Croston with SBA bias correction (statsforecast CrostonSBA)

    The standard intermittent-demand method with the documented bias of plain Croston corrected; a strong, interpretable reference.

  • TSB (statsforecast TSB)

    Updates the demand probability every week, so forecasts fall when a product stops selling; gift-ware has many seasonal and discontinued lines where obsolescence matters.

Validation design

Weekly rolling origins over the final 26 weeks; horizons 1-4 weeks (replenishment lead time assumed 2-4 weeks, a stated assumption). Parameters selected on the preceding 26 weeks only. Products enter the evaluation only if they had at least 13 weeks of history at the origin, and the eligibility rule is applied as of each origin (no selecting products on future activity).

Leakage checks

Product eligibility and classification (ADI/CV²) are recomputed at each origin from history only. Selecting "products active in the final year" up front is survivorship leakage; test by comparing results with the origin-wise rule.

Bias and representativeness

Cumulative bias per product class; Croston-type point forecasts are rates, so compare them with the mean, not with the median, and show why MAE prefers zero.

Robustness checks

Re-run excluding the five largest wholesale customers to see how much of the error is driven by single orders; vary the smoothing parameter grid.

Technical metrics

MASE and RMSSE (scaled by in-sample naive error), cumulative bias, pinball loss at 0.9 scaled by in-sample mean absolute difference; quantiles for Croston-type methods come from an explicit assumption (for example a Poisson or negative binomial around the rate), which must be stated.

Business interpretation

Simulated fill rate and average on-hand units for an order-up-to policy using the 0.9 quantile, with hypothetical holding cost per unit-week; zero forecast shows 0% fill rate despite good MAE.

Error slices

  • Product class: smooth, erratic, intermittent, lumpy.

  • New products (fewer than 26 weeks of history) vs established.

  • Pre-Christmas weeks (gift-ware peak) vs the rest.

  • Weeks dominated by a single wholesale order.

Implementation steps

  • Download and cache; profile cancellations, non-product codes, missing CustomerID; write the cleaning rules as tests.

  • Aggregate to product-week with series starting at first sale; compute ADI/CV² classes as of each origin.

  • Run baselines and both candidates in a rolling-origin loop; store forecasts with origin and horizon.

  • Score point and quantile metrics; demonstrate the zero-forecast MAE trap in one table.

  • Simulate the order-up-to policy with the hypothetical holding cost; report fill rate vs stock.

  • Write the recommendation per product class and the obsolescence rule (when to stop forecasting a product).

Reproduction

Learner script with a cached download, checksum check and fixed parameter grid; a 200-product sample runs on a laptop in minutes, the full catalogue in a longer optional run.

Limitations

  • Sales are not demand: stock-outs and wholesale minimum order sizes are not visible in the data.

  • Two years of history give at most one prior Christmas season for seasonal comparison.

  • Quantiles from Croston-type rates depend on a distributional assumption that must be checked per class.

Production and monitoring

Weekly batch per product with class-specific methods; monitor cumulative bias and the share of products switching class; retire products when TSB probability falls below a threshold agreed with buyers; route new products to a cold-start rule (category average) until they have enough history.

Interview questions

  • Why can a zero forecast have the best MAE on intermittent demand, and which metric would you use instead?

  • When does TSB beat Croston-SBA, and how would you show it on this data?

  • How do you avoid survivorship bias when choosing which products to evaluate?

Postmortem and alternatives

Likely findings: most of the error comes from a handful of wholesale lumps no univariate method can foresee; class assignments move between origins. Alternative: forecast customer-level order probability for the top wholesale accounts plus a Croston-type model for the remainder, or aggregate to monthly buckets (temporal aggregation) for the slowest movers.

Rubric3 criteria

CriterionMeetsExceeds
Data cleaningCancellations, non-product codes and pre-listing zeros handled with tests and reported shares.Shows how each cleaning choice changes the class mix and the winner.
Metric choiceDemonstrates the zero-forecast MAE trap and evaluates with scaled and quantile metrics.Links the pinball quantile to a stated service-level target and shows sensitivity.
Honest evaluationOrigin-wise eligibility and classification; no future-based product selection.Quantifies how much survivorship selection would have flattered results.

References5