Skip to content
ML Academy

project-020 · Forecasting · energy

Day-ahead quarter-hour load with daily, weekly and annual seasonality

An energy supplier serving a portfolio of business clients must submit tomorrow's 96 quarter-hour volumes by a day-ahead gate (assumed 10:00 the day before, a hypothetical setting). Which forecasting approach handles the overlapping daily, weekly and annual cycles well enough to reduce imbalance volume against a weekly seasonal naive?

Licence
CC BY 4.0 (as stated on the UCI page)
Reference run
none

At a glance

  • Data

    ElectricityLoadDiagrams20112014 (UCI) (public) · 370 client columns x 140,256 quarter-hour records (listed as 140,256 'features' on the UCI page), 2011-2014; one semicolon-separated text file.

    Details on data

  • Baseline

    Weekly seasonal naive (same quarter-hour, same weekday, one week earlier) and daily seasonal naive using the most recent available day (two days earlier because of the…

    Details on baseline

  • Validation

    Daily rolling origins at 10:00 over the final year, expanding window; each origin forecasts the 96 intervals of the next calendar day using data up to 00:00 of the…

    Details on validation

  • Metric

    MAE and WAPE per quarter-hour of day, MASE scaled by in-sample weekly seasonal-naive error, peak-interval absolute error.

    Details on metric

Success criterion

Across rolling daily origins over the final year, the chosen method reduces mean absolute imbalance (kWh per day) versus weekly seasonal naive by a margin larger than twice its origin-level standard error, and does not lose on Portuguese public holidays or daylight-saving days by more than 10%.

Context

Load data at sub-hourly resolution carries several seasonal periods at once (96 intervals per day, 672 per week, roughly 35,000 per year). Single-season methods cannot represent all of them; options include MSTL decomposition, TBATS, or regression on Fourier terms plus lags. The horizon starts 14 hours after the gate and ends 38 hours after it; with meter data assumed to arrive one day late, the newest usable value is from 00:00 of the origin day, so every target is 24 to 48 hours away from the last observation.

Data

Dataset
ElectricityLoadDiagrams20112014 (UCI) (public)
Licence
CC BY 4.0 (as stated on the UCI page)
Size
370 client columns x 140,256 quarter-hour records (listed as 140,256 'features' on the UCI page), 2011-2014; one semicolon-separated text file.
Verification
identity verified

Access

Direct download from UCI, no login; the zip is about 250 MB (678 MB unzipped), so download once, convert to Parquet, and record the checksum. Fallback if the download is blocked: none verified; do not substitute a mirror without checking its provenance.

Grain

Raw: quarter-hour timestamp (Portuguese local time) x client, values in kW (divide by 4 for kWh per interval). Modelling grain: portfolio total of clients with complete history from 2012-01-01, per quarter hour.

ColumnMeaning
timestamp (first column)Quarter-hour in Portuguese local time, 'yyyy-mm-dd hh:mm:ss'
MT_001 ... MT_370Client consumption in kW for the interval (column naming to be confirmed on load)
zero values before a client's startClient not yet connected (page note), not zero consumption
DST change daysMarch: 1:00-2:00 recorded as zeros; October: 1:00-2:00 holds two hours aggregated (page note)

EDA and data-quality rules

  • Drop clients with pre-start zeros after 2012-01-01 from the portfolio, or start their contribution at connection; document which.

  • Repair DST artefacts explicitly: impute the March zero hour from adjacent weeks; split the October aggregated hour; never let these enter training as real values.

  • Convert kW to kWh per interval before computing energy totals.

  • Check each day has exactly 96 intervals after the DST repair.

Baseline

Weekly seasonal naive (same quarter-hour, same weekday, one week earlier) and daily seasonal naive using the most recent available day (two days earlier because of the gate).

Features

For LightGBM: quarter-hour of day, weekday, Portuguese public holiday flags from the holidays package (known in advance), Fourier terms for the annual cycle, lags that end before 00:00 of the origin day (for example 2 days and 7 days at the same quarter hour), and rolling means ending at the data cutoff. No temperature is available in this dataset; record that as a limitation rather than inventing a weather feed.

Candidate methods

  • MSTL decomposition (periods 96 and 672) with AutoETS on the seasonally adjusted series (statsforecast)

    Handles several seasonal periods explicitly and remains interpretable: each seasonal component can be plotted and checked.

  • LightGBM with calendar, annual Fourier terms and gate-safe lags

    Can learn holiday and weekday interactions that additive decomposition misses; must respect the data cutoff at 00:00 of the origin day. TBATS is a documented alternative left out for runtime at this resolution.

Validation design

Daily rolling origins at 10:00 over the final year, expanding window; each origin forecasts the 96 intervals of the next calendar day using data up to 00:00 of the origin day (meter data assumed to arrive with a one-day lag, a stated assumption). Hyperparameters fixed from the preceding year.

Leakage checks

Unit test that every lag feature refers to a timestamp before 00:00 of its origin day; check holiday flags come from a calendar, not from observed low load.

Bias and representativeness

Bias by quarter-hour of day and by weekday; a model that is unbiased overall can systematically under-forecast the morning ramp.

Robustness checks

Evaluate separately on DST days and on holidays; re-run with 2013 as the evaluation year to see whether the winner changes.

Technical metrics

MAE and WAPE per quarter-hour of day, MASE scaled by in-sample weekly seasonal-naive error, peak-interval absolute error.

Business interpretation

Daily imbalance volume (sum of absolute interval errors in kWh) and an imbalance cost with hypothetical asymmetric prices (shortfall dearer than surplus); show how the asymmetry would shift the optimal bid away from the mean forecast (links to project-022).

Error slices

  • Quarter-hour of day (night, morning ramp, midday, evening peak).

  • Weekday vs weekend vs public holiday.

  • DST change days.

  • Season (summer vs winter).

Implementation steps

  • Download once, convert to Parquet, write tests for 96 intervals per day and DST repairs.

  • Select the portfolio and aggregate in kWh; plot one week and one year to see the seasonal layers.

  • Implement both baselines with the gate-aware data cut.

  • Fit MSTL + AutoETS and LightGBM in the daily rolling loop; store forecasts with origin.

  • Score per quarter-hour and slice; compute daily imbalance volume and hypothetical cost.

  • Recommend a method and a fallback, with the DST and holiday handling written down.

Reproduction

Learner script with a cached, checksummed download; a 3-month evaluation window runs on a laptop in minutes, the full year optionally.

Limitations

  • No weather variable: temperature-driven load changes are unexplained and show up as residual error.

  • The portfolio is an arbitrary subset of anonymous clients, not a real supplier's book.

  • The gate time and data lag are assumptions; a different market design changes the horizon.

Production and monitoring

Daily job before the gate; publish 96 values with a version id; monitor interval-level bias and daily imbalance; alert when the DST repair rule fires; fallback to weekly seasonal naive if the job fails or data arrive incomplete.

Interview questions

  • How would you model three seasonal periods at 15-minute resolution, and what does each option cost?

  • Your forecast is issued at 10:00 for tomorrow. Which lags can you use?

  • The March DST day shows an hour of zeros. What do you do and why?

Postmortem and alternatives

Likely findings: holidays and bridge days dominate the largest errors; MSTL handles the weekly shape well but misses holiday interactions. Alternatives: add a weather feed with archived forecasts, or a hybrid (MSTL seasonal components as features for LightGBM).

Rubric3 criteria

CriterionMeetsExceeds
Data repairDST artefacts, pre-start zeros and units handled with tests.Quantifies how unrepaired DST days would have biased the evaluation.
Gate-aware validationDaily origins with the data cutoff enforced and tested.Shows the error difference between a gate-aware and a naive next-day setup.
Seasonality reasoningSeasonal components plotted and checked against the error by quarter hour.Explains which seasonal layer each candidate fails on and why.

References5