Skip to content
ML Academy

project-021 · Forecasting · logistics

Next-day hourly bike rentals: which exogenous inputs exist at forecast time

A bike-share operator plans tomorrow's rebalancing trucks and staff shifts at 18:00 today. How many rentals will occur in each hour of tomorrow, and how much of the apparent value of weather features survives once you use only the weather information that would actually exist at 18:00?

Licence
CC BY 4.0 (as stated on the UCI page)
Reference run
none

At a glance

  • Data

    Bike Sharing (UCI), hourly file (public) · The UCI page lists 17,389 instances; hour.csv is 1.1 MB and day.csv 56.2 KB, covering 2011-2012. Count rows on load and record the number.

    Details on data

  • Baseline

    Hour-of-week seasonal naive (same hour, same weekday, last week) and the same hour on the previous available day adjusted to the 18:00 cutoff (only hours up to 18:00…

    Details on baseline

  • Validation

    Rolling daily origins at 18:00 for every day of 2012 (or every third day to save time), expanding window from 2011-01-01;

    Details on validation

  • Metric

    WAPE and MAE per hour of day, MASE scaled by hour-of-week seasonal naive, bias.

    Details on metric

Success criterion

The report shows three clearly separated numbers on the same rolling backtest: (1) a model without weather, (2) the same model with realised weather (an upper bound that is not deployable), and (3) the model with weather degraded to mimic forecast error under stated assumptions. The deployed recommendation is based on (1) or (3), never (2), and beats hour-of-week seasonal naive on WAPE across 2012 origins.

Context

The dataset records realised weather for each hour. Backtesting with it treats tomorrow's weather as known, which inflates accuracy in exactly the way a stakeholder will not see in production. Holidays and working days are known in advance; weather is not; casual and registered users sum to the target.

Data

Dataset
Bike Sharing (UCI), hourly file (public)
Licence
CC BY 4.0 (as stated on the UCI page)
Size
The UCI page lists 17,389 instances; hour.csv is 1.1 MB and day.csv 56.2 KB, covering 2011-2012. Count rows on load and record the number.
Verification
identity verified

Access

Direct download from the UCI page, no login (hour.csv about 1.1 MB). Cache and checksum it.

Grain

One row per date x hour (dteday, hr); target cnt = casual + registered rentals in that hour.

ColumnMeaning
dteday / hrDate and hour (0-23)
holiday / workingday / weekday / season / yr / mnthCalendar fields, known in advance
weathersitObserved weather category 1 (clear) to 4 (severe); realised, not forecast
temp / atemp / hum / windspeedObserved, normalised temperature, feeling temperature, humidity and wind speed
casual / registeredRentals by user type; they sum to cnt and are outcomes, never features
cntTarget: total rentals in the hour

EDA and data-quality rules

  • Check for missing hours (some date-hours may have no row); decide whether a missing hour means zero rentals or missing data, and document it.

  • Assert casual + registered = cnt on every row, then drop both from the feature set.

  • Plot hourly profiles for working days vs non-working days; expect commute peaks only on working days.

  • Note that weather values are normalised; keep the normalisation rule when simulating forecast error.

Baseline

Hour-of-week seasonal naive (same hour, same weekday, last week) and the same hour on the previous available day adjusted to the 18:00 cutoff (only hours up to 18:00 today are known).

Features

Available at 18:00: calendar fields, hour, lags of cnt at least 6 hours old for tomorrow's first hour and correspondingly older for later hours (use same-hour values from today if before 18:00, otherwise from yesterday or last week), rolling daily totals, and today's observed weather up to 18:00. Not available: tomorrow's realised weather. Scenario (3) replaces it with realised weather plus noise whose size is an explicit, labelled assumption, because the dataset contains no archived weather forecasts.

Candidate methods

  • LightGBM with calendar and cutoff-safe lags only

    Uses only information that exists at 18:00; the honest deployable baseline for a model.

  • LightGBM plus a weather-forecast proxy (realised weather with assumed forecast error)

    Estimates how much weather could help if a forecast feed were bought; the oracle version is reported only as an upper bound.

Validation design

Rolling daily origins at 18:00 for every day of 2012 (or every third day to save time), expanding window from 2011-01-01; each origin forecasts hours 0-23 of the next day. Tune on 2011 Q4 origins only. Because 2012 volumes are higher than 2011, check whether trees under-forecast a growing level and, if so, model the ratio to a trailing level.

Leakage checks

Three leakage tests: casual/registered excluded; no lag younger than the 18:00 cutoff; realised next-day weather present only in the labelled oracle run. Show the WAPE of each leaking variant so the size of each leak is visible.

Bias and representativeness

Bias by hour and by weathersit; heavy-rain hours are rare, so check whether the model systematically over-forecasts them.

Robustness checks

Vary the assumed weather-forecast error (small, medium, large) and report how the value of weather changes; re-run with the trend adjustment switched off.

Technical metrics

WAPE and MAE per hour of day, MASE scaled by hour-of-week seasonal naive, bias.

Business interpretation

Peak-hour error at commute hours (8, 17, 18), which drives truck and staff planning; translate into hypothetical cost per bike short or idle at stations.

Error slices

  • Hour of day (commute peaks vs night).

  • Working day vs non-working day vs holiday.

  • Weather category 3-4 vs 1-2.

  • First vs second half of 2012 (growth).

Implementation steps

  • Download, assert the cnt identity, list missing hours.

  • Build cutoff-safe lag features with a test that checks the 18:00 rule.

  • Run seasonal naive, LightGBM without weather, oracle-weather and noisy-weather variants in one rolling loop.

  • Tabulate the three-number comparison and the leakage variants.

  • Write the recommendation: deploy without weather or buy a forecast feed, with the evidence and assumptions.

Reproduction

Learner script, CPU only, seconds to minutes; download cached with checksum.

Limitations

  • No archived weather forecasts: scenario (3) is a simulation of forecast error, not evidence of a real feed's value.

  • Two years of data from one system; annual seasonality is estimated from a single repeat.

  • Rentals are not demand: empty or full stations suppress observed rentals.

Production and monitoring

Daily 18:00 batch; if a weather forecast feed is bought, archive every forecast as issued so future backtests use the true information set. Monitor hourly bias and the forecast-feed latency; fall back to the no-weather model if the feed is late.

Interview questions

  • Your backtest uses realised weather. What does that number represent, and what would you report instead?

  • Which features in this dataset are known at forecast time, and which are outcomes?

  • How would you decide whether buying a weather-forecast feed is worth it?

Postmortem and alternatives

Likely findings: the oracle-weather gain is large, the noisy-weather gain is much smaller, and most remaining error sits in bad-weather peaks. Alternative: probabilistic scenarios over weather (forecast ensembles) feeding a staffing decision.

Rubric3 criteria

CriterionMeetsExceeds
Availability reasoningEach feature classified as known in advance, observed by cutoff, or unavailable.Availability table reviewed against the code by a second person.
Three-number comparisonNo-weather, oracle and noisy-weather results on the same backtest.Sensitivity over forecast-error assumptions with a buy/no-buy threshold.
Leak demonstrationcasual/registered and cutoff leaks shown with their inflated scores.Leak tests run automatically in the pipeline.

References3