project-022 · Forecasting · aviation

Meal uplift per flight: prediction intervals and a newsvendor quantile from asymmetric costs

Catering must fix the number of meals per flight two days before departure (an assumed lead time). Too few meals cost a service-recovery voucher; too many are wasted. Which quantity should be ordered for each flight, and are the prediction intervals behind that quantity trustworthy?

Dataset: Synthetic airline simulator output (flight_day and booking_curve)
Licence: Synthetic data generated by the academy's own simulator (project code); no third-party data.
Reference run: executed
Lessons: day 13 day 16 day 25 day 26 day 28

Data
Synthetic airline simulator output (flight_day and booking_curve) (synthetic) · 20,824 flight-days with 13 booking snapshots each (2022-01-01 to 2024-12-31).
Details on data
Baseline
Point forecast = cumulative bookings at d = 2 times the historical ratio of passengers to d = 2 bookings for the same flight and weekday (last eight departures).
Details on baseline
Validation
Rolling origin, monthly, across 2024: train on all departures before the origin minus the 8-week calibration window, calibrate on that window, test on the next month.
Details on validation
Metric
Empirical coverage at 80% and 95%, mean interval width (sharpness), pinball loss at 0.81, PIT histogram from the quantile model.
Details on metric

Success criterion

On a rolling backtest over 2024 synthetic departures: 80% and 95% intervals from the chosen method reach empirical coverage within 3 percentage points of nominal overall and within 6 points in every market slice; and the newsvendor order rule has lower total cost than ordering the point forecast and than point plus a flat 5% buffer, under the stated hypothetical costs.

Context

With shortage cost $c_u$ and overage cost $c_o$ , the cost-minimising order is the $q^* = c_u / (c_u + c_o)$ quantile of the passenger distribution (newsvendor critical fractile). With hypothetical costs of 25 EUR per missing meal and 6 EUR per wasted meal, $q^* = 25/31 \approx 0.81$ . A good quantile therefore matters more than a good mean. Unlike demand forecasting, the target here is boarded passengers, which the physical seat limit genuinely caps, so capacity censoring is part of the target, not a flaw in it. This is the specification for reference implementation r05.

Data

Dataset: Synthetic airline simulator output (flight_day and booking_curve) (synthetic)
Licence: Synthetic data generated by the academy's own simulator (project code); no third-party data.
Size: 20,824 flight-days with 13 booking snapshots each (2022-01-01 to 2024-12-31).
Verification: executed

Access

No network or login: simulate(SimParams(end="2024-12-31")) from capstone.airline_sim, or the SQLite sample built by uv run python -m capstone.airline_sim. Synthetic; costs are hypothetical.

Grain

Flight_no x departure date; forecast issued at the d = 2 snapshot (known on date - 2).

Column	Meaning
passengers	Target: boarded passengers (bookings minus cancellations and no-shows, capped at capacity)
cumulative_bookings at days_to_departure = 2	Main point-in-time feature at the order cutoff
capacity	Physical upper bound of the target
market / route / weekday / holiday	Slice and calendar features known in advance

EDA and data-quality rules

Confirm the d = 2 snapshot exists for every departure and is observable on date - 2.
Check the distribution of passengers minus cumulative bookings at d = 2 per market; it should be mostly negative pickup minus cancellations, and its spread sets the interval width.
Verify passengers never exceed capacity; intervals must be clipped to [0, capacity].

Baseline

Point forecast = cumulative bookings at d = 2 times the historical ratio of passengers to d = 2 bookings for the same flight and weekday (last eight departures). Order rules compared: point forecast, point plus 5%.

Features

Cumulative bookings at d = 2 and pace from d = 5 to d = 2, flight_no, market, weekday, holiday window, historical ratio features from departed flights only.

Candidate methods

Model-based intervals: LightGBM quantile regression at 0.1, 0.5, 0.81, 0.9 (and 0.025/0.975)
Learns quantiles directly, so widths can vary by market and booking level. Coverage is not guaranteed and must be checked.
Split-conformal intervals around a LightGBM point model, calibrated on the most recent 8 weeks before each origin
Gives finite-sample marginal coverage when calibration and test data are exchangeable; time series break that assumption, so a recent calibration window and a coverage check per origin are part of the method.

Validation design

Rolling origin, monthly, across 2024: train on all departures before the origin minus the 8-week calibration window, calibrate on that window, test on the next month. Report coverage and mean width per origin, per market and per nominal level; the newsvendor cost is computed on the same test departures.

Leakage checks

Calibration residuals come only from departures before the origin; no snapshot later than d = 2 enters any feature; the conformal quantile is recomputed at each origin.

Bias and representativeness

Coverage per market and per load band: intervals that are right on average can under-cover full flights and over-cover empty ones.

Robustness checks

Vary the calibration window (4, 8, 16 weeks); test coverage on the break routes after 2024-03-01, where exchangeability is most violated; optionally compare adaptive conformal inference.

Technical metrics

Empirical coverage at 80% and 95%, mean interval width (sharpness), pinball loss at 0.81, PIT histogram from the quantile model.

Business interpretation

Total and per-flight catering cost under the hypothetical 25/6 EUR costs for each order rule; sensitivity of the best rule to the cost ratio (from 2:1 to 8:1).

Error slices

Market: domestic, intl-short, intl-long (different cancellation rates).
Load band at d = 2 (below 60%, 60-90%, near capacity).
Break routes after 2024-03-01.
Holiday windows.

Implementation steps

Simulate, join the d = 2 snapshot to flight_day, write the availability table.
Fit the baseline ratio model and the LightGBM point and quantile models per origin.
Compute split-conformal intervals from the recent calibration window per origin.
Score coverage, width and pinball per slice; draw a coverage-vs-width plot.
Apply the four order rules and compute hypothetical costs; run the cost-ratio sensitivity.
Write the decision: which rule, under which cost assumptions, and what would change it.

Reproduction

cd python && uv run python -m reference.r05_intervals_decision.run

Design target: CPU only, no network, under five minutes, fixed seeds. Outputs: coverage.csv, costs.csv, sensitivity.csv, report.md, and a pytest smoke test that checks coverage computation on a toy fixture with known quantiles.

Limitations

Synthetic flights and hypothetical costs; the ranking of rules depends on the cost ratio, which must come from catering and customer teams.
Conformal coverage is marginal (averaged over flights), not per flight, and degrades under drift.
Crew meals, special meals and multi-sector catering are out of scope.

Production and monitoring

Daily job at the d = 2 cutoff producing an order quantity per flight with the quantile used and the model version. Monitor realised shortage and waste counts per week, rolling coverage per market, and calibration window freshness; alert when coverage drifts more than 5 points from nominal for two consecutive weeks.

Interview questions

Why is the order quantity a quantile and not the mean? Derive the fractile from the two costs.
Your conformal intervals have 80% coverage overall but 65% on intl-long. What do you do?
Why is capacity censoring harmless for this target but harmful for demand forecasting?

Postmortem and alternatives

Likely findings: quantile regression is sharper but under-covers in thin slices; conformal is better calibrated overall but too wide for small flights; the cost ranking flips at low cost ratios. Alternatives: conformalised quantile regression (CQR) to combine both, or adaptive conformal updates under drift.

Reference implementation

Status: executed — outputs below are from the recorded run

cd python && uv run python -m reference.r05_intervals_decision.run

r05 results - prediction intervals and a newsvendor order quantity

Recorded run of cd python && uv run python -m reference.r05_intervals_decision.run (full mode). Every number below is copied from the files in outputs/ written by that run (coverage.csv, costs.csv, sensitivity.csv, quantile_calibration.csv, calibration_window_robustness.csv, exchangeability_dispersion.csv, success_criteria.csv, decision.md, report.md, summary.json). The data are synthetic (academy airline simulator, seed 42) and all costs are hypothetical: 25 EUR per missing meal and 6 EUR per wasted meal are teaching assumptions, not catering economics.

Item	Value
Run date	2026-10-11
Seeds	simulator 42; LightGBM `random_state` 42; cluster bootstrap seed 7 (500 resamples over departure dates)
Python / numpy / pandas / scipy / LightGBM	3.12.13 / 2.5.3 / 3.0.6 / 1.18.1 / 4.7.0
Runtime of the recorded run	30.7 s inside `run.py` (33.1 s wall including start-up; single process, LightGBM `n_jobs=1`; Apple M5, 10 cores, 16 GB)
Smoke mode	`--smoke`: 15 months of data, 3 monthly origins, 3.8 s wall when it was timed
Table	20,824 flight-days (19 flights, 2022-01-01 to 2024-12-31); 20,292 rows have all history features; d = 2 snapshot missing for 0; passengers above capacity: 0
Rolling origins	12 monthly origins (first day of each month of 2024), 6,954 scored departures; model fitted once per origin on departures up to 59 days before it, calibrated on the 8 weeks that end 3 days before it
Checks inside the run	for every origin the calibration window ends before the origin (asserted); feature names contain no snapshot newer than d = 2 and no outcome of the departure day (`True`)

Findings

What the problem looks like at the cutoff. Passengers fall short of d = 2 cumulative bookings by 6.8 on average on domestic flights (sd 2.5), 7.0 on intl-short (sd 2.6) and 22.5 on intl-long (sd 5.0): cancellations and no-shows dominate pickup in the last two days, and the spread is about twice as large on long-haul. That spread is what sets interval width. The baseline order (bookings at d = 2 times the same-flight, same-weekday ratio of the last eight departures) has MAE 1.853 meals and LightGBM's mean forecast has 1.187.
Overall coverage is on target for all three interval methods. At the nominal 80% level the empirical coverage is 0.8004 (quantile regression), 0.7993 (split conformal with one width) and 0.8016 (split conformal by market); at 95% it is 0.9445, 0.9502 and 0.9520. All six are within 3 points (the first half of success criterion (a)). Mean widths are 3.85 / 3.75 / 3.83 meals at 80% and 6.31 / 6.37 / 6.45 at 95%, so none is clearly sharper at equal coverage.
Split conformal with one width fails the market criterion; both fixes work. Because it uses a single residual quantile, the one-width conformal interval has the same mean width in every market (3.75 meals at 80%), while the quantile-regression intervals are 5.40 meals wide on intl-long against 3.70 on domestic at 80% (11.73 against 5.68 at 95%). Its coverage on intl-long is 0.6585 at 80% (-14.2 points) and 0.8566 at 95% (-9.3 points), against a limit of 6; domestic and intl-short are slightly over-covered (0.8110 and 0.8247 at 80%). Computing the conformal quantile within each market brings every market within 1.0 point at 80% and 0.5 at 95%, and the quantile-regression intervals are within 0.5 and 0.8 points. Quantile regression and conformal-by-market therefore meet success criterion (a) in full; conformal with one width does not. This is exactly the "80% overall but poor on intl-long" situation in the project's interview question, reproduced on data.
Load bands expose a second kind of miscoverage that the market fix does not touch. The d = 2 load factor is known when the order is issued, so it is a legitimate slice. Conformal intervals keep almost the same width in all three bands (about 3.7 meals at 80%) and therefore over-cover flights below 60% load (0.9168 at 80%, 0.9981 at 95%) and under-cover the 60 to 90% band (0.7716 at 80%); by-market conformal gives 0.9074 and 0.7677. Quantile regression adapts its width to the band (2.91, 4.04 and 3.78 meals) and covers 0.7883, 0.7955 and 0.8089. The below-60% band has only 529 departures, so its coverage has a binomial standard error of about 1.7 points at 80%.
The break routes are not under-covered. From 2024-03-01 the competitor-capacity step is the textbook exchangeability violation, but coverage on the three break routes is 0.7927 (quantile regression), 0.8058 (conformal) and 0.7885 (by market) at 80%, against 0.8053, 0.7977 and 0.8099 on the other routes in the same months; the 95% gaps are -1.3, +1.5 and +0.0 points. The likely reason is that the d = 2 bookings already contain the new level, so the residual distribution barely moves; I did not test that explanation. The break routes before the step (420 departures) are within the same range.
Calibration-window length does not matter here. With 4, 8 or 16 weeks the one-width conformal coverage at 80% is 0.8023, 0.7993 and 0.8003, and on the break routes after the step 0.8114, 0.8058 and 0.8067. Mean widths at 80% change by under 0.02 meals (about 0.04 at 95%).
Month by month, coverage stays inside sampling noise. Each origin scores 551 to 589 departures (binomial standard error about 1.7 points at 80%). At 80% the monthly coverage ranges from 0.7861 to 0.8149 (quantile regression), 0.7725 to 0.8285 (conformal) and 0.7708 to 0.8351 (by market). No month is far out, and I see no trend. The rule in the project's production note (alert when rolling coverage drifts more than 5 points for two consecutive weeks) would not have fired on these monthly figures; weekly windows are much noisier and were not computed.
The exchangeability probe finds little structure. The lag-1 autocorrelation of the daily mean residual is -0.0629. The variance of daily coverage is 1.11 times (80%) and 1.05 times (95%) what independent flights would produce, so the flights of one day are close to independent in this synthetic world. This probes the assumption; it does not verify it, and real flights share weather and disruption days.
Quantile calibration. The share of departures with y at or below the model's predicted quantile is 0.0280 for tau = 0.025, 0.1008 for 0.1, 0.4967 for 0.5, 0.7962 for the order fractile 0.806, 0.9012 for 0.9 and 0.9725 for 0.975; the largest gap is 1.0 point at the fractile that matters for ordering. Passengers are integers, so this is a check of the discrete analogue of a PIT histogram at six levels, not a full PIT histogram.
Pinball loss at the order fractile ranks the methods the way cost does. 0.396 for quantile regression, 0.418 for conformal by market, 0.420 for conformal with one width, 0.601 for the mean forecast used as a quantile and 0.912 for the baseline point.
The newsvendor order beats ordering the point forecast and point plus 5%. At the stated costs the critical fractile is 25 / 31 = 0.806. On 6,954 departures in 2024 the cost per flight is 28.06 EUR for the baseline point, 42.22 for baseline plus 5%, 18.25 for the LightGBM mean forecast, 12.04 for the quantile-regression order, 12.78 for conformal and 12.74 for conformal by market. The quantile order is 16.0 EUR per flight cheaper than the baseline point (95% interval 15.2 to 16.7) and 6.2 cheaper than the mean-forecast order (interval 5.8 to 6.7; bootstrap over departure dates). Success criterion (b) is met by the quantile and both conformal orders. The 5% buffer is the most expensive rule: it orders about 7.0 meals too many on every flight, and at 6 EUR per wasted meal that is almost the whole cost.
Conformal costs a little more than quantile regression, and by-market adds nothing in cost. The two conformal orders are 0.75 and 0.71 EUR per flight above quantile regression at the base ratio, and one-width conformal and by-market conformal differ by 0.04. Their advantage is the finite-sample marginal guarantee, not cost.
Rounding convention (changed once, disclosed). The first full run of this code rounded every quantile order up to the next integer. That adds a meal that the coverage statement does not need, because for integer passengers P(y <= q) = P(y <= floor(q)). The share of flights short was 5.9% under that rule against 10.3% with nearest-integer rounding. I switched all order rules to nearest-integer rounding, the same convention as the baseline rules, before recording this run. The rounded-up variants stay in the table as a sensitivity: they cost 12.86 (quantile), 13.48 (conformal by market) and 14.47 (mean forecast) EUR per flight. Rounding the mean forecast up recovers about 61% of the quantile order's saving over rounding it to the nearest integer, so a large part of "ordering a quantile" in this world is "ordering a little more than the mean". The share of flights short stays well below the nominal 19.4% under every quantile rule; that is consistent with an integer target whose conditional distribution is only a few meals wide, so that the cumulative distribution jumps over the fractile (I did not test this further).
The cost ranking is stable across the tested ratios; the size of the gain is not. The quantile order is the cheapest primary rule at every ratio from 1:1 to 8:1 (the project asked for 2:1 to 8:1; 1:1 and 1.5:1 were added to look for a break). Its advantage over the mean-forecast order is -0.10 EUR per flight at 1:1 (-1.4%), -1.12 at 2:1, -3.26 at 3:1, -6.21 at 4.17:1, -11.50 at 6:1 and -17.47 at 8:1, with bootstrap intervals that exclude zero at every ratio. At near-symmetric costs the gain is small in relative terms; the decision matters when shortage is several times as expensive as waste. I did not find a ratio at which the recommended rule stops winning within the tested range. Outside it (a cost ratio below 1 would put the fractile below 0.5) I did not look.
Decision. Order the model-based quantile at the critical fractile for each flight, issued at d = 2. If a per-market coverage statement is needed for governance, use split conformal by market, at about 0.71 EUR per flight more under these costs. Re-estimate the cost ratio with catering and customer teams before using any number from this run.

Coverage and width

Nominal 80%. QR is LightGBM quantile regression (0.1 and 0.9), CP is split conformal with one width, CP-market is split conformal within market; the gap is empirical coverage minus nominal in percentage points; width is the mean of upper minus lower in meals, after clipping to [0, seats].

slice	n	QR coverage	QR gap pp	QR width	CP coverage	CP gap pp	CP width	CP-market coverage	CP-market gap pp	CP-market width
all departures	6,954	0.8004	+0.0	3.85	0.7993	-0.1	3.75	0.8016	+0.2	3.83
market: domestic	4,026	0.7993	-0.1	3.70	0.8110	+1.1	3.75	0.7971	-0.3	3.65
market: intl-short	2,196	0.8042	+0.4	3.61	0.8247	+2.5	3.75	0.8069	+0.7	3.58
market: intl-long	732	0.7951	-0.5	5.40	0.6585	-14.2	3.75	0.8101	+1.0	5.56
d-2 load below 60%	529	0.7883	-1.2	2.91	0.9168	+11.7	3.77	0.9074	+10.7	3.72
d-2 load 60 to 90%	3,599	0.7955	-0.5	4.04	0.7716	-2.8	3.76	0.7677	-3.2	3.80
d-2 load 90% or more	2,826	0.8089	+0.9	3.78	0.8125	+1.2	3.73	0.8248	+2.5	3.89
break routes, from 2024-03-01	2,142	0.7927	-0.7	3.71	0.8058	+0.6	3.74	0.7885	-1.1	3.63
other routes, from 2024-03-01	3,672	0.8053	+0.5	3.93	0.7977	-0.2	3.74	0.8099	+1.0	3.94
break routes, before 2024-03-01	420	0.7952	-0.5	3.67	0.8000	+0.0	3.77	0.7881	-1.2	3.66
holiday window	1,064	0.8139	+1.4	3.79	0.8158	+1.6	3.79	0.8167	+1.7	3.86
outside holiday windows	5,890	0.7980	-0.2	3.86	0.7963	-0.4	3.74	0.7988	-0.1	3.83

Nominal 95%:

slice	n	QR coverage	QR gap pp	QR width	CP coverage	CP gap pp	CP width	CP-market coverage	CP-market gap pp	CP-market width
all departures	6,954	0.9445	-0.6	6.31	0.9502	+0.0	6.37	0.9520	+0.2	6.45
market: domestic	4,026	0.9416	-0.8	5.68	0.9610	+1.1	6.37	0.9503	+0.0	6.05
market: intl-short	2,196	0.9472	-0.3	5.65	0.9617	+1.2	6.37	0.9540	+0.4	6.15
market: intl-long	732	0.9522	+0.2	11.73	0.8566	-9.3	6.37	0.9549	+0.5	9.61
d-2 load below 60%	529	0.9509	+0.1	4.44	0.9981	+4.8	6.37	0.9981	+4.8	6.23
d-2 load 60 to 90%	3,599	0.9403	-1.0	6.17	0.9519	+0.2	6.36	0.9505	+0.1	6.38
d-2 load 90% or more	2,826	0.9487	-0.1	6.83	0.9391	-1.1	6.37	0.9452	-0.5	6.59
break routes, from 2024-03-01	2,142	0.9365	-1.3	5.64	0.9645	+1.5	6.43	0.9500	+0.0	6.11
other routes, from 2024-03-01	3,672	0.9510	+0.1	6.71	0.9466	-0.3	6.43	0.9553	+0.5	6.65
break routes, before 2024-03-01	420	0.9429	-0.7	5.63	0.9381	-1.2	6.07	0.9405	-1.0	5.89
holiday window	1,064	0.9464	-0.4	6.38	0.9464	-0.4	6.34	0.9474	-0.3	6.36
outside holiday windows	5,890	0.9441	-0.6	6.29	0.9509	+0.1	6.37	0.9528	+0.3	6.47

Coverage by monthly origin

origin (month)	n	QR 80%	CP 80%	CP-market 80%	QR 95%	CP 95%	CP-market 95%
2024-01	589	0.8014	0.7895	0.7929	0.9406	0.9270	0.9389
2024-02	551	0.7967	0.7949	0.8058	0.9365	0.9437	0.9510
2024-03	589	0.7878	0.7929	0.7912	0.9355	0.9525	0.9677
2024-04	570	0.8070	0.7930	0.8035	0.9456	0.9526	0.9596
2024-05	589	0.8149	0.8285	0.8183	0.9457	0.9542	0.9576
2024-06	570	0.8123	0.8228	0.8351	0.9491	0.9439	0.9386
2024-07	589	0.8149	0.8234	0.8098	0.9423	0.9508	0.9576
2024-08	589	0.7861	0.7759	0.7759	0.9321	0.9372	0.9219
2024-09	570	0.7947	0.7772	0.7912	0.9614	0.9544	0.9526
2024-10	589	0.7929	0.7725	0.7708	0.9474	0.9643	0.9593
2024-11	570	0.8000	0.8211	0.8158	0.9404	0.9667	0.9579
2024-12	589	0.7963	0.7997	0.8098	0.9576	0.9559	0.9610

Success criteria of project-022 (coverage)

Overall coverage within 3 points of nominal and every market within 6 points.

method	level	overall coverage	overall within 3 pp	worst market gap (pp)	every market within 6 pp
LightGBM quantile regression	80%	0.8004	yes	0.5	yes
LightGBM quantile regression	95%	0.9445	yes	0.8	yes
split conformal, one width (8 weeks)	80%	0.7993	yes	14.2	no
split conformal, one width (8 weeks)	95%	0.9502	yes	9.3	no
split conformal by market (8 weeks)	80%	0.8016	yes	1.0	yes
split conformal by market (8 weeks)	95%	0.9520	yes	0.5	yes

Quantile calibration of the model-based quantiles

tau	share of departures with y <= q-hat	gap (pp)
0.0250	0.0280	+0.3
0.1000	0.1008	+0.1
0.5000	0.4967	-0.3
0.8065	0.7962	-1.0
0.9000	0.9012	+0.1
0.9750	0.9725	-0.2

Cost of each order rule at the stated hypothetical costs (25 EUR short, 6 EUR waste, fractile 0.806)

Paired per-flight cost differences use a cluster bootstrap over departure dates. "Flights short" is the share of departures with fewer meals than passengers.

order rule	total cost, EUR	EUR per flight	short meals	wasted meals	flights short	vs LightGBM mean, EUR per flight [95% interval]	vs baseline point, EUR per flight [95% interval]
baseline: bookings at d-2 x same-flight weekday ratio	195,137	28.06	6,227	6,577	40.5%	+9.81 [+9.13, +10.36]	reference
baseline point + 5%	293,613	42.22	3	48,923	0.0%	+23.97 [+23.18, +24.78]	+14.16 [+13.19, +15.19]
LightGBM mean forecast	126,905	18.25	4,139	3,905	36.8%	reference	-9.81 [-10.36, -9.13]
LightGBM mean + 5%	298,008	42.85	0	49,668	0.0%	+24.61 [+23.88, +25.36]	+14.79 [+13.94, +15.75]
LightGBM quantile at the fractile	83,706	12.04	990	9,826	10.3%	-6.21 [-6.69, -5.75]	-16.02 [-16.73, -15.24]
split conformal at the fractile (one width)	88,900	12.78	1,228	9,700	11.9%	-5.47 [-5.88, -5.06]	-15.28 [-15.97, -14.52]
split conformal at the fractile, by market	88,613	12.74	1,181	9,848	11.6%	-5.51 [-5.90, -5.11]	-15.32 [-16.03, -14.57]
LightGBM mean, rounded up (rounding sensitivity)	100,600	14.47	2,602	5,925	24.1%	-3.78 [-4.06, -3.55]	-13.59 [-14.16, -12.92]
LightGBM quantile, rounded up (rounding sensitivity)	89,437	12.86	535	12,677	5.9%	-5.39 [-5.89, -4.84]	-15.20 [-15.94, -14.33]
split conformal by market, rounded up (rounding sensitivity)	93,749	13.48	683	12,779	7.2%	-4.77 [-5.24, -4.23]	-14.58 [-15.34, -13.77]

Cost-ratio sensitivity (mean EUR per flight; primary rules, rounding sensitivities excluded)

short : waste	fractile	baseline point	baseline + 5%	LightGBM mean	LightGBM mean + 5%	LightGBM quantile	conformal	conformal by market	cheapest
1.00	0.500	11.05	42.21	6.94	42.85	6.84	6.97	6.97	LightGBM quantile at the fractile
1.50	0.600	13.73	42.22	8.73	42.85	8.30	8.46	8.48	LightGBM quantile at the fractile
2.00	0.667	16.42	42.22	10.51	42.85	9.39	9.56	9.59	LightGBM quantile at the fractile
3.00	0.750	21.79	42.22	14.08	42.85	10.82	11.29	11.28	LightGBM quantile at the fractile
4.17	0.806	28.06	42.22	18.25	42.85	12.04	12.78	12.74	LightGBM quantile at the fractile
6.00	0.857	37.91	42.23	24.80	42.85	13.30	14.56	14.40	LightGBM quantile at the fractile
8.00	0.889	48.66	42.23	31.94	42.85	14.47	15.95	15.77	LightGBM quantile at the fractile

Quantile order against the mean-forecast order and the baseline point, by ratio:

short : waste	fractile	quantile order, EUR per flight	minus mean-forecast order	95% interval	minus baseline point	flights short
1.00	0.500	6.84	-0.10	[-0.16, -0.05]	-4.21	35.9%
1.50	0.600	8.30	-0.42	[-0.54, -0.33]	-5.43	27.6%
2.00	0.667	9.39	-1.12	[-1.31, -0.95]	-7.03	20.8%
3.00	0.750	10.82	-3.26	[-3.56, -2.96]	-10.97	13.7%
4.17	0.806	12.04	-6.21	[-6.69, -5.75]	-16.02	10.3%
6.00	0.857	13.30	-11.50	[-12.19, -10.84]	-24.61	7.3%
8.00	0.889	14.47	-17.47	[-18.41, -16.54]	-34.19	5.7%

Calibration-window robustness (split conformal)

calibration weeks	method	level	slice	n	coverage	gap (pp)	mean width
4	one width	80%	all departures	6,954	0.8023	+0.2	3.76
4	one width	80%	break routes, from 2024-03-01	2,142	0.8114	+1.1	3.75
4	one width	80%	other routes, from 2024-03-01	3,672	0.7990	-0.1	3.75
4	by market	80%	all departures	6,954	0.8044	+0.4	3.85
4	by market	80%	break routes, from 2024-03-01	2,142	0.7946	-0.5	3.64
4	by market	80%	other routes, from 2024-03-01	3,672	0.8096	+1.0	3.94
4	one width	95%	all departures	6,954	0.9498	-0.0	6.39
4	one width	95%	break routes, from 2024-03-01	2,142	0.9617	+1.2	6.44
4	one width	95%	other routes, from 2024-03-01	3,672	0.9464	-0.4	6.44
4	by market	95%	all departures	6,954	0.9521	+0.2	6.53
4	by market	95%	break routes, from 2024-03-01	2,142	0.9505	+0.1	6.14
4	by market	95%	other routes, from 2024-03-01	3,672	0.9559	+0.6	6.75
8	one width	80%	all departures	6,954	0.7993	-0.1	3.75
8	one width	80%	break routes, from 2024-03-01	2,142	0.8058	+0.6	3.74
8	one width	80%	other routes, from 2024-03-01	3,672	0.7977	-0.2	3.74
8	by market	80%	all departures	6,954	0.8016	+0.2	3.83
8	by market	80%	break routes, from 2024-03-01	2,142	0.7885	-1.1	3.63
8	by market	80%	other routes, from 2024-03-01	3,672	0.8099	+1.0	3.94
8	one width	95%	all departures	6,954	0.9502	+0.0	6.37
8	one width	95%	break routes, from 2024-03-01	2,142	0.9645	+1.5	6.43
8	one width	95%	other routes, from 2024-03-01	3,672	0.9466	-0.3	6.43
8	by market	95%	all departures	6,954	0.9520	+0.2	6.45
8	by market	95%	break routes, from 2024-03-01	2,142	0.9500	+0.0	6.11
8	by market	95%	other routes, from 2024-03-01	3,672	0.9553	+0.5	6.65
16	one width	80%	all departures	6,954	0.8003	+0.0	3.76
16	one width	80%	break routes, from 2024-03-01	2,142	0.8067	+0.7	3.75
16	one width	80%	other routes, from 2024-03-01	3,672	0.7979	-0.2	3.75
16	by market	80%	all departures	6,954	0.8004	+0.0	3.81
16	by market	80%	break routes, from 2024-03-01	2,142	0.7894	-1.1	3.60
16	by market	80%	other routes, from 2024-03-01	3,672	0.8069	+0.7	3.91
16	one width	95%	all departures	6,954	0.9504	+0.0	6.41
16	one width	95%	break routes, from 2024-03-01	2,142	0.9608	+1.1	6.40
16	one width	95%	other routes, from 2024-03-01	3,672	0.9436	-0.6	6.40
16	by market	95%	all departures	6,954	0.9517	+0.2	6.40
16	by market	95%	break routes, from 2024-03-01	2,142	0.9500	+0.0	6.05
16	by market	95%	other routes, from 2024-03-01	3,672	0.9529	+0.3	6.56

Exchangeability probe

Lag-1 autocorrelation of the daily mean residual: -0.0629.

nominal level	overall coverage	variance of daily coverage	variance if independent	ratio
80%	0.7993	0.0094	0.0084	1.11
95%	0.9502	0.0026	0.0025	1.05

What this does not prove

Nothing here is about real catering. Flights, passengers and cancellations are simulated, the costs are invented, and the cost ranking depends on the ratio, which has to come from catering and customer teams. One simulated world (seed 42) was used.
Coverage is marginal, not per flight. Slice coverage is an empirical check on 420 to 4,026 departures per slice; slices with a few hundred departures carry binomial standard errors of 1.5 to 2 points, so differences of that size between methods are not evidence. No slice was tuned on.
Exchangeability is probed, not verified. The run found little day-level dependence and no break-route penalty; a real schedule with disruption days, a different break (one that moves the booking curve shape, not only the level) or a longer lead time would not be covered by this evidence.
The model is fixed within each month. Each origin trains on data that end 59 days before the first test day and is not refitted within the month, so late-month orders use a model up to about three months old. That is conservative for accuracy; a daily refit was not tried. The calibration window is not used for fitting, which wastes eight weeks of labels.
The cost comparison is in sample for design choices. The rounding convention was changed once after seeing the first full run (finding 13). The 80% and 95% levels and the 2:1 to 8:1 ratio grid come from the project record; the 1:1 and 1.5:1 ratios and the conformal-by-market variant were added after seeing results (the latter after the market-level miscoverage of one-width conformal). The 2024 costs are therefore a description of this backtest, not a forecast of savings.
Meals equal boarded passengers. Crew meals, special meals, multi-sector catering, minimum order quantities, salvage value and per-flight cost differences are out of scope. Passengers are physically capped by seats, so clipping intervals to [0, seats] cannot lower coverage; capacity censoring is part of this target and is harmless here, unlike for demand forecasting.
Conformalised quantile regression, adaptive conformal inference and a full PIT histogram were not run; they are the alternatives the project's postmortem names.
The bootstrap resamples departure dates, which respects same-day dependence but not dependence between consecutive months through the booking behaviour or the model refit.

Reproduce

cd python && uv run python -m reference.r05_intervals_decision.run          # full run, about 30 seconds
cd python && uv run python -m reference.r05_intervals_decision.run --smoke  # about 5 seconds, writes to a temp dir
cd python && uv run pytest tests/test_r05_intervals_decision.py             # 14 tests, about 10 seconds

Builder-authored reference. It is not learner evidence and is never credited to your portfolio.

Rubric3 criteria

Criterion	Meets	Exceeds
Interval evaluation	Coverage and width per nominal level, origin and market.	Explains under-coverage slices and fixes one with evidence.
Decision economics	Newsvendor fractile derived from labelled hypothetical costs and applied.	Cost-ratio sensitivity identifies where the recommended rule stops winning.
Assumption honesty	States the exchangeability assumption and its violation in time series.	Tests coverage specifically on the break routes and reports the gap.

References6

A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification
Split conformal prediction and its marginal coverage guarantee.
Conformalized Quantile Regression
Conformalised quantile regression as the combined alternative.
Conformal prediction beyond exchangeability
Coverage gaps when exchangeability fails (time series, drift).
Adaptive Conformal Inference Under Distribution Shift
Adaptive conformal inference under distribution shift.
Probabilistic Forecasts, Calibration and Sharpness
Maximise sharpness subject to calibration.
Strictly Proper Scoring Rules, Prediction, and Estimation
Pinball loss as a proper scoring rule for quantiles.

At a glance