Skip to content
ML Academy

Week 2 · Bonus — optional extra studio

Bonus studio 2: SQL cohort and window analytics on the airline sample database

Project thread forecasting · linked lessons day 6, day 7, day 8

Optional bonus. It adds no milestone and does not change core or studio completion. The three guard queries it produces can be reused in the Week 5 backtest harness and the Week 6 data-quality checks.

  • 90 minsession length
  • 5timed blocks
  • 4tasks
  • —status

Session plan90 min

  1. 10′Plan the queries and their control totals
  2. 25′Build 1 - departure-month and market cohorts of booking pace
  3. 25′Build 2 - rolling windows and year-over-year on a clean and a gappy table
  4. 20′Build 3 - top-N, ties and share of total
  5. 10′Reconcile and record

Before you start

Prepare before the session (under 2 minutes).

  • The sample database python/artifacts/airline_sample.sqlite (seed 42, synthetic; rebuild with cd python && uv run python -m capstone.airline_sim if missing).
  • The sqlite3 command-line shell (version 3.25 or later for window functions; version 3.28 or later for RANGE with a numeric offset).
  • A folder forecasting-project/sql/ for the files.

Tables used: flight_day (flight number x departure date), booking_curve (flight number x departure date x snapshot; snapshots at 90, 60, 45, 30, 21, 14, 10, 7, 5, 3, 2, 1 and 0 days out), flight_day_raw (the messy copy for 2022-2023 from Day 6). ROW_NUMBER, RANK, DENSE_RANK, LAG, AVG ... OVER and conditional aggregation are all allowed.

This is analytics, not modelling: the queries read departures through 2023-12-31 and no test-period row.

Blocks5

  1. Block 1: Plan the queries and their control totals

    10 min

    Before writing SQL, do the first half of task reconcile-and-record: write down, for each of the three build tasks, the grain of the result, the number of rows you expect and one control total it must reconcile to. This takes 10 minutes. Run your SQL with sqlite3 -header -column python/artifacts/airline_sample.sqlite < sql/<file>.sql.

  2. Block 2: Build 1 - departure-month and market cohorts of booking pace

    25 min

    Work on task cohort-pace.

    1. Minutes 0-5 - the fan-out (step 1): see the wrong number first.
    2. Minutes 5-18 - the cohort matrix (steps 2 and 3).
    3. Minutes 18-25 - the second cut and the reading (steps 4 and 5).
  3. Block 3: Build 2 - rolling windows and year-over-year on a clean and a gappy table

    25 min

    Work on task rolling-and-yoy.

    1. Minutes 0-12 - the clean table (steps 1 to 3).
    2. Minutes 12-25 - the gappy table (steps 4 and 5).
  4. Block 4: Build 3 - top-N, ties and share of total

    20 min

    Work on task topn-ties-share.

    1. Minutes 0-12 - top five per route under three ranking functions (steps 1 and 2).
    2. Minutes 12-20 - share of total (steps 3 and 4).
  5. Block 5: Reconcile and record

    10 min

    Finish task reconcile-and-record: fill in observed values against the control totals you wrote in the first block, and write the three guards you will reuse.

Tasks4

Sign in to save your work.
  1. Task 1

    code

    A booking-pace cohort groups departures and asks what share of their final bookings had been sold at a given number of days out. Use departures in calendar year 2023.

    1. Join flight_day to booking_curve on (date, flight_no) with no snapshot filter. Report the row count and SUM(bookings_final) against the same figures from flight_day alone. Explain the ratio in one sentence.
    2. Build the matrix with one row per departure month and one column for each of D-60, D-30, D-14 and D-7. Each cell is the sum of cumulative_bookings at that snapshot divided by the sum at D-0, over all departures in the month. Write one query using conditional aggregation.
    3. Check the D-0 sum of each month against SUM(bookings_final) for the same month in flight_day.
    4. Cut the same share at D-30 by market for the whole year.
    5. Write three sentences: how much do the month cohorts differ, how much do the market cohorts differ, and what that implies for using the month as a segmentation of booking pace.

    The expected evidence for this task is shown after you save your own output.

  2. Task 2

    code

    Part A uses the complete table flight_day; part B uses the messy flight_day_raw.

    1. For flight ZX118, for departures 2022-01-01 to 2023-12-31, compute passengers and a 7-day rolling mean (the current day and the 6 before it) with AVG(...) OVER (ORDER BY date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW).
    2. Make the rolling mean NULL while the window holds fewer than 7 rows. Report how many rows that affects.
    3. Add the value from the same weekday one year earlier with LAG(passengers, 364) and the percentage change. Report passengers, the rolling mean, the lagged value and the lag date for 2023-07-15 and 2023-12-31. Say why 364 and not 365.
    4. For flight ZX109 in flight_day_raw, departures in 2023, remove duplicate rows first (one row per date), then compute the same rolling mean two ways: with ROWS BETWEEN 6 PRECEDING and with a calendar window ORDER BY julianday(date) RANGE BETWEEN 6 PRECEDING AND CURRENT ROW. Report both values on 2023-03-13 and 2023-03-14, the first date of the 7 rows the ROWS window used on 2023-03-13, and the number of days present in the calendar window.
    5. Write the guard you will add to any rolling or lag calculation on a feed that may have gaps.

    The expected evidence for this task is shown after you save your own output.

  3. Task 3

    code

    Use flight_day, departures in 2023.

    1. For each route, return the 5 departures with the most bookings_final, using ROW_NUMBER with a tie-break (date, then flight_no). Report the total row count.
    2. Repeat with RANK() and with DENSE_RANK() and keep rows with rank at most 5. Report the total row count for each and, for each route, how many rows sit at the booking cap (bookings_final = CAST(capacity * 1.06 AS INT)).
    3. For December 2023, return each route's SUM(bookings_final) and its percentage of the network total using a window over the grouped result (no subquery for the total). Add the rank of the route.
    4. Repeat step 3 from the join of flight_day to booking_curve with no snapshot filter. State what changed and what did not.

    The expected evidence for this task is shown after you save your own output.

  4. Task 4

    review

    In the first block, before running anything, write a table with one row per build task: the grain of the result, the number of rows you expect, and one control total or invariant it must reconcile to (for example, which table's sum, and for which period).

    After the builds, add the observed values and the difference. Then write three guards, each as a SQL query that returns zero rows when everything is fine, that you will reuse:

    • a grain guard for a join (the joined row count equals the left table's count);
    • a window guard (every windowed value has the minimum number of days behind it);
    • a tie guard (a top-N query returns exactly N rows per group, or the group is reported).

    State which of your three builds you would trust least and why.

    The expected evidence for this task is shown after you save your own output.

Rubric3 criteria

CriterionMeetsExceeds
Grain and reconciliationEvery query states its grain and reconciles to a control total taken from another query or table. The fan-out is reproduced and explained.The reconciliations are written before the queries run, and a guard demonstrably fails on the defect it targets (fan-out, gap or tie).
Window correctnessFrames are explicit (`ROWS` or `RANGE`), partial windows are handled, and the year-over-year lag is justified by the weekday.The learner shows where `ROWS` and calendar frames disagree on a gappy table and adds a minimum-count guard.
Ranking and interpretationTie behaviour of the three ranking functions is demonstrated with counts, and the learner explains the cap.The learner replaces a misleading top-N with the question the business is asking, and reports shares with the caveat that uniform fan-outs do not change them.

Required output

Saved in forecasting-project/sql/ and notes/:

  • bonus02_cohort_pace.sql, bonus02_rolling_yoy.sql, bonus02_topn_share.sql;
  • the reconciliation table (expected against observed);
  • the three guard queries, each shown failing on its target defect;
  • a short note on which query you trust least.

The studio is optional and changes no completion count.

Studios are tracked separately from the 60 core lessons; completing one never completes a lesson.