Skip to content
ML Academy

Week 2 · Sunday — consolidate and defend

Forecasting studio 2: messy feed diagnosis, chart defense and Week 2 routing (Day 10 extended assessment)

Project thread forecasting · linked lessons day 6, day 7, day 8, day 9, day 10

Milestone W2-b. Three outputs:

  • the Saturday quality checks proven on an unseen defective feed;
  • point-in-time fix rules with a corrected D-14 query;
  • the Week 2 routing decision.

Together with Saturday's outputs, this is the Week 2 main evidence in Appendix C: quality-check outputs, a corrected query and the routing decision.

  • 90 minsession length
  • 4timed blocks
  • 5tasks
  • —status

Session plan90 min

  1. 20′Spaced recall (assessment section 1)
  2. 40′Improve - diagnose a new messy feed (assessment section 2)
  3. 20′Teach-back - explain a chart (assessment section 3)
  4. 10′Weekly review - workload and routing (assessment section 4)

Before you start

Prepare before the session (about 2 minutes). Ideally someone else does it. If you do it yourself, paste and run the block without studying it: finding the defects is the assessment.

# from the repository root
mkdir -p forecasting-project/data
cp python/artifacts/airline_sample.sqlite forecasting-project/data/w02_variant.sqlite
sqlite3 forecasting-project/data/w02_variant.sqlite < forecasting-project/data/w02_variant.sql

where forecasting-project/data/w02_variant.sql contains:

DROP TABLE IF EXISTS booking_feed;
CREATE TABLE booking_feed AS
SELECT date, flight_no, days_to_departure, cumulative_bookings,
       date(date, '-' || days_to_departure || ' days') AS loaded_at
FROM booking_curve WHERE date BETWEEN '2024-01-01' AND '2024-06-30';
INSERT INTO booking_feed
SELECT date, flight_no, days_to_departure, cumulative_bookings + 6, date(date, '-' || (days_to_departure - 3) || ' days')
FROM booking_curve WHERE days_to_departure = 14 AND date BETWEEN '2024-04-08' AND '2024-04-14';
DELETE FROM booking_feed WHERE flight_no = 'ZX111' AND days_to_departure = 14 AND date BETWEEN '2024-02-05' AND '2024-02-18';
UPDATE booking_feed SET days_to_departure = 15 WHERE flight_no IN ('ZX114','ZX115') AND days_to_departure = 14 AND date BETWEEN '2024-05-01' AND '2024-05-31';
UPDATE booking_feed SET flight_no = lower(flight_no) || ' ' WHERE flight_no = 'ZX118' AND date BETWEEN '2024-03-10' AND '2024-03-23';
UPDATE booking_feed SET cumulative_bookings = cumulative_bookings / 10 WHERE flight_no = 'ZX107' AND days_to_departure = 10 AND date BETWEEN '2024-06-01' AND '2024-06-07';
UPDATE booking_feed SET cumulative_bookings = NULL WHERE flight_no = 'ZX104' AND days_to_departure = 14 AND date BETWEEN '2024-01-20' AND '2024-01-24';

The variant is a separate database copy, so the project's seed-42 dataset version stays unchanged. Also have ready:

  • Saturday's quality-check script;
  • your Day 10 checkpoint result;
  • your weekday time log for Days 6-10.

The departures in the feed lie inside the development period. Nothing from the protected test period is used.

Blocks4

  1. Block 1: Spaced recall (assessment section 1)

    20 min

    This session is also the Day 10 extended assessment, in four saved sections. Submit each section before opening any reference notes or answer keys.

    Section 1: answer the cumulative prompts in task recall-cumulative closed-book (12 minutes) and save them. Then check against lessons 1-9 (8 minutes). Log each miss with a correction in notes/recall-log.md.

  2. Block 2: Improve - diagnose a new messy feed (assessment section 2)

    40 min

    Use the feed booking_feed in forecasting-project/data/w02_variant.sqlite (see setup). It covers departures 2024-01-01 to 2024-06-30 and has an extra loaded_at column: the date on which each row was loaded.

    1. Minutes 0-15 - run Saturday's checks on booking_feed, adapting table and column names only (task diagnose-feed).
    2. Minutes 15-30 - locate and explain every defect: what it is, where it is (flight and dates), how it was detected, and its effect on bk_d14.
    3. Minutes 30-40 - fix rules and corrected query (task fix-rules).

    Save your attempt before reading the answer key in the task's expected evidence.

  3. Block 3: Teach-back - explain a chart (assessment section 3)

    20 min

    Do task explain-chart. Build one chart (10 minutes), then write the explanation (10 minutes). The audience is a simulated RM manager deciding whether the feed can be used for the D-14 forecast.

  4. Block 4: Weekly review - workload and routing (assessment section 4)

    10 min

    Do task routing-review. This is the Week 2 workload review the spec requires. The decision it records adjusts how Weeks 3-4 use their existing time; it never adds sessions.

Tasks5

Sign in to save your work.

  1. Task 1

    recall

    Answer closed-book, then save:

    1. State your project's grain, cutoff and horizon.
    2. Name MCAR, MAR and MNAR, with one airline example each.
    3. Is an outlier an error, a rare truth or a regime change? How would you tell which?
    4. Why does a join on (date, flight_no) to booking_curve inflate sums?
    5. Event time vs processing time: which one decides point-in-time correctness?
    6. Why must the holdout boundary be drawn before fitting a scaler or an encoder?
    7. What does a 95% interval for a difference in means not tell you?
    8. Name one confounder in the fare-and-bookings relationship.
    9. Name one chart that misleads by its axis, and the fix.
    10. What is fold-safe target encoding protecting against?

    The expected evidence for this task is shown after you save your own output.

  2. Task 2

    analysis

    Run your Saturday checks against booking_feed. For every defect you find, record:

    • what it is;
    • the flights and departure dates affected, using counts from your queries;
    • the check that found it;
    • its effect on the D-14 feature bk_d14, or on any later feature.

    Add any check you were missing.

    The expected evidence for this task is shown after you save your own output.

  3. Task 3

    code

    For each defect, write a fix rule and its justification. Then write sql/w02_feed_d14_fixed.sql. It returns exactly one D-14 value per departure in the feed period, with an explicit bk_d14_status column (ok, repaired or missing). Report row counts before and after.

    The expected evidence for this task is shown after you save your own output.

  4. Task 4

    defense

    Build one chart that helps a simulated RM manager decide whether the feed is usable. A good choice is the daily count of D-14 rows per departure date, coloured by bk_d14_status, or a small-multiples view by flight.

    Then:

    1. Give it an insight-led title stating the finding.
    2. Write a 100-150 word explanation that includes one interpretation limit.
    3. Answer the manager's simulated question, "Can we just drop the bad days?", in at most 60 words.

    The expected evidence for this task is shown after you save your own output.

  5. Task 5

    review

    Week 2 workload and routing review. Write 100-200 words in decision-log.md using these inputs:

    • recall accuracy, from section 1 and the app's review queue;
    • the Day 10 checkpoint result;
    • task difficulty: which studio tasks overran and by how much;
    • backlog: any studio id still open, and its unfinished tasks;
    • actual weekday minutes against the 30-minute budget.

    Choose one routing decision:

    • (a) keep the plan;
    • (b) reduce exercise depth in named areas;
    • (c) allocate named studio blocks in Weeks 3-4 to weak concepts.

    Name the concept ids. State what evidence would trigger re-routing at the Week 4 review.

    The expected evidence for this task is shown after you save your own output.

Rubric4 criteria

CriterionMeetsExceeds
Defect detectionAt least five of the six planted defects are located, with flight and date ranges, and their effect on `bk_d14` is stated.All six are found. The note explains why Saturday's clean-data checks missed any of them, and adds checks that would catch them next time, such as a `loaded_at` versus snapshot-date check.
Point-in-time repairThe duplicates are resolved by load time as known at the cutoff, not by "latest wins". The corrected query returns one row per departure with a status column.Every repair is justified with evidence from the feed itself, such as `loaded_at` for the relabelled rows. Imputations are flagged so that later evaluation can exclude or slice them.
Chart communicationThe chart type fits the question. The title states the finding. One genuine interpretation limit is given.The explanation leads to a decision for the RM manager, with a clear condition under which the feed would be safe to use.
Routing decisionThe decision is based on at least two evidence items, names concept ids, and keeps the budgets.It sets a measurable trigger for the Week 4 review and says which Week 3-4 blocks change.

Required output

Four saved assessment sections: recall answers, the feed diagnosis with fix rules and the corrected query, the chart with its explanation, and the routing decision.

Saved in forecasting-project/:

  • notes/recall-log.md (updated);
  • notes/feed-diagnosis-w02.md;
  • sql/w02_feed_d14_fixed.sql, with before and after counts;
  • the chart file and its explanation;
  • the routing entry in decision-log.md, with next week's priority.

This completes the Week 2 main evidence: quality-check outputs, a corrected query and the Week 2 routing decision. It also serves as the Day 10 extended assessment.

Studios are tracked separately from the 60 core lessons; completing one never completes a lesson.