Skip to content
ML Academy

Week 6 · Sunday — consolidate and defend

Forecasting studio 6: final report, Part I defense and a transfer case (Day 30 extended assessment)

Project thread forecasting · linked lessons day 26, day 27, day 28, day 29, day 30

Milestone W6-b, which closes the Weeks 1-6 thread. It contains:

  • the finished forecasting report (500-800 words plus computed tables);
  • a Part I defense;
  • an unfamiliar transfer answer for a new horizon and data-availability case.

Together with Saturday, this is the Week 6 main evidence in Appendix C: forecasting project evidence, limitations and an unfamiliar transfer answer.

  • 90 minsession length
  • 4timed blocks
  • 5tasks
  • —status

Session plan90 min

  1. 20′Spaced recall (assessment section 1)
  2. 40′Improve - finish the report, then the transfer case (assessment section 2)
  3. 20′Part I defense (assessment section 3)
  4. 10′Part I review and Part II planning (assessment section 4)

Before you start

Prepared before the session (under 2 minutes):

  • notes/report-w06.md (the skeleton with computed tables);
  • configs/frozen_w06.yaml;
  • outputs/test_w06.csv and outputs/test_run_w06.json;
  • the Week 1 contract and brief, the Week 4 runbook and the Week 5 backtest files;
  • decision-log.md.

No new data are needed. The transfer case uses only the seed-42 sample database's structure; its new facts are given in the task and are hypothetical. Have the Day 30 core probe result available for the Part I review. The panel and the network-planning team are simulated.

Blocks4

  1. Block 1: Spaced recall (assessment section 1)

    20 min

    This session is also the Day 30 extended assessment, in four saved sections. Submit each one before reading any reference notes.

    Section 1: answer the cumulative Part I prompts in task recall-part1, closed-book (13 minutes). Then check them against lessons 1-29 and your recall log (7 minutes), and log misses with corrections.

  2. Block 2: Improve - finish the report, then the transfer case (assessment section 2)

    40 min
    1. Minutes 0-15 - finish the report (task final-report). The tables already exist from Saturday; write the prose only.
    2. Minutes 15-40 - transfer case (task transfer-case; see transfer_task). Work without the reference solution, without your lesson notes and without reusing earlier text verbatim. Your own project files are allowed.

    If Saturday's held-out run was carried forward, run it first and cut the report to five sections: question, design, held-out results, decision, limitations. Never cut the transfer case.

  3. Block 3: Part I defense (assessment section 3)

    20 min

    Answer the simulated panel's questions in task part1-defense, in at most 80 words each, citing your saved evidence. If you can, answer out loud first (2 minutes per question), then write the answer down.

  4. Block 4: Part I review and Part II planning (assessment section 4)

    10 min

    Do task part1-review. Use evidence, not confidence, to choose the Part II review priorities (the Day 30 outcome). Record:

    • the studio completion status for all 12 Week 1-6 studio ids;
    • any carried-forward minimum tasks.

    The forecasting thread closes here. Week 7 starts the neural thread.

Tasks5

Sign in to save your work.

  1. Task 1

    recall

    Answer closed-book:

    1. Name the six decision-first questions.
    2. A rule flags 90% of no-shows and 5% of shows; 8% of passengers no-show. What share of flagged passengers no-show?
    3. Effect size vs p-value: one sentence.
    4. Name one confounder in fares and bookings, and the design that removes it.
    5. What is a join fan-out, and the check that catches it?
    6. Why must the boundary come before the preprocessing fit?
    7. What does log loss reward?
    8. Permutation importance is not an intervention effect. Why?
    9. Data drift vs concept drift.
    10. What is the minimum safe lag for a D-14 forecast of final bookings, and why?
    11. Prediction interval vs confidence interval.
    12. What does the newsvendor critical ratio choose?

    The expected evidence for this task is shown after you save your own output.

  2. Task 2

    writing

    Write the prose of notes/report-w06.md: 500-800 words around the computed tables. Cover:

    • the decision and the question;
    • the data and point-in-time contract;
    • the design and why it was frozen;
    • the backtest and held-out results against both baselines;
    • uncertainty and the decision rule;
    • the main error slices;
    • limitations;
    • next steps.

    Include one paragraph explaining why the forecast cannot answer three what-ifs: IST-TZX capacity 160 to 180, price -5%, and competitor capacity +25%.

    The expected evidence for this task is shown after you save your own output.

  3. Task 3

    analysis

    Complete the case described in transfer_task in 25 minutes. Write notes/transfer-w06.md with these headings:

    • clarifying questions and assumptions;
    • cutoff and actual availability;
    • target and its limits;
    • features with minimum lags;
    • baselines;
    • backtest design;
    • decision metric;
    • what would change the recommendation.

    The expected evidence for this task is shown after you save your own output.

  4. Task 4

    defense

    Simulated panel questions:

    1. "Prove your forecast is point-in-time safe."
    2. "Your test result and your backtest disagree, or agree. Which do you believe, and why?"
    3. "Why should RM trust your interval?"
    4. "Competitor capacity on IST-LHR rises 25% next spring. What do you do before and after it happens?"
    5. "Why not try a time-series foundation model instead?"

    The expected evidence for this task is shown after you save your own output.

  5. Task 5

    review

    Write the Part I review in decision-log.md (150-250 words). It contains:

    • evidence per competency dimension, from recall scores, the Day 10, 20 and 30 checkpoint results, and studio rubric outcomes;
    • the three Part II review priorities, named by concept id, each with the evidence that selected it;
    • the status of all 12 Week 1-6 studios (completed, or open with their minimum tasks);
    • how carried-forward items fit into Week 7 without adding sessions.

    The expected evidence for this task is shown after you save your own output.

Transfer task

New horizon and data-availability case. All details are hypothetical and the requester is simulated.

Network planning wants to decide, per departure of IST-JFK (ZX118, currently 280 seats), whether to swap to a hypothetical 250-seat aircraft. The decision is taken at D-45 in a weekly planning meeting held every Monday. Two new data facts apply:

  • The snapshot feed for intl-long routes now lands two days late, so the D-45 snapshot is loaded at D-43.
  • competitor_price is published only from D-21.

Using the sample database's structure, design the forecast and its evaluation. Cover:

  • (a) clarifying questions and assumptions;
  • (b) the true cutoff, and which snapshot and features are actually available;
  • (c) the target, and why bookings observed with 280 seats are a limited guide to a 250-seat decision;
  • (d) minimum safe lags;
  • (e) baselines;
  • (f) a rolling-origin backtest with the right gap;
  • (g) a decision metric with labelled hypothetical costs;
  • (h) what evidence would change your recommendation.

Do not reuse the reference solution or copy earlier text; adapt your own project's reasoning.

Rubric4 criteria

CriterionMeetsExceeds
Project report500-800 words, every number traceable to an output, both baselines shown, uncertainty and the decision rule included, and the limitations and causal what-ifs addressed.The recommendation is conditional and decision-ready. It states what evidence would change it, and how the forecast should be monitored using the Week 4 runbook.
Transfer to a new horizon and availability caseWorks out, from the stated data facts, what is actually available at the real decision cutoff, and chooses features, minimum lags and a backtest gap that are consistent with it.Also handles the remaining practical constraints of the case (decision cadence, amount of data per flight) and the limits of observed bookings and of causal what-ifs for a capacity change. It asks the clarifying questions that would change the design.
Part I defenseAll five answers cite the learner's own evidence and are technically correct.The answers weigh conflicting evidence honestly and anticipate follow-ups, especially on test against backtest and on the foundation-model question.
Evidence-based reviewThe Part II priorities are named by concept id with supporting evidence. All 12 studio statuses are recorded.The priorities are tied to specific Part II days or studio blocks, with a measurable check of whether they worked.

Required output

Four saved assessment sections:

  • the recall answers;
  • the finished report and the transfer answer (notes/report-w06.md, notes/transfer-w06.md);
  • the Part I defense answers;
  • the Part I review with Part II priorities and studio status.

This completes the Week 6 main evidence: forecasting project evidence, limitations and an unfamiliar transfer answer. It also serves as the Day 30 extended assessment.

Studios are tracked separately from the 60 core lessons; completing one never completes a lesson.