Skip to content
ML Academy

Week 1 · Sunday — consolidate and defend

Forecasting studio 1: challenge the assumptions and revise the brief

Project thread forecasting · linked lessons day 1, day 2, day 3, day 4, day 5

Milestone W1-b: a revised problem and assumption brief (v1). It separates the D-14 booking forecast from causal price questions, carries an explicit DAG and a cost-sensitivity note, and has a change log against v0. Together with the dataset contract this is the Week 1 main evidence in Appendix C.

  • 90 minsession length
  • 4timed blocks
  • 6tasks
  • —status

Session plan90 min

  1. 20′Spaced recall
  2. 40′Improve - causal challenge and cost sensitivity
  3. 20′Teach-back and decision defense
  4. 10′Weekly review and planning

Before you start

Prepared before the session (no installation; under 2 minutes):

  • forecasting-project/notes/problem-brief-v0.md and dataset-contract-v0.md from studio-w01-sat.
  • The sample database at python/artifacts/airline_sample.sqlite. If it is missing, rebuild it with cd python && uv run python -m capstone.airline_sim; it is deterministic.
  • Lesson 5 (causal thinking) open for the DAG notation. Keep the lessons closed during the recall block.

The stakeholder in the scenario is simulated. The pricing mechanism quoted in task dag-and-design is documented behaviour of the synthetic simulator, not a claim about any real airline.

Blocks4

  1. Block 1: Spaced recall

    20 min

    Do the eight prompts in task recall-week1 closed-book. Take about 10 minutes and write short answers. Then spend 10 minutes checking each one against lessons 1-5 and Saturday's notes. For every miss or partial answer, write the corrected version in your own words in notes/recall-log.md, together with the reason you missed it. A correct answer reached by guessing counts as a miss.

  2. Block 2: Improve - causal challenge and cost sensitivity

    40 min

    This block uses a related scenario to stress-test the brief. Spend about 25 minutes on the scenario and 15 minutes on the revision.

    1. Commit first (3 min). Read the simulated stakeholder note in task fare-confounding. Before running any query, write your first answer in two sentences. Keep this answer even if it later turns out to be wrong.
    2. Look at the data (12 min). Run the comparison described in the task.
    3. Explain and design (10 min). Do task dag-and-design.
    4. Costs and revision (15 min). Do tasks cost-sensitivity and brief-v1.

    If studio-w01-sat is unfinished, close its minimum tasks first, using the first 15 minutes of this block, and shorten step 2 accordingly.

  3. Block 3: Teach-back and decision defense

    20 min

    Explain brief v1 in 3 minutes, out loud, recorded or written, to a simulated RM director who has 3 minutes and no statistics background. Then answer the three challenge questions in task defend-brief, in at most 80 words each. Every answer must point to evidence you produced: a query output, a contract row or a calculation.

  4. Block 4: Weekly review and planning

    10 min

    Update decision-log.md with:

    • lessons 1-5 status;
    • the number of recall misses and their topics;
    • studio status (both Week 1 ids: completed or carried forward);
    • the open questions;
    • one sentence naming next week's priority.

    If something is carried forward, say which task, and in which block of which studio it will be finished.

Tasks6

Sign in to save your work.

  1. Task 1

    recall

    Answer closed-book:

    1. Name the six decision-first questions.
    2. For your project, state the grain, cutoff, horizon and label window from memory.
    3. 5% of departures sell out. A rule flags 80% of the sell-outs and 10% of the other departures. Using natural frequencies, what share of flagged departures sell out?
    4. What does "95% confidence interval" mean? Name one wrong interpretation.
    5. Why can a tiny p-value go with a useless effect?
    6. Define a confounder and give one from the airline data.
    7. Why is a demand forecast not an answer to "what if we cut the price 5%?"
    8. In a sell-out alert, which error is the false positive, and what does it cost in your decision?

    The expected evidence for this task is shown after you save your own output.

  2. Task 2

    analysis

    Simulated stakeholder note: "Departures with higher average fares end up with more bookings. So we should raise fares on weak flights to lift demand."

    Write your two-sentence first response. Then, for at least two routes (one domestic, one international), compare bookings_final across bands of avg_fare, for example quartiles within each route. Repeat the comparison within one seasonality level. Save the query and output as sql/w01_fare_bands.sql.

    The expected evidence for this task is shown after you save your own output.

  3. Task 3

    writing

    According to the simulator documentation, revenue management sets higher fares when demand is expected or observed to be high. True own-price elasticities are negative (-0.6 to -1.35 by route), and bookings are capped by capacity plus overbooking.

    1. Draw a DAG with at least these nodes: season/holiday/event, expected demand, early-booking signal (unobserved), fare, bookings, capacity.
    2. Explain which path produces the stakeholder's pattern.
    3. Propose one design that could estimate the price effect, either a randomized fare test on selected departures or a qualified quasi-experiment. Give its main cost and risk.
    4. State which parts of this question your D-14 forecasting project will not answer.

    The expected evidence for this task is shown after you save your own output.

  4. Task 4

    analysis

    Write down two hypothetical per-seat costs, labelled as assumptions:

    • c_over, from closing class L on a departure that then flies with empty seats (seats spoiled);
    • c_under, from keeping L open on a departure that would have filled with higher fares (dilution).

    Show how the preferred action for a borderline departure changes as the ratio c_under / c_over moves from 0.5 to 2. A short table or a few lines of arithmetic is enough.

    The expected evidence for this task is shown after you save your own output.

  5. Task 5

    writing

    Revise the brief into notes/problem-brief-v1.md. Add:

    • a "What this forecast cannot answer" section;
    • the DAG;
    • the cost assumptions;
    • a numbered list of assumptions that could fail, with how you would detect each;
    • a change log against v0, one line per change, each with its reason.

    The expected evidence for this task is shown after you save your own output.

  6. Task 6

    defense

    Answer these simulated challenge questions, in at most 80 words each:

    1. "Why not just use last year's bookings for the same flight?"
    2. "Can your model tell us whether a 5% price cut fills the flight?"
    3. "What happens to your forecast if the D-14 snapshot arrives two days late?"

    The expected evidence for this task is shown after you save your own output.

Rubric4 criteria

CriterionMeetsExceeds
Causal reasoningRejects the price-lifts-demand reading. Explains confounding by expected demand with a DAG, and says the forecasting project does not estimate the price effect.Also identifies the unobserved early-booking signal and capacity censoring as reasons why adjusting for observed variables would still not recover elasticity. Proposes a feasible test with a guardrail and an honest cost.
Brief revisionv1 adds the required sections, and every change is logged with a reason.The assumptions are ranked by consequence, each with a detection method and a fallback plan.
Recall correctionAll eight prompts were attempted closed-book. Misses are corrected in the learner's own words.The misses are grouped into a pattern, such as "base-rate errors", with a concrete plan to revisit them in Week 2.
DefenseThe three answers are short, correct and each cites the learner's own evidence.The answers anticipate the follow-up question. For example, answer 3 proposes adding a load timestamp to the contract.

Required output

Saved in forecasting-project/:

  • notes/recall-log.md, with corrections;
  • sql/w01_fare_bands.sql and its output;
  • notes/problem-brief-v1.md, with the DAG, cost assumptions, the "cannot answer" section and the change log;
  • the teach-back answers;
  • the Week 1 review entry in decision-log.md, naming next week's priority.

Together with the Saturday contract, this is the Week 1 main evidence: a dataset contract and a defensible problem and assumption brief.

Studios are tracked separately from the 60 core lessons; completing one never completes a lesson.