Skip to content

ML Academy60-day senior track

Today
Curriculum
Practice
Labs
Library
Portfolio
Progress
Settings

Day 1

Recall block

Diagnostic

This is today's five-minute recall block: 10 short items across statistics, SQL, data quality, classical ML, validation, forecasting, experimentation, business judgment and Python. Answer without looking anything up and rate your confidence honestly for each item - a confident wrong answer is the most useful signal you can give.

This is not a grade and it does not gate anything. It sets your starting review plan and tells the course where to replace basic exposition with harder cases. Some items are deliberately hard; experienced practitioners miss them too. Self-reported experience is not evidence; your answers are. Routing is provisional and is revisited at the Week 2 review.

Your progress

0 / 10

0 of 10 answered. Results feed your evidence profile; they are a starting point, revised at the Week 2 review.

Validation and metricsStatistics and probabilityAnalytics and SQLEDA and data qualityForecastingClassical ML and model selectionCausality and experimentationBusiness judgment

Sign in to record your diagnostic.

Question 1 · easy

2% of bookings are fraudulent. A fraud model scores 98% accuracy on a holdout set. What is the most useful first response?

OptionsA. The model is ready for a pilot; 98% is well above the usual bar, and a holdout confirms it generalizes.B. Report ROC-AUC instead; it does not depend on prevalence, so it tells you what share of the flags are real fraud.C. A model that flags nothing also scores 98%. Ask for precision and recall at a threshold chosen from the cost of a missed fraud versus a blocked customer.D. Keep accuracy, but make sure the decision threshold is 0.5, the neutral choice for probabilities, and report the accuracy with a bootstrap confidence interval.

How confident are you? (optional, used for calibration)

Question 2 · medium

1% of bookings are fraudulent. A screening rule flags 90% of fraudulent bookings and also flags 5% of legitimate ones. Among flagged bookings, what percentage are actually fraudulent? (Answer in %, one decimal.)

Your answer%

How confident are you? (optional, used for calibration)

Question 3 · hard

`flight_day` has 10,000 rows; 600 of them are on IST-FRA or IST-LHR, and no row has a NULL route. A small table `suspended_routes` has three rows: `'IST-FRA'`, `'IST-LHR'` and `NULL`. What does this query return? ```sql SELECT COUNT(*) AS n FROM flight_day WHERE route NOT IN (SELECT route FROM suspended_routes); ```

OptionsA. 9,400: the two suspended routes are excluded and the NULL row is ignored.B. 0C. 10,000: once the list contains a NULL the condition cannot be evaluated, so the filter is skipped.

How confident are you? (optional, used for calibration)

Question 4 · easy

In a raw flight table, `avg_fare` is missing on 6% of rows, mostly on long-haul international flights, which have the highest fares. You need the average fare across all flights. What should you do?

OptionsA. Drop the rows with missing fares; 94% of the data is plenty for an average, and the remaining rows are unbiased.B. Fill the gaps with the overall mean of the observed fares, so no rows are lost and the sample size stays at 100%.C. The values are missing not at random, so no valid average can be computed; the data are unusable.D. Average within each market, weight by each market's flight count (or impute within market), and report the missing share.

How confident are you? (optional, used for calibration)

Question 5 · medium

Daily passengers on IST-AYT have a strong weekly pattern. A vendor's model has an MAE of 41 passengers per day on the last 8 weeks, which it did not see in training. What is the most informative next comparison?

OptionsA. The MAE of a seasonal-naive forecast (same weekday last week) on the same 8 weeks.B. The model's AIC against an ARIMA's AIC fitted to the same history.C. Re-evaluate on a random 20% of the full three-year history, so the estimate uses more data.D. No comparison is needed; an MAE of 41 on unseen weeks is already an honest number.

How confident are you? (optional, used for calibration)

Question 6 · easy

A gradient-boosted tree model predicts bookings from fare and other features. Fares in the training data range from EUR 60 to EUR 300. What does it predict for a fare of EUR 450, other features unchanged?

OptionsA. It continues the downward trend it learned, so predicted bookings keep falling as the fare rises beyond EUR 300.B. About the same as at the highest fares it saw; tree predictions are flat beyond the training range.C. A reliable what-if estimate of demand at EUR 450, because the model learned how bookings respond to fare.

How confident are you? (optional, used for calibration)

Question 7 · medium

In a randomized test of a new seat-selection page, more treatment users reach the payment step. Among users who reach payment, purchase rate is 62% in treatment versus 66% in control. The product manager concludes the new page hurts conversion at payment. What is the right reading?

OptionsA. The PM is right; randomization makes any subset of users, including those who reach payment, comparable between the groups.B. The payment-step comparison is the better one, because it removes users who never intended to buy and is therefore less noisy and more precise.C. Compare purchase rate across all randomized users; the new page sends more, less committed users to payment, so those groups differ.D. Keep all users, but add "reached payment" as a control variable in a regression of purchase on treatment.

How confident are you? (optional, used for calibration)

Question 8 · hard

Two independent booking-page variants: A converts 10.0% (95% CI 9.0% to 11.0%), B converts 11.6% (95% CI 10.6% to 12.6%). The intervals overlap. What follows?

OptionsA. The difference is not statistically significant at the 5% level, because the two confidence intervals overlap by 0.4 points.B. There is a 95% probability that B's true conversion rate lies between 10.6% and 12.6%, so B is better by at least 0.6 points.C. Keep the test running and check daily until the intervals stop overlapping, then stop and declare B the winner at 5% significance.D. Overlap does not settle it: the 1.6-point difference has a standard error of about 0.72, so its 95% CI is roughly 0.2 to 3.0.

How confident are you? (optional, used for calibration)

Question 9 · medium

`X` has 300 rows and 5,000 columns; `y` is a binary label. A colleague reports 0.75 accuracy from: ```python X_sel = SelectKBest(f_classif, k=20).fit_transform(X, y) scores = cross_val_score(LogisticRegression(max_iter=1000), X_sel, y, cv=5) print(scores.mean()) ``` What is the main problem?

OptionsA. The selector used the labels of all 300 rows, including each validation fold. Put selector and model in one Pipeline and cross-validate that.B. Nothing; cross_val_score refits the model in every fold, so each fold's score is out-of-sample.C. Five folds are too few for 300 rows; use 10-fold stratified cross-validation to remove the optimism and reduce variance.

How confident are you? (optional, used for calibration)

Question 10 · hard

You load meals on a flight before you know how many passengers will want one. Each passenger left without a meal costs EUR 24 (voucher and goodwill); each unused meal costs EUR 8. You have a forecast distribution of meal demand. Which quantile of that distribution minimizes expected cost? (Answer as a percentage, e.g. 50 for the median. Hypothetical costs; assume costs are linear per meal.)

Your answer% quantile

How confident are you? (optional, used for calibration)