Skip to content
ML Academy

Week 7 · Saturday — build and investigate

Neural project: first training loop beside a classical baseline

Project thread neural · linked lessons day 31, day 32, day 33, day 34, day 35

Milestone N1 of the Weeks 7-8 neural thread: a frozen split, a classical baseline, and one executed small network with shape, gradient and training checks on the same split and metric.

  • 90 minsession length
  • 3timed blocks
  • 5tasks
  • —status

Session plan90 min

  1. 15′Recap and planning
  2. 60′Practical work
  3. 15′Error analysis and recording

Before you start

Prepare before the session (about 10 minutes, not part of the 90):

  • Environment: the repository's Python project already lists numpy and scikit-learn. Confirm it runs with uv run --project python python -c "import numpy, sklearn; print(numpy.__version__, sklearn.__version__)". Record both versions in your notes. No GPU is needed.
  • Dataset: scikit-learn's bundled digits dataset (sklearn.datasets.load_digits): 1,797 grayscale 8x8 images flattened to 64 features, 10 classes. It ships with scikit-learn, so there is no download or account. If your diagnostic evidence says image data would distract you, you may instead use a tabular binary task built from the synthetic airline sample database (for example, "flight-day load factor at or above 0.90" using only features known before departure). Then read the task texts with your own sizes: your feature count instead of 64, 2 outputs instead of 10, an initial loss near ln 2 instead of ln 10, and the 2x2 confusion table instead of digit pairs. Decide once today and keep the same dataset and split for Weeks 7 and 8.
  • Folders: one project repository with scaffold/ (any provided starter code, unchanged) and src/ (your own code), plus data/, notes/ and runs/. Keep the scaffold separate so reviewers can see what you authored.

Blocks3

  1. Block 1: Recap and planning

    15 min

    Open your project notebook or script and the Week 7 lesson notes (Days 31-35). In five minutes, write from memory: the shape of every tensor in a 64 -> h -> 10 network for a batch of 32, and the loss you expect at initialization for a 10-class softmax with near-uniform outputs (it should be close to ln 10).

    Then write a three-line plan in notes/w07-sat-plan.md: the split you will freeze (seed, proportions, stratified), the single metric you will report on validation (accuracy plus mean log loss), and what would make you stop and debug instead of tuning. The test portion is created today and stays unopened until Week 8.

  2. Block 2: Practical work

    60 min

    Work in this order and do not skip ahead; each step is a check that later steps depend on.

    1. Split once (10 min). Stratified train / validation / test split with a fixed seed. Save the index arrays to data/split_v1.npz and print the class counts per part.
    2. Classical baseline (10 min). Standardize with statistics fitted on the training part only, then fit multinomial logistic regression. Report validation accuracy and log loss.
    3. Shapes and parameters (10 min). Implement the forward pass of a one-hidden-layer MLP in numpy (ReLU hidden layer, softmax output). Add assert statements for every intermediate shape and compute the parameter count twice: by formula and by summing array sizes.
    4. Gradient check (15 min). Write the backward pass (Day 33), then, on 5-10 training examples, compare its analytic gradients with central-difference numerical gradients for a handful of weights in each layer. Print the maximum relative error per layer.
    5. Training loop (15 min). Mini-batch gradient descent (plain SGD or Adam; say which) for a fixed number of epochs. Log train and validation loss each epoch. Before the full run, confirm the network can drive the loss close to zero on a tiny batch of about 32 training examples; if it cannot, debug, do not tune.

    PyTorch is optional. If you use it, keep the same split file and metric and still print shapes and one gradient-check result.

  3. Block 3: Error analysis and recording

    15 min

    Compare the MLP and the baseline on the validation part only: the metric table, the confusion matrix of each, and the three digit pairs the MLP confuses most often. Write a 100-200 word note: what you ran (seed, epochs, learning rate, hidden size), the actual numbers printed, which checks passed or failed, and the next unresolved question for Sunday. Quote only numbers your code printed. A network that does not beat the baseline is a valid result; say so plainly.

Tasks5

Sign in to save your work.

  1. Task 1

    code

    Create the stratified train / validation / test split with a fixed seed, save the indices, fit the scaler on training data only, and train multinomial logistic regression. Report validation accuracy and mean log loss.

    The expected evidence for this task is shown after you save your own output.

  2. Task 2

    code

    Implement the numpy forward pass for 64 -> h -> 10 with ReLU and softmax. Assert the shape of the input batch, pre-activation, hidden activation, logits and probabilities, and compute the parameter count by formula and by summing array sizes.

    The expected evidence for this task is shown after you save your own output.

  3. Task 3

    code

    Compare analytic and central-difference numerical gradients for sampled weights in both layers on a few training examples, and check that the initial loss is near ln 10 for small random weights.

    The expected evidence for this task is shown after you save your own output.

  4. Task 4

    code

    Train the MLP with mini-batches for a fixed number of epochs, logging train and validation loss per epoch. First show that it can overfit a tiny batch of about 32 training examples.

    The expected evidence for this task is shown after you save your own output.

  5. Task 5

    writing

    Write the 100-200 word Saturday note comparing MLP and baseline on validation, listing which checks passed, and stating the next unresolved question.

    The expected evidence for this task is shown after you save your own output.

Rubric4 criteria

CriterionMeetsExceeds
Split and baseline integrityOne seeded, stratified split is saved and reused; preprocessing is fitted on training rows only; the baseline is reported on validation with the same metric as the MLP; test data are untouched.Also states why logistic regression is a fair baseline for this data and checks that the split is reproducible by re-running it and comparing index hashes.
Shape and gradient correctnessAll intermediate shapes are asserted, the parameter count is derived and matches, and the gradient check passes or a failing layer is located and explained.Explains why central differences are used, why the check runs on a few examples rather than a batch of thousands, and what a relative error that is too large would point to.
Training evidenceThe tiny-batch overfit check is shown before the full run, and train and validation loss are logged per epoch with all hyperparameters recorded.Reads the curves correctly (for example, whether validation loss has started to rise) and links that reading to a concrete Sunday experiment.
Honest recordingThe note quotes only printed numbers, says plainly whether the MLP beat the baseline, and names one open question.Separates what the result shows (one seed, one split, validation only) from what it does not show, without overclaiming.

Required output

An executed small network with shape and update checks and an explained training result: the split file, the baseline and MLP validation metrics from the same split, passing shape assertions, the gradient-check output, the train/validation loss log, and a 100-200 word note with the next unresolved question.

Studios are tracked separately from the 60 core lessons; completing one never completes a lesson.