Neural networks, deep learning, NLP · insight-0403
Count parameters by hand before training and compare with your number of rows
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Data scientist, ML engineer
- Industries
- aviation
Summary: situation, action, reason, boundary
Situation
You size a multilayer perceptron for a tabular problem such as predicting final bookings from 20 booking-curve features.
Action
Compute for each linear layer, sum, and check it against
sum(p.numel() for p in model.parameters()). Write the ratio of parameters to training rows in the experiment log.Reason
A linear layer stores an (out, in) weight matrix plus an out-sized bias, so the count is exact arithmetic. A mismatch with the framework count reveals an unintended layer or shape.
Boundary
Any dense or convolutional architecture before the first full training run.
Symptom or risk
Without a count you cannot tell whether the model is tiny or has more parameters than training rows, so overfitting and underfitting diagnoses become guesswork.
Exceptions
Parameter count is a weak capacity measure for heavily regularized or pretrained models; use it as a sanity check, not a rule.
Trade-offs
Two minutes of arithmetic; does not replace learning curves for judging capacity.
Pitfalls
Forgetting biases, embeddings or normalization scales; counting frozen parameters as trainable.
Example
20 -> 64 -> 64 -> 1 gives 1,344 + 4,160 + 65 = 5,569 parameters, small next to 40,000 synthetic flight-day rows.
Lessons and concepts
Claims and their sources
A PyTorch Linear layer has a weight of shape (out_features, in_features) and an optional bias of shape (out_features).