Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Assert prediction and target shapes right before every loss call
A regression network returns predictions of shape (N, 1) while the target tensor is (N,), for example predicted load factor per flight-day.
easydocumented fact - Count parameters by hand before training and compare with your number of rows
You size a multilayer perceptron for a tabular problem such as predicting final bookings from 20 booking-curve features.
easydocumented fact - Feed raw logits to CrossEntropyLoss, never softmax outputs
A multi-class model ends in nn.Softmax and its output goes straight into nn.CrossEntropyLoss.
easydocumented fact - Use BCEWithLogitsLoss instead of a sigmoid followed by BCELoss
A binary model, such as a no-show or fraud classifier, ends in a sigmoid layer and is trained with BCELoss.
easydocumented fact - Check that the first loss value equals what chance predicts
You start a new classifier training run, for example a 5-class complaint router.
easypractitioner heuristic - Make sure BatchNorm uses running statistics when you evaluate and serve
Validation accuracy differs between two scoring scripts, or a model scores the same image differently in a batch of one and in a full batch.
easydocumented fact - Turn dropout off at inference unless you are deliberately sampling
A deployed demand model returns slightly different predictions for the same request, and nobody changed the code.
easydocumented fact - Overfit a tiny batch before launching full training
A new model or data pipeline is ready and you are about to start a multi-hour training run.
easypractitioner heuristic - Compute input normalization statistics on the training split only
You standardize tabular inputs for a neural net with a mean and standard deviation computed on the full dataset before splitting.
easydocumented fact - Beat a tuned gradient-boosting baseline before shipping a neural net on tabular data
A team proposes a deep model for a tabular problem, such as predicting final bookings per flight from booking-curve, fare and calendar features.
easyempirical finding - Start text classification with TF-IDF and a linear model
You need to route airline customer complaints into 8 queues and the first idea is to fine-tune a Transformer.
easyempirical finding - Fit the vectorizer vocabulary and IDF weights on training text only
A notebook calls TfidfVectorizer().fittransform(alltexts) and then splits the matrix into train and test.
easydocumented fact - Report per-class recall and macro averages for long-tail intent classifiers
A banking chatbot intent model reports 94% accuracy over 60 intents, and most traffic falls into five intents.
easypractitioner heuristic - Do not treat a t-SNE or UMAP picture of embeddings as evidence of separation
A slide shows customer-message embeddings in 2D with neat colored clusters and claims the model separates segments.
easydocumented fact - Look at a labelled grid of augmented samples before you train on them
You add random crops, flips, jitter or noise to a training pipeline and plan to judge the result only by validation accuracy.
easypractitioner heuristic - Write a run manifest that lets a stranger repeat the experiment
A model that scored well three weeks ago cannot be rebuilt: the notebook, the library versions and the data extract have all changed.
easypractitioner heuristic - Train once with the inputs zeroed to prove the model uses its features
A network reaches a decent score, but its loss curve looks similar to what a constant prediction would give.
easypractitioner heuristic - Quote an error bar that reflects the size of a small test set before comparing networks
You compare two neural classifiers on a hold-out of 120 rows and one is ahead by four points.
easyempirical finding - Get a model that overfits the full training set before you add regularisation
A new network underperforms and you immediately add dropout, weight decay and augmentation to it all at once.
easypractitioner heuristic - Shuffle the training loader explicitly, because the default does not
A DataLoader is built from rows exported sorted by route, date or label, and training loss is jagged with a repeating pattern of the same length as the epoch.
easydocumented fact - Normalise Unicode text to one form before tokenising, hashing or deduplicating
Turkish messages arrive from several apps and keyboards, and the same letter with a cedilla or breve is sometimes one precomposed character and sometimes a base letter plus a combining mark.
easydocumented fact - Print what a stock stop-word list removes before applying it to complaint or sentiment text
A pipeline strips the packaged English stop-word list before TF-IDF on airline complaints because the tutorial did.
easyempirical finding - Measure pretrained word-vector coverage on your own tokens, by slice, before relying on static embeddings
GloVe or word2vec vectors feed a complaint classifier, and the text is full of flight codes, brand names and inflected Turkish forms.
easypractitioner heuristic - Do not use raw BERT [CLS] or mean-pooled outputs as sentence embeddings for similarity
A prototype embeds sentences with a base BERT checkpoint, takes the [CLS] vector or the mean of the last layer, and ranks by cosine.
easyempirical finding - Set padding_idx on embeddings and still mask the pool, or padding length leaks into the features
A text model uses nn.Embedding with pad id 0 and mean-pools token vectors over the padded length.
easydocumented fact