Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Treat Monte Carlo dropout uncertainty as a signal to validate, not a calibrated interval
Stakeholders want an uncertainty band around a neural demand forecast and you consider keeping dropout on at inference and averaging many passes.
hardpractitioner heuristic - Watch for vocabulary drift in embedding-based models and adapt the encoder only when it pays
A complaint-topic clustering or classification model built on general-purpose sentence embeddings degrades after a new loyalty programme and new fare products introduce unfamiliar jargon.
hardempirical finding - Re-check calibration on shifted data, because a temperature fitted in distribution may not travel
A temperature-scaled network is deployed to a new region, season or device mix and its probabilities feed a threshold chosen on the old validation split.
hardempirical finding - Include Adam's epsilon in the search space before ranking optimizers
A comparison of SGD with momentum and Adam is run with default settings, and Adam loses by a point on a tabular-style network.
hardempirical finding - Embed numerical features before the MLP when a tabular network trails gradient boosting
A plain MLP on raw standardised numerical features of a tabular problem (fare, days to departure, load factor) underperforms a gradient-boosted model by a clear margin.
hardempirical finding - Do not stop at the first rise in validation loss without checking a longer horizon
An over-parameterised network's validation error rises in mid-training, then, with patience set to five epochs, training halts and a smaller model is chosen instead.
hardempirical finding - Centre or whiten sentence embeddings before nearest-neighbour search and compare with the raw vectors
Scores between unrelated texts from a BERT-based encoder are all high, and every query seems to retrieve similar neighbours.
hardpractitioner heuristic - Count how often each item shows up in top-k lists to catch hub documents in embedding search
In a vector search over support articles, the same handful of articles appears in the top 10 for many unrelated queries.
hardpractitioner heuristic - Count selection data as part of the budget when you only own a few dozen labels
A few-shot text classifier is reported at a score tuned on prompts and learning rates using a large extra validation set, though production will have only 60 labels.
hardempirical finding - Keep per-item annotator votes and test soft labels before collapsing them to a majority
Three annotators label each tone or intent example, the pipeline keeps the majority label, and a third of the items were split 2 to 1.
hardempirical finding - Reweight to the production class prior when labelling effort was split evenly across intents
A labelling budget was split equally over six intents so each has the same count, while production traffic has one dominant intent and several rare ones.
harddocumented fact - Detach the hidden state between truncated-BPTT chunks and treat chunk length as the learnable dependency horizon
You train an RNN on one long stream by carrying the hidden state across consecutive chunks.
harddocumented fact - Do not assume a CNN uses its full theoretical receptive field
You size a CNN on paper so its theoretical receptive field covers the whole image or sensor window and conclude the context is covered.
hardempirical finding