Practical insights
- 1000published
- 25in review
- 8topic families
Results
- Test that examples in a batch cannot influence each other's predictions
You wrote custom reshaping, pooling or attention code with view, transpose or reductions over several axes.
mediumpractitioner heuristic - Check how many parameters your embedding tables hold before adding ID features
You add learned embeddings for high-cardinality identifiers such as customer, store or SKU IDs to a neural model.
mediumpractitioner heuristic - Pick softmax or independent sigmoids by asking whether labels can co-occur
You classify flight disruption reports into causes (weather, crew, technical, ATC) and some reports cite several causes.
mediumdocumented fact - Leave regression outputs linear and choose the loss for the statistic you need
A network predicts a non-negative quantity such as daily energy load or claim cost, and someone adds a ReLU on the output.
mediumpractitioner heuristic - Calibrate a neural classifier before using its softmax output as a probability
A network's predicted probability will drive a cost-based decision, such as how much overbooking risk to accept or when to route a claim to manual review.
mediumempirical finding - Match weight initialization to the activation function
A deep MLP or CNN with ReLU activations trains very slowly, or a tanh network's loss does not move for many steps.
mediumdocumented fact - Run a learning-rate range test before tuning anything else
You start training a new architecture or dataset and are about to grid-search learning rate together with other hyperparameters.
mediumpractitioner heuristic - Use Adam to iterate fast, but test SGD with momentum before final training
Your model trained with Adam is close to final and the remaining question is test accuracy, not speed of iteration.
mediumempirical finding - Early-stop neural training on the decision metric and restore the best checkpoint
A churn network uses early stopping on validation loss with default patience, and the final weights are whatever the last epoch produced.
mediumpractitioner heuristic - Triage a NaN loss in a fixed order, starting with the data
The loss of a fraud model becomes NaN in epoch 3, and the team starts changing the architecture.
mediumpractitioner heuristic - Split by source item before augmenting, never after
You multiply a small labelled image set, such as photos of damaged baggage or defect images, with crops and flips, then split the enlarged set randomly.
mediumpractitioner heuristic - Give the baseline and the neural model the same tuning budget
A comparison report shows a neural model beating logistic regression and boosting on a churn dataset.
mediumpractitioner heuristic - Report results over several seeds before claiming a neural improvement
A fine-tuned text classifier scores 1.2 points higher than the current one on a single run.
mediumempirical finding - Switch on deterministic GPU mode while bisecting a training regression
After a refactor, validation accuracy of a credit-risk network drops by half a point, and you want to find the commit responsible.
mediumdocumented fact - Watch the flatten-to-dense layer, not the convolutions, for parameter blow-up
A small CNN for shelf images or X-ray triage has far more parameters than expected and overfits quickly.
mediumdocumented fact - Start transfer learning with a linear probe, then fine-tune if it pays
You have a few thousand labelled examples and a pretrained backbone, and must decide whether to freeze it or fine-tune it.
mediumempirical finding - Do not expect a plain RNN to carry information across hundreds of steps
You model long sequences, such as months of daily meter readings or long support transcripts, with a vanilla RNN.
mediumdocumented fact - Compare a sequence model of booking curves against lag features in a GBM
You want an LSTM or Transformer to forecast final bookings from the sequence of booking-curve snapshots per flight.
mediumpractitioner heuristic - Use causal masking when the prediction must not see later tokens or events
You use a bidirectional encoder or an unmasked Transformer over a customer's event stream to predict their next action in real time.
mediumdocumented fact - Do not present attention heatmaps as the reason for a prediction
A stakeholder asks why the model flagged a loan application or complaint, and the team shows an attention heatmap over the input words.
mediumempirical finding - Deduplicate near-identical texts across splits before trusting a text score
A complaint or FAQ dataset contains forwarded emails, copy-pasted templates and resubmitted tickets, and you split it randomly.
mediumempirical finding - Check whether a text model learned the template instead of the content
A ticket classifier scores very high, and many tickets come from web forms, partner systems or agents with their own boilerplate.
mediumpractitioner heuristic - Compare tokenizer fertility before choosing a multilingual encoder for a non-English task
You must classify Turkish customer complaints and are choosing between a multilingual encoder and a model pretrained mainly on Turkish text.
mediumempirical finding - Inspect how the tokenizer splits your domain codes and numbers
Inputs contain flight numbers, fare-class codes, booking references, part numbers or ICD codes, and a pretrained model handles them poorly.
mediumpractitioner heuristic - Check where the label evidence sits before truncating long texts
Insurance claim narratives or incident reports often exceed the encoder's maximum length and are cut at the limit.
mediumpractitioner heuristic - Prefer loss weighting to duplicating rare texts when classes are imbalanced
In a ticket classifier, safety-related complaints are 0.5% of messages and the model rarely predicts them.
mediumpractitioner heuristic - Audit test-set labels before ranking models that differ by a point or two
Two text classifiers differ by 1.5 points on a test set labelled quickly by several annotators.
mediumempirical finding - Expect large networks to memorize noisy training labels and plan for it
Training labels come from agents' quick tags and you train a large model for many epochs until training accuracy nears 100%.
mediumempirical finding - Do not reuse a cosine-similarity threshold across embedding models
A duplicate-ticket detector flags pairs above cosine 0.8, and the team swaps in a new embedding model without changing the threshold.
mediumpractitioner heuristic - Judge custom-trained embeddings by downstream lift, not by nearest-neighbour anecdotes
You trained product or route embeddings from co-purchase or co-search sequences and the team validates them by listing the nearest neighbours of a few favourite items.
mediumpractitioner heuristic - Use contextual embeddings when word sense changes the label
You classify short airline messages with averaged static word vectors, and words like "hold" (baggage hold, on hold, hold a seat) carry the meaning.
mediumempirical finding - Route low-confidence predictions to people, but validate the confidence signal first
A claims-intake text classifier will auto-route confident predictions and send uncertain ones to human agents.
mediumempirical finding - Fix the total step budget before you choose a decaying learning-rate schedule
You pick a cosine or linear decay for a run and later decide to train "a few more epochs" because validation is still improving.
mediumpractitioner heuristic - Do not place dropout directly before a BatchNorm layer
A convolutional or dense network has a dropout layer immediately followed by BatchNorm and validation accuracy is slightly worse in evaluation mode than the training curve suggests.
mediumempirical finding - When you add mixup, treat the targets as soft and change the metrics accordingly
You add mixup to an image or tabular network that overfits a few thousand labelled rows, and keep the old accuracy and threshold code.
mediumempirical finding - Recalibrate after adopting focal loss for an imbalanced classifier
A rare-event network (for example, a synthetic no-show flag at 2 percent prevalence) is trained with focal loss and its scores are fed into a cost-based threshold.
mediumempirical finding - Put the task losses on one scale before adding them in a multi-task network
A shared network predicts a revenue amount in euros and a cancellation probability, and the total loss is the plain sum of a squared error and a cross-entropy.
mediumempirical finding - Rank training examples by out-of-fold loss to find labels worth re-checking
A classifier plateaus below what the data seems to allow, and the labels come from a rule, a vendor or a stressed annotation team.
mediumempirical finding - Do not quote one expected-calibration-error number without its binning and a reliability diagram
A report compares two networks by a single ECE value computed with ten equal-width bins on a test set of 800 rows.
mediumempirical finding - Prefer a small deep ensemble over MC dropout when uncertainty must hold up
A neural classifier's confidence will gate automated decisions, and you can afford several times the training and inference compute.
mediumempirical finding - Average the weights of late checkpoints, then recompute BatchNorm statistics
Validation loss of a small network fluctuates by a few percent between neighbouring epochs and you must pick one checkpoint to ship.
mediumempirical finding - Vary the initialisation seed and the data-order seed separately to see which one moves the metric
Two runs of the same configuration differ by half a point and you do not know whether to blame initialisation, shuffling, augmentation or hardware.
mediumempirical finding - Shuffle the training labels once and confirm the validation score collapses to chance
A network scores surprisingly high on a new dataset, and the pipeline joins several tables and applies preprocessing before the split.
mediumpractitioner heuristic - Fit a learning curve over training-set size before buying more labels or a larger model
A team plans to triple labelling spend because validation error is still above target, without knowing how error has moved with data so far.
mediumempirical finding - Test character n-grams before building a stemmer for agglutinative Turkish text
A Turkish complaint classifier uses word-level TF-IDF, and one stem shows up in many surface forms because suffixes stack onto it.
mediumpractitioner heuristic - Measure how much Turkish text is typed without diacritics before trusting a clean benchmark score
A model trained on correctly accented Turkish complaints will read chat and mobile messages where people type c, s, g, i, o, u for the letters with diacritics.
mediumpractitioner heuristic - Use feature hashing when the vocabulary keeps growing, and accept what it gives up
A ticket classifier is refit nightly and a fitted vocabulary keeps meeting new product names, flight codes and typos, while the vocabulary dictionary grows in memory.
mediumdocumented fact - Build a contrast set before letting embedding similarity merge texts automatically
A duplicate-ticket merger links two tickets whenever the cosine similarity of their embeddings passes a threshold chosen on random pairs.
mediumpractitioner heuristic - Remove the common component before averaging static word vectors into text features
Short texts are represented by the plain mean of GloVe or word2vec vectors for a classifier or nearest-neighbour search.
mediumempirical finding - Evaluate zero-shot cross-lingual transfer on labelled Turkish data before launching in Turkish
A multilingual encoder is fine-tuned on English tickets and expected to work on Turkish tickets with no Turkish labels.
mediumempirical finding - Report metrics per detected language when messages mix Turkish and English
Customer messages for an airline are Turkish, English and mixed, and one classifier reports a single accuracy.
mediumpractitioner heuristic - Compare frozen features and fine-tuning at several training sizes before committing
A team has 600 labelled Turkish complaints and argues about frozen embeddings with a linear head versus fine-tuning.
mediumempirical finding - Hold out annotators, not only items, when a few people produced most of the labels or texts
A tone classifier is trained on labels from five agents, two of whom labelled 70 percent of the rows, and the test split is random.
mediumempirical finding - Run a partial-input baseline to expose annotation artifacts before celebrating a text score
A text-pair or template-labelled dataset scores high and you suspect the label writers left stylistic traces.
mediumempirical finding - Split text data by thread or customer so quoted replies and personal style cannot cross the boundary
Support emails are split by message at random, though each thread has several messages that quote earlier ones and come from the same customer.
mediumempirical finding - Exclude text written after the outcome, such as resolution notes and closing remarks
You predict at ticket creation whether a complaint will be escalated, using a text column that holds the full conversation including later agent notes.
mediumdocumented fact - Keep invariance and directional tests for a text classifier next to its accuracy report
A complaint-priority classifier passes its test set and then fails on trivial rewrites in the first demo.
mediumpractitioner heuristic - Read the last real time step of a padded RNN batch, not position minus one
An LSTM classifier over right-padded batches uses output[:, -1] as the sequence summary.
mediumpractitioner heuristic - Use a 1-D CNN with max-pooling as a fast pattern-detector baseline for long sequences
A label depends on short local patterns in long event or text sequences, and recurrent models are slow to train.
mediumpractitioner heuristic - Probe a CNN with texture and capture shifts before moving it to a new camera or catalogue
A product-image or defect CNN trained under one camera and lighting setup is moved to another source, such as customer photos instead of catalogue shots.
mediumempirical finding - Tune the look-back window of a neural forecaster and check it against a linear model on the same windows
A Transformer forecaster is given a much longer history in the hope of better accuracy.
mediumempirical finding - Ensemble neural forecasters over seeds, look-back windows and losses instead of trusting one run
A neural forecaster's backtest error changes visibly from one random seed to the next.
mediumempirical finding - Backtest sequence models in free-running mode, not with teacher-forced inputs
A recurrent forecaster predicts the next 14 days of bookings, and validation loss is computed by feeding the true previous values as inputs.
mediumempirical finding - Exclude duplicate positives from a batch when training a retriever with in-batch negatives
A bi-encoder is fine-tuned on question and answer pairs with in-batch negatives, and several questions share the same answer or the same question appears twice.
mediumpractitioner heuristic