Practical insights

Results

11 results (published only)

• Neural networks

  Feed raw logits to CrossEntropyLoss, never softmax outputs

  A multi-class model ends in nn.Softmax and its output goes straight into nn.CrossEntropyLoss.

  easydocumented fact

• Neural networks

  Overfit a tiny batch before launching full training

  A new model or data pipeline is ready and you are about to start a multi-hour training run.

  easypractitioner heuristic

• Neural networks

  Reproduce the pretrained model's preprocessing exactly before fine-tuning

  You load a pretrained image or text backbone and feed it data prepared by your own pipeline.

  easypractitioner heuristic

• Neural networks

  Start text classification with TF-IDF and a linear model

  You need to route airline customer complaints into 8 queues and the first idea is to fine-tune a Transformer.

  easyempirical finding

• Neural networks

  Report per-class recall and macro averages for long-tail intent classifiers

  A banking chatbot intent model reports 94% accuracy over 60 intents, and most traffic falls into five intents.

  easypractitioner heuristic

• Neural networks

  Score embeddings with the similarity function the model was trained for

  You build semantic search over fare-rule documents and pick cosine similarity by habit for an embedding model you have not read about.

  easydocumented fact

• Neural networks

  Do not treat a t-SNE or UMAP picture of embeddings as evidence of separation

  A slide shows customer-message embeddings in 2D with neat colored clusters and claims the model separates segments.

  easydocumented fact

• Neural networks

  Run the first training step on the largest batch and longest sequence you will see

  A text model trains with dynamic padding, and batch memory depends on the longest document in each batch.

  easypractitioner heuristic

• Neural networks

  Measure pretrained word-vector coverage on your own tokens, by slice, before relying on static embeddings

  GloVe or word2vec vectors feed a complaint classifier, and the text is full of flight codes, brand names and inflected Turkish forms.

  easypractitioner heuristic

• Neural networks

  Do not use raw BERT [CLS] or mean-pooled outputs as sentence embeddings for similarity

  A prototype embeds sentences with a base BERT checkpoint, takes the [CLS] vector or the mean of the last layer, and ranks by cosine.

  easyempirical finding

• Neural networks

  Left-pad batched prompts for decoder-only generation and confirm batched outputs match single runs

  You batch prompts of different lengths for a decoder-only model, using the tokenizer's default right padding.

  easydocumented fact