Concept map: Embeddings and representation learning

Taught and used in

Deep learning and neural networks, NLP

Insights 31

• Cluster in the original feature space, not on a t-SNE map, and use the map only to look
• Check how many parameters your embedding tables hold before adding ID features
• Start transfer learning with a linear probe, then fine-tune if it pays
• Inspect how the tokenizer splits your domain codes and numbers
• Score embeddings with the similarity function the model was trained for
• Do not reuse a cosine-similarity threshold across embedding models
• Watch for vocabulary drift in embedding-based models and adapt the encoder only when it pays
• Judge custom-trained embeddings by downstream lift, not by nearest-neighbour anecdotes

Projects 9

• Same product, different shop: TF-IDF vs sentence embeddings for product matching on Abt-Buy
• Routing airline complaints: TF-IDF + logistic regression vs a small neural text classifier
• Flagship: passenger-rights document Q&A with a retrieval baseline, one controlled improvement and an honest generation mode
• Hybrid lexical/latent retrieval and lightweight reranking ablation over the academy's own knowledge base
• Chunking experiment on long technical papers: evidence recall at a fixed context budget
• Intent classification for banking support: LLM zero/few-shot vs a TF-IDF baseline on cost, accuracy and failure modes