Neural networks, deep learning, NLP · insight-0462

Score embeddings with the similarity function the model was trained for

Situation
You build semantic search over fare-rule documents and pick cosine similarity by habit for an embedding model you have not read about.
Action
Read the model card for its intended score function. Use dot product for dot-tuned models; for models that output normalized vectors, cosine and dot product are equivalent, so normalize once and use the faster dot product.
Reason
Sentence Transformers documents some models as tuned for dot-product similarity and others as producing length-1 vectors usable with dot product, cosine or Euclidean distance.
Boundary
Semantic search, deduplication, clustering and retrieval-augmented generation.

Symptom or risk

Models trained for dot-product scoring encode useful information in vector length; cosine discards it and ranking quality can drop without any error.

If no guidance exists, compare both functions on labelled queries.

Dot-product scores reward vector length, so check whether length tracks something unwanted, such as document length, in your data.

Changing the index's distance metric without re-evaluating retrieval quality.

Sentence Transformers documents that some pretrained models were tuned for dot-product scoring, while others produce normalized vectors usable with dot product, cosine similarity or Euclidean distance.
Pretrained Models (Sentence Transformers documentation)
Steck et al. note cosine similarity can work better or worse than the unnormalized dot product in practice.
Is Cosine-Similarity of Embeddings Really About Similarity?