LLMs, GenAI, RAG, agents, evaluation · insight-1528
Follow the embedding model's own input format: query and passage prefixes, normalisation, similarity metric
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Applied AI, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
You swap an open embedding model into a working pipeline using the same encode call, and recall drops without any error message.
Action
Read the model card, encode queries and passages with the prescribed prefixes or instructions, use its stated similarity metric and normalisation, and cover this with a unit test that fails if the prefix is dropped.
Reason
The E5 model card states that its inputs must start with a query or passage prefix because that is how the model was trained, and that omitting them degrades performance; other models have different conventions.
Boundary
Self-hosted or open-weight embedding models, and any model swap in an existing retrieval path.
Symptom or risk
Models trained with asymmetric prefixes or a specific similarity function degrade quietly when queries and passages are encoded the way the previous model expected.
Exceptions
Hosted models whose API applies the required formatting for you; confirm in the vendor documentation.
Trade-offs
Per-model formatting logic belongs in one wrapper, which adds code to maintain per model family.
Pitfalls
Applying the passage prefix at index time but forgetting the query prefix at serving time, so only half of the pair is wrong.
Lessons and concepts
Claims and their sources
The e5-base-v2 model card says inputs should start with "query: " or "passage: ", that the prefix is required because of how the model is trained, and that omitting it causes a performance degradation.