LLMs, GenAI, RAG, agents, evaluation · insight-1531
Measure approximate-index recall against exact search on your own vectors before tuning anything else
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
Retrieval quality is poor after moving from brute-force search to an HNSW or IVF index, and the team starts changing models and prompts.
Action
Run exact search on a sample of queries, compute the overlap of its top-k with the ANN top-k, and raise the query-time knob (ef_search, probes) until overlap is acceptable for the latency you can afford.
Reason
Approximate indexes trade recall for speed and expose parameters that move along that trade-off; pgvector documents both the trade and default values, and the defaults are not tuned to your corpus.
Boundary
Any HNSW, IVF or graph index serving a RAG retriever, especially after metadata filters shrink candidate sets.
Symptom or risk
The approximate index returns different neighbours than exact search, so some of the apparent embedding-model weakness is index recall loss at default query-time settings.
Exceptions
Small corpora where exact search is fast enough; then skip the index.
Trade-offs
Higher query-time effort raises latency and CPU per query.
Pitfalls
Measuring ANN recall without the production filters, which can change how many candidates survive.
Lessons and concepts
Claims and their sources
The pgvector README says an approximate index trades some recall for speed, that query results differ after adding one, and lists hnsw.ef_search (default 40) and ivfflat.probes (default 1) as query-time parameters (as read on 2026-10-12; defaults can change between versions).