Neural networks, deep learning, NLP · insight-1445
Count how often each item shows up in top-k lists to catch hub documents in embedding search
- Evidence type
- practitioner heuristic
- Difficulty
- hard
- Roles
- Data scientist, Applied AI, ML engineer
- Industries
- e-commerce, saas
Summary: situation, action, reason, boundary
Situation
In a vector search over support articles, the same handful of articles appears in the top 10 for many unrelated queries.
Action
Over a sample of queries, count how many top-k lists each item enters and inspect the skew; if a few items dominate, test local scaling or mean centring on labelled queries.
Reason
Radovanovic et al. show that in high dimensions the distribution of k-occurrences becomes skewed, with a few hubs in many neighbour lists; Conneau et al. introduced CSLS, which lowers scores of words in dense neighbourhoods using their average similarity to their nearest neighbours.
Boundary
Dense retrieval, duplicate detection and cross-lingual matching.
Symptom or risk
Averaged relevance metrics hide that a few hubs crowd out the specific answers users needed.
Exceptions
Broad overview pages may be correct answers to many queries.
Trade-offs
Neighbour statistics per item add compute; scaled scores are no longer plain cosine.
Pitfalls
Hand-blacklisting hubs; keeping a cosine threshold after rescaling.
Example
Hypothetical: in 1,000 queries with top-10 lists, 50 of 20,000 articles fill 3,000 of the 10,000 slots.
Lessons and concepts
Claims and their sources
Radovanovic, Nanopoulos and Ivanovic show that the k-occurrence distribution becomes skewed as dimensionality grows, producing hubs among nearest neighbours of many points.
Conneau et al. define CSLS in Section 2.3 to counter hubness by subtracting each word's average similarity to its K nearest neighbours in the other language.