Skip to content
ML Academy

RAG and agents

Interview question

  • seniorlevel
  • 15 minsuggested time
  • 3Applied AI, ML engineer, Data scientist

A support assistant answers policy questions with retrieval over the policy library. Policies are re-published under the same title, and the index contains the current version and up to two superseded versions of each. In a 500-question audit (hypothetical), 24% of questions are about policies that have three indexed versions. Users complain that answers quote old fees, yet the retrieval hit rate for "any version of the right policy" is 98%. Explain the failure and design the fix and the evaluation.

Tests: Handling versioned or near-duplicate documents in retrieval: how duplicates crowd the top-k and mix conflicting facts, metadata filters and supersession, and evaluating currency separately from relevance.

Sign in to practise: answers are saved before model answers are shown.