Introduced on day 50
Retrieval-by-answer table
A two-way count of labelled questions by whether the gold evidence reached the context and whether the answer was correct and supported, used to locate which pipeline stage fails.
In detail
The four cells have different remedies. Evidence present and answer correct is the success cell. Evidence missing points at ingestion, chunking, filters or ranking. Evidence present and answer wrong points at context assembly or generation. Evidence missing and answer correct means the model answered from memory, which is fragile for facts that change. It needs evidence labels, not only reference answers.