Introduced on day 45
Pairwise comparison
An evaluation format in which a judge or a person is shown two answers to the same prompt and picks the better one, instead of scoring each answer alone.
In detail
Pairwise judgements are easier and more reliable than absolute scores for ranking close variants, and they are the raw data for preference tuning. They scale quadratically with the number of systems and need order control. For setting a pass bar, rubric grading of single answers is usually more appropriate.