Introduced on day 43
Evaluation set
A versioned, labelled collection of real or realistic cases, written and frozen before any prompt or weight change, against which every candidate system is scored.
In detail
Each item carries an id, input, expected answer or rubric, capability tag, source, author and date. A hash or version number makes scores comparable over time; changes produce a new version rather than an edit. A locked portion is used once for final decisions. Overlap between the set and training or few-shot data invalidates the comparison.