LLMs, GenAI, RAG, agents, evaluation · insight-0564

Split the eval set into a development set for iteration and a locked test set for decisions

Situation
A team iterates on prompts for weeks against one 300-item eval set and reports the final score as expected production quality.
Action
Keep a development split for daily iteration and a locked test split scored only at decision points; log how many times the test split was used, and refresh it when that count grows.
Reason
Selecting among many variants on one set overfits that set; this is the multiple-comparisons and selection-bias problem from classical validation.
Boundary
Prompt engineering, fine-tune selection, retrieval tuning.

Symptom or risk

Dozens of prompt variants were chosen against the same items, so the final score is optimistic, as with repeated test-set peeking.

Very small eval sets may need cross-validation-style rotation instead of a fixed split.

Fewer items for iteration and some slowdown at decision points.

Peeking at test failures and fixing those specific items.