LLMs, GenAI, RAG, agents, evaluation · insight-0563

Turn every production failure into a regression test that gates changes

Situation
A model upgrade reintroduces a bug that was fixed months ago, quoting outdated pet-travel rules.
Action
For every confirmed incident, add the case, its expected behaviour and its capability tag to a regression suite; add invariance tests (for example, changing a passenger name must not change the decision); run the suite on every change and block releases that fail.
Reason
Behavioural testing work showed held-out accuracy overestimates real-world performance and that capability-targeted tests reveal failures aggregate metrics miss.
Boundary
Any LLM feature with releases, prompt edits or model upgrades.

Symptom or risk

Fixed failures recur after prompt, model or retrieval changes because the fix was never encoded as a test.

Cases tied to retired policies should be retired with documentation, not deleted silently.

The suite grows and must be pruned and kept fast.

Letting the regression suite double as the tuning set, which turns it into a memorisation target.

Ribeiro et al. showed held-out accuracy overestimates NLP model performance and that a matrix of capability tests exposed critical failures in commercial models.
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList