LLMs, GenAI, RAG, agents, evaluation · insight-1598
Map red-team coverage as a grid of risk category by input channel and count empty cells
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, ML engineer
- Industries
- cross-industry
Summary: situation, action, reason, boundary
Situation
A red-team report lists attacks that succeeded and failed, but nobody can say what was never tried.
Action
Build a grid of harm or failure category by input channel and user type, record the number of test cases per cell, mark empty cells explicitly, and fill the highest-risk ones first with human and model-generated cases.
Reason
Test-case coverage is the denominator of any safety claim. Manually written cases are costly and limit coverage, which is why model-generated cases were proposed to widen it.
Boundary
Assistants with several input channels, tools or user populations.
Symptom or risk
Findings cluster where testers were comfortable (direct text prompts), so untested channels such as uploaded documents, tool outputs or other languages look safe by absence.
Exceptions
A narrow single-channel assistant needs a smaller grid.
Trade-offs
Categories are judgment calls, and filled cells are not proof of safety.
Pitfalls
Counting near-identical cases in one cell as coverage.
Lessons and concepts
Claims and their sources
Perez et al. report that manually written test cases are costly and limit coverage, and use one language model to generate test cases for another with a classifier to detect offensive replies.