LLMs, GenAI, RAG, agents, evaluation · insight-0656
Run GenAI design reviews against a shared threat checklist, not ad-hoc brainstorming
- Evidence type
- documented fact
- Difficulty
- easy
- Roles
- Applied AI, ML engineer, Data scientist
- Industries
- public-sector, cross-industry
Summary: situation, action, reason, boundary
Situation
A GenAI feature reaches design review and the security section says "prompt injection mitigated by system prompt".
Action
Walk the data flow against OWASP's LLM Top 10, map relevant adversary techniques from MITRE ATLAS, and record each risk with an owner, a control and a test; use NIST AI 600-1 to connect them to governance.
Reason
Shared catalogues make coverage visible and reviewable. OWASP lists ten LLM risk categories, MITRE ATLAS catalogues adversary techniques against AI systems, and NIST AI 600-1 lists GenAI-specific risks with suggested actions.
Boundary
Day 55 and day 58 design defences, procurement reviews, launch checklists.
Symptom or risk
Whole risk classes (supply chain, data poisoning, unbounded consumption, vector store access) are never discussed.
Exceptions
None; scale the depth of the review to the feature's data and actions.
Trade-offs
Checklists can become box-ticking; require a concrete test for each claimed control.
Pitfalls
Marking risks "not applicable" without stating the assumption that makes them so.
Lessons and concepts
Claims and their sources
The OWASP Top 10 for LLM Applications 2025 lists risk categories from LLM01 Prompt Injection to LLM10 Unbounded Consumption.
MITRE ATLAS is a knowledge base of adversary tactics and techniques against AI systems.
NIST AI 600-1 provides a GenAI risk profile with suggested actions mapped to the AI RMF.