LLMs, GenAI, RAG, agents, evaluation · insight-1622
Keep an evidence pack indexed to a risk framework, and label it evidence rather than compliance
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- Applied AI, Data scientist, ML engineer
- Industries
- banking, public-sector
Summary: situation, action, reason, boundary
Situation
Security, privacy or audit teams ask for proof that an LLM feature was assessed, tested and approved.
Action
Store a versioned pack per release: intended use and a model card (performance across conditions, evaluation procedure), data flow, threat model, eval and red-team reports with dataset and prompt versions, approvals, open risks with owners and the vendor terms snapshot. Index entries by the framework's functions (Govern, Map, Measure, Manage in the NIST AI RMF), state scope and limits, and leave the legal meaning to counsel.
Reason
NIST describes the AI RMF as a voluntary framework and publishes a generative-AI profile with suggested actions. A mapping shows where evidence lives; it does not establish conformity or legal compliance.
Boundary
Regulated or high-consequence features and any organisation that expects audits.
Symptom or risk
Evidence is scattered across chat threads and notebooks, and people assert that they follow a framework without showing which artefact answers which control.
Exceptions
Prototypes with no real users or data.
Trade-offs
Upkeep cost, and a stale pack is worse than an honest gap.
Pitfalls
Writing the pack once at launch, and claiming conformity from a mapping table.
Lessons and concepts
Claims and their sources
NIST AI RMF 1.0 was released January 2023 for voluntary use and is organised around the Govern, Map, Measure and Manage functions.
NIST AI 600-1 is a generative-AI profile of the AI RMF with suggested actions.
Model cards are short documents reporting performance across conditions, intended use contexts and evaluation procedure.