LLMs, GenAI, RAG, agents, evaluation · insight-1624

Rotate every credential that has entered a prompt, a tool result or a trace

Situation
An incident review finds an API key, session token or database URL in a prompt, a retrieved document, a tool response or a stored trace.
Action
Treat exposure as compromise: revoke and rotate first, then search traces, caches, vector indexes, datasets and tickets for the value or its prefix, purge or mask it, and check access logs for use of the old credential. Add the pattern to the output and ingestion filters.
Reason
You cannot establish who copied the text, so rotation bounds the exposure window, and time to rotate is the control. NIST's incident-response guidance frames the goals as preparing, limiting damage and improving detection, response and recovery.
Boundary
Any LLM system whose prompts, tool results or traces can contain secrets.

Symptom or risk

The team closes the leak path and moves on, while the secret remains valid and sits in logs, vendor stores, evaluation sets and screenshots.

Credentials that were provably never valid, such as placeholder values.

Rotation can break integrations; automate it so that the safe action is also the easy one.

Rotating production but not a staging key that shares the secret, and leaving the old key valid for a grace period.

NIST SP 800-61 Rev. 3 states its goals as preparing for incidents, reducing their number and damage, and improving detection, response and recovery.
SP 800-61 Rev. 3: Incident Response Recommendations and Considerations for Cybersecurity Risk Management: A CSF 2.0 Community Profile