LLMs, GenAI, RAG, agents, evaluation · insight-0653
Run model-written code in a disposable sandbox with no network and no secrets
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- finance, cross-industry
Summary: situation, action, reason, boundary
Situation
An analytics assistant writes and executes Python or SQL to answer questions about uploaded files or data.
Action
Execute in an isolated, ephemeral container: no credentials, no outbound network by default, read-only data mounts, CPU, memory and time limits, and output size caps; destroy it after each task.
Reason
Code execution turns any prompt manipulation into arbitrary computation. OWASP warns that open-ended extensions such as shell runners increase excessive agency and that improper output handling can reach remote code execution.
Boundary
Code interpreters, notebook agents, data-analysis copilots.
Symptom or risk
Generated or injected code reads environment variables, calls external hosts, fills the disk or runs for hours on shared infrastructure.
Exceptions
Developer tools where the human reviews and runs the code themselves.
Trade-offs
Sandboxes block useful package installs and live data access; provide curated packages and read-only replicas.
Pitfalls
Reusing one long-lived sandbox across users, which leaks files and state between sessions.
Lessons and concepts
Claims and their sources
OWASP recommends avoiding open-ended extensions such as a shell-command runner in favour of granular tools.
Improper handling of LLM output can lead to remote code execution in downstream components.