LLMs, GenAI, RAG, agents, evaluation · insight-1601
Strip invisible characters and inspect extracted text before documents reach the model
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- insurance, cross-industry
Summary: situation, action, reason, boundary
Situation
An assistant summarises uploaded PDFs, emails and web pages, and a reviewer approves its output after skimming the rendered source.
Action
Normalise inbound text by removing the Unicode Tags block (U+E0000 to U+E007F) and other control or format characters you cannot justify, count what was removed per source, and keep the exact extracted text the model received. Flag documents whose extracted text is far longer than what renders.
Reason
The model receives code points and parsed layers, not the rendered page. OWASP notes that injected content can affect a model even when humans cannot perceive it, so a human skim is not a control for this channel.
Boundary
Any pipeline that ingests files, email or web content from outside your team.
Symptom or risk
Text a person cannot see, such as invisible tag-block characters, white-on-white runs or metadata fields, carries instructions that the model reads; the reviewer sees a clean page.
Exceptions
Some scripts and emoji flag sequences legitimately use joiners or tag characters; allow-list by script.
Trade-offs
Aggressive stripping can damage multilingual text; test on real documents first.
Pitfalls
Cleaning only the chat box while files arrive on other paths, and calling this a full injection defence when visible text can inject too.
Lessons and concepts
Claims and their sources
The Unicode Standard defines a Tags block spanning U+E0000 to U+E007F.
OWASP describes injected inputs that affect a model even when humans cannot perceive them, including hidden instructions on a web page and instructions hidden in an image.