LLMs, GenAI, RAG, agents, evaluation · insight-1615
Route data by classification at prompt assembly instead of trusting every caller
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- healthcare, public-sector
Summary: situation, action, reason, boundary
Situation
Several features share one LLM gateway, some documents are restricted, and the approved provider terms cover only less sensitive classes.
Action
Carry a classification label with every chunk and field. At prompt assembly, take the highest label in the request, compare it with the endpoint's approved class (provider, region, retention mode) and block or route to an approved endpoint. Log every denial.
Reason
A rule keyed to data class can be enforced mechanically, while a policy document cannot stop a prompt that is already built. The label of a concatenation is the highest label among its parts.
Boundary
Organisations with more than one data class and more than one model endpoint or provider.
Symptom or risk
A new feature concatenates a restricted document into a prompt for an endpoint not approved for that class, because nothing in the gateway knows the class.
Exceptions
Single-purpose systems that only ever handle one class.
Trade-offs
Labelling effort up front, and some features lose context when restricted parts are dropped.
Pitfalls
Labels lost during chunking or joins, and a gateway bypassed by direct provider keys in feature code.