LLMs, GenAI, RAG, agents, evaluation · insight-1543
Resolve group membership at query time, because access lists copied into chunk metadata go stale
- Evidence type
- practitioner heuristic
- Difficulty
- medium
- Roles
- ML engineer, Applied AI
- Industries
- cross-industry, banking
Summary: situation, action, reason, boundary
Situation
Each chunk carries an allowed-groups list copied from the source system at ingestion, and the retriever filters on the user's groups from the session.
Action
Store a stable document id and its access-control reference, resolve the caller's current entitlements from the identity or source system on each query, and test with a user whose access was just revoked.
Reason
Permissions are mutable state owned elsewhere; OWASP lists weak access control and cross-context leaks among the risks of vector stores and recommends permission-aware retrieval.
Boundary
Enterprise assistants over shared drives, wikis, tickets and CRM data.
Symptom or risk
A user who left a team, or a document whose sharing was revoked, still matches until the next re-sync, so the assistant answers with content the person can no longer open.
Exceptions
Public corpora, or very small closed groups where a periodic sync with a short interval is acceptable and documented.
Trade-offs
Live entitlement lookups add latency and a dependency; cache them briefly with an explicit maximum staleness.
Pitfalls
Post-filtering the top-k after retrieval, which can return too few results or leak counts and titles.
Lessons and concepts
Claims and their sources
OWASP's LLM08:2025 page lists weak access control and cross-context leaks in multi-tenant vector databases among vector-store risks and recommends permission-aware vector stores with logical partitioning.