08-13-2026, 08:17 AM
Heads-up for anyone building internal knowledge bots / RAG systems.
I've reviewed a dozen RAG implementations in the last six months and most of them (yes, most) have a single-failure-mode I'd classify as a serious security issue: they do embedding similarity search over ALL documents in the corpus, and rely on the LLM "to be smart about not revealing unauthorized information."
That is not how security works. An LLM is a text completer, not an access control system.
If your vector database contains HR documents, executive compensation files, customer PII, and engineering source code — and every employee's chat session can potentially pull chunks of all of those into its context window when answering innocent questions — then you are one prompt-injection or confused-deputy bug away from a major data leak. This is not theoretical. It's happened.
The fix isn't complicated, it's just engineering:
- Tag every document/chunk with an ACL (who is allowed to see it) at INGEST time, not at query time.
- FILTER by ACL at retrieval time — pass the user's identity/groups to the vector query and exclude chunks they shouldn't see. The LLM should never even see those chunks.
- Same for source citation: don't link to documents the user can't open.
- Log which documents are retrieved for every query. You want an audit trail if something leaks.
- Don't rely on "don't answer if the user doesn't have permission" in the system prompt. That's the weakest possible control.
If you're using a managed vector DB (Pinecone, Weaviate, Qdrant, pgvector), all of them support metadata filtering. There's no excuse for skipping this, and IMO any RAG tutorial that doesn't include permissions in its example schema is teaching people to build vulnerable systems.
Anyone else had to retrofit ACLs onto an existing RAG deployment after the fact? Way more painful than building it in from day one.
I've reviewed a dozen RAG implementations in the last six months and most of them (yes, most) have a single-failure-mode I'd classify as a serious security issue: they do embedding similarity search over ALL documents in the corpus, and rely on the LLM "to be smart about not revealing unauthorized information."
That is not how security works. An LLM is a text completer, not an access control system.
If your vector database contains HR documents, executive compensation files, customer PII, and engineering source code — and every employee's chat session can potentially pull chunks of all of those into its context window when answering innocent questions — then you are one prompt-injection or confused-deputy bug away from a major data leak. This is not theoretical. It's happened.
The fix isn't complicated, it's just engineering:
- Tag every document/chunk with an ACL (who is allowed to see it) at INGEST time, not at query time.
- FILTER by ACL at retrieval time — pass the user's identity/groups to the vector query and exclude chunks they shouldn't see. The LLM should never even see those chunks.
- Same for source citation: don't link to documents the user can't open.
- Log which documents are retrieved for every query. You want an audit trail if something leaks.
- Don't rely on "don't answer if the user doesn't have permission" in the system prompt. That's the weakest possible control.
If you're using a managed vector DB (Pinecone, Weaviate, Qdrant, pgvector), all of them support metadata filtering. There's no excuse for skipping this, and IMO any RAG tutorial that doesn't include permissions in its example schema is teaching people to build vulnerable systems.
Anyone else had to retrofit ACLs onto an existing RAG deployment after the fact? Way more painful than building it in from day one.
