Your RAG pipeline has no idea who is asking

The setup is common enough that you have probably seen it. A support assistant answers questions from a customer’s own documents. Each customer is a workspace. Documents get chunked, embedded, and written to one shared vector index, and every chunk carries a tenant_id in its metadata so the system knows who owns it.

Then the retrieval code looks like this:

def answer(question: str, tenant_id: str) -> str:
    hits = index.similarity_search(question, k=8)
    context = "\n\n".join(h.page_content for h in hits)
    return llm.invoke(PROMPT.format(context=context, question=question))

tenant_id arrives as an argument and is never used. The top eight chunks across the whole index become context, no matter who uploaded them. The workspace boundary exists in the ingestion pipeline and in the metadata, and nowhere else.

Why nobody noticed

The answers were good. Good enough that nobody looked at where the context came from.

Most questions a customer asks are about their own documents, and their own chunks score highest, so the retrieved set is usually correct by accident. Cross-tenant leakage only shows up when a question is semantically closer to somebody else’s document than to any of yours. The test suite ingested a single fixture workspace, so there was never a second tenant to leak from. Staging had one tenant too.

In the version of this I keep seeing, it surfaces the same way every time: a user asks something generic (“what are the renewal terms?”) and the assistant quotes a clause with another company’s name in it. Then someone greps the retriever and finds the unused argument.

Spotting it in your own code

Look at every call into the vector store and ask where the tenant filter is applied.

• Not at all. The case above. Search for calls to similarity_search, query, search or whatever your client exposes, and check whether a filter argument is passed at every call site.
• After retrieval. Some code fetches k results and then drops the ones with the wrong tenant_id in Python. That stops the leak, but a tenant with less data can get zero results back when other tenants’ chunks fill the top k. It is also a tell that the filter was bolted on later.
• Through a kwarg that is easy to omit. retriever.invoke(q, filter={...}) is correct exactly as long as every caller remembers the kwarg. New endpoints will forget.

Metadata filters also behave differently across stores. Some apply the filter before the nearest-neighbour search, some apply it after fetching a candidate set, and a couple will silently ignore a filter key that was never indexed. Read your store’s docs for the phrase “pre-filtering” and check that the metadata field is actually indexed for filtering, not just stored.

The fix

Filter at query time, in the store, and make the filter impossible to leave out.

With pgvector it is a WHERE clause on the same query that does the distance ranking:

SELECT id, content
FROM chunks
WHERE tenant_id = $1
ORDER BY embedding <=> $2
LIMIT 8;

Check that the index cooperates. An HNSW index on embedding alone ranks first and filters second, and on older pgvector versions a selective WHERE can return fewer rows than the LIMIT asks for. Newer releases handle this with iterative scans, or you can partition the table by tenant so the planner narrows before it ranks.

With a hosted store, pass the filter in the query, and then stop passing it. Wrap the client so the tenant is bound at construction time:

class TenantRetriever:
    def __init__(self, store, tenant_id: str):
        self._store = store
        self._filter = {"tenant_id": {"$eq": tenant_id}}

    def search(self, query: str, k: int = 8):
        return self._store.similarity_search(query, k=k, filter=self._filter)

A request handler builds a TenantRetriever(store, request.tenant_id) and there is no unfiltered method left to call. Any shape works as long as the unsafe call is gone from the codebase.

Three ways to draw the boundary, in rough order of cost:

Approach	Isolation	Cost	Use it when
Metadata filter on one index	Logical, depends on every query	Cheapest	Many small tenants
Namespace or collection per tenant	Enforced by the store	Cheap until the tenant count is large	Tens to low thousands of tenants
Separate index or database per tenant	Physical	Expensive, operationally heavy	Regulated data, a few large tenants

The middle row is a good default. Pinecone namespaces, Qdrant collections, Weaviate’s multi-tenancy mode, or a Postgres schema per tenant will all refuse a cross-tenant query outright, so a forgotten filter fails loudly instead of leaking quietly.

Add the test that would have caught it

Ingest two tenants with documents on the same topic. Query as tenant A. Assert that every returned chunk has tenant_id == "A". Do this in the integration suite against the real store, not a mock. It takes ten minutes to write and it turns a data leak into a red build.

Log the tenant of every retrieved chunk next to the tenant of the request, and page someone on a mismatch.

The vector store is a database. It has all of your customers’ data in it and none of the row-level security you already built for the other databases.

Application code rarely talks to the vector store through the same layer that enforces authorization everywhere else, so the checks on the ORM and the API never reach it. You have to put them back. This is the one people regret most, and it is usually a two-line fix once someone finds it.