How should a multi-tenant RAG platform partition vector indexes for scalability, isolation, and retrieval performance?

Start with the boundary you must enforce

An index layout is a performance and operations choice. It is not sufficient authorization on its own. Every chunk needs immutable provenance and current access metadata, and the retrieval service must derive the tenant and caller permissions from authenticated server-side context. It must not accept an arbitrary tenant_id, namespace, index name, or region from the browser as a trusted routing instruction.

For each candidate chunk, the minimum useful metadata normally includes a canonical tenant ID, document ID, chunk ID, source version, access-policy or ACL revision, content classification, embedding schema version, region or data-residency cell, and lifecycle state. The exact ACL representation can be a policy ID, group IDs, or another compact authorization key. The key point is that the authorization data travels with every embedded chunk and is evaluated at retrieval time. OWASP recommends per-chunk access-control metadata, query-time enforcement, tenant isolation, and retrieval logging because document-level checks made only at ingestion can become stale. OWASP RAG Security Cheat Sheet

This changes the common shortcut of searching first and filtering later. Suppose a global search returns ten nearest chunks, all from tenant B, for a request from tenant A. If the application removes tenant B's chunks after the search, it returns no usable context even if tenant A has many relevant chunks ranked just below the first ten. Raising global k helps unpredictably, but cannot guarantee enough authorized results. Worse, the unauthorized candidate text, IDs, or scores have already crossed a security boundary and may be logged, cached, reranked by another service, or included in a prompt by mistake. OWASP explicitly warns against relying solely on post-retrieval filtering in multi-tenant vector stores. OWASP vector and embedding guidance

The four index patterns

The names differ by product. A namespace can be logical in one system and backed by a separate index or shard in another. Treat the following as architectural patterns, then confirm the actual query, filtering, isolation, backup, and billing behavior of the chosen service.

The table is a placement decision, not a permission model. A dedicated index can reduce the chance that one query accidentally searches another tenant, but it does not remove the need to verify the caller's tenant and in-tenant permissions. Conversely, a shared index can be safe only if the database applies the authorized filter during search and the service prevents callers from weakening it.

Vendor documentation illustrates the difference. Qdrant recommends payload partitioning for large numbers of small, similarly sized tenants, dedicated user-defined shards for a modest number of larger tenants, and a tiered combination of both. It warns that each collection or dedicated shard has resource overhead. Qdrant multitenancy guide Weaviate's multi-tenancy feature stores each tenant in a separate shard and gives each shard its own vector index, while its documentation also describes active, inactive, and offloaded tenant states. Weaviate tenant operations Pinecone documents namespaces as its tenant-isolation approach and notes that a metadata-filtered query in one large namespace scans that namespace regardless of the filter. Pinecone multitenancy guide

How filtering affects recall and latency

Approximate nearest-neighbor search, commonly HNSW or IVF, does not behave like a relational WHERE clause. In an ANN search, a restrictive filter may force the engine to explore more candidates to find enough permitted points. Some engines can combine the filter with graph traversal, can choose an exact search for a small filtered set, or maintain filter-aware structures. Others apply a filter after the ANN results are gathered. The API shape alone does not reveal which case applies.

The safe target is an authorized top k result set: the engine receives the tenant and ACL predicate as part of the vector query and returns up to k chunks that satisfy it. OpenSearch calls this efficient kNN filtering and notes that it applies the filter during vector search. It contrasts that with post-filtering, which can return significantly fewer than k results under a restrictive predicate. OpenSearch vector filtering Elasticsearch makes the same distinction and documents a useful tradeoff: a highly selective pre-filter can make an ANN search explore more of the graph, while an engine may choose brute force over the filtered set when that is cheaper. Elasticsearch approximate kNN and filtering

Example with many small tenants

This is a hypothetical capacity example. Imagine 100,000 tenants with 1,000 chunks each, or 100 million chunks total. Giving every tenant a private HNSW index creates 100,000 independent graph structures, write queues, backup objects, and lifecycle records. Even if each tenant's data is small, the fixed index and operational cost is repeated 100,000 times. A regional shared index with a server-enforced tenant filter is usually the sensible starting point, provided its filtered-retrieval behavior is measured at the observed tenant sizes.

Now consider one query with k = 20 against a global index that uses application-side post-filtering. If a small tenant's chunks are only 1 in 100,000 of the corpus and similarity is not strongly tenant-clustered, it is entirely plausible that none of the global top 20 belongs to that tenant. Returning an empty answer would be a recall failure, not evidence that the tenant has no relevant document. The takeaway is to execute the tenant predicate in the database search path, or route the request to a tenant-specific partition before vector search.

Example with a few large tenants

Assume 20 enterprise tenants each have 5 million chunks, together still 100 million chunks. Each has its own retention schedule, peak traffic window, and potentially different data-residency contract. Dedicated indexes or dedicated shards let the platform tune index parameters, scale replicas, rebuild one tenant's embedding generation, restore one tenant from backup, and apply a regional placement without disturbing the others.

The action is to compare dedicated indexes against an equivalent shared filtered index using the same queries and k, then measure authorized recall, p95 and p99 latency, peak memory, ingest throughput, and recovery time. The takeaway is that the same total vector count can justify the opposite layout when its distribution, traffic, and policy requirements change.

Example with uneven growth

Consider 20,000 tenants with 500 chunks each, plus 25 tenants with 2 million chunks each. Start the 20,000 small tenants in a shared regional pool. Put the 25 large tenants into their own partitions from the outset, or promote them once tests show that their ingest, query volume, reindex churn, or security requirements consume a disproportionate share of the shared pool.

This is the core hybrid case. The routing catalog sends a request to either the shared pool or a dedicated destination. It does not fan every query across both permanently. During a migration, temporary dual-read or shadow comparison is useful, but the steady state has one authoritative retrieval destination for each tenant and embedding generation.

A practical placement framework

Use the following factors together. A hard compliance or security requirement overrides a cost optimization.

Start with written thresholds, even if they are provisional. For example, promote a tenant when it requires a separate region or encryption key, when it has a non-compatible embedding schema, when its sustained query or ingest rate breaches its shared-pool quota, or when a controlled benchmark misses the authorized-recall or p99 target. Tune numerical vector-count or query-rate thresholds from your own measurements. Qdrant publishes one product-specific promotion example of roughly 20,000 points, but it also makes clear that shards have meaningful overhead, so that value should not be copied into another engine or workload. Qdrant tiered-multitenancy limitations

Build a routing catalog before building many indexes

The retrieval gateway needs a small, strongly controlled catalog that maps an authenticated tenant to its current storage placement. Keep it outside the vector record itself so it can be updated atomically and audited.

tenant_id
  -> region or data-residency cell
  -> isolation tier: shared | logical partition | dedicated
  -> physical destination: cluster, collection, namespace or shard key
  -> active embedding schema and index generation
  -> retrieval policy revision and quota class
  -> migration state: stable | dual-write | shadow-read | cutover | draining

The gateway should obtain tenant_id and user or service principal from the authentication layer, look up the route, construct the database query with the mandatory tenant and ACL predicate, and reject an invalid or stale route. A client can request a search term, not a privileged destination. For shared pools, add the tenant predicate even when the router has selected a tenant-aware namespace or shard. Defence in depth catches bad writes, bugs, and migration mistakes.

For documents shared intentionally across several tenants, avoid copying an ever-growing tenant-ID list into every chunk. Model an authorized audience, such as an organization, project, or role group, and use a database capability that can enforce the resulting predicate efficiently. Large dynamic $in lists are a warning sign. Pinecone, for example, documents a 10,000-value limit for $in and $nin metadata filters. Pinecone metadata filter limits and tenancy guidance

Lifecycle, rebalancing, and embedding upgrades

Partitioning becomes valuable when it makes change safer. A dedicated tenant can be re-embedded, compacted, restored, or moved without forcing every tenant through the same event. A shared pool still needs these operations, but should make them generation-based rather than in-place and global where possible.

For an embedding upgrade, do not mix incompatible vector dimensions, distance metrics, or preprocessing rules in one physical index. Create a new index generation or compatible collection, backfill chunks from a consistent source snapshot, and retain each chunk's source version and embedding schema. Shadow queries against an evaluation set, compare authorized retrieval quality and latency, then update the routing catalog. Roll back by restoring the previous route, not by deleting the old generation first.

For a tenant promotion from shared to dedicated, use an ordered migration:

1. Create the destination in the same approved region, with the target schema, encryption policy, and access controls.
2. Backfill a source snapshot, then dual-write new, changed, and deleted chunks using idempotent document and chunk IDs.
3. Verify vector counts, source-version coverage, mandatory tenant metadata, ACL revision, and a tenant-scoped retrieval test set.
4. Shadow-read or compare results without exposing the shadow result to users. Cut the route to the destination only after correctness and SLO gates pass.
5. Keep the old partition read-only through a defined rollback window, then delete it and confirm that backups, caches, and derived data follow the tenant's retention policy.

Demotion uses the same discipline in reverse. Never rebuild a dedicated tenant into a shared pool by bulk-copying data without applying current permissions, deletion tombstones, and region rules. Rebalancing across nodes or regions is a data migration with an access-control change surface, not merely a placement optimization.

Security, encryption, backup, and regional placement

Encrypt vectors and associated chunk text in transit and at rest, but do not confuse encryption with query authorization. Embeddings can expose information through similarity probing or inversion, so treat them as sensitive derived data. OWASP recommends encrypting embeddings at rest for regulated data, keeping source permissions on chunks, enforcing retrieval-time access checks, and logging retrievals. OWASP RAG security guidance

If a tenant needs a distinct customer-managed key, separate cloud account, legal hold, or data-residency boundary, make that requirement explicit in the router and use a physical placement that the provider can prove meets it. A namespace may not be enough. Confirm where replicas, snapshots, backups, object storage, observability exports, reranking calls, and response caches reside. In a shared pool, one backup commonly contains several tenants, so selective restore and deletion are operationally harder. In a dedicated index, tenant-level restore is simpler, but the backup count and test burden rise.

Authorization should fail closed. If the routing record is missing, the ACL service is unavailable, or the filter cannot be applied in the vector engine, return an error rather than performing an unscoped similarity search. Record the caller identity, resolved route, filter policy revision, selected chunk IDs, and outcome. Avoid logging raw query text, full embeddings, or chunk bodies by default because those logs can become another cross-tenant data store.

Control noisy neighbors and operations

Shared indexes require workload isolation beyond the vector filter. Enforce per-tenant request concurrency, query rate, candidate budget, result size, ingest rate, bulk-rebuild quota, and storage quota at the gateway. Keep background compaction, index build, and embedding jobs in separate queues from interactive retrieval. A tenant that writes millions of new chunks should not saturate the same resources that serve another tenant's chat request.

Dedicated placement is not a free pass. It multiplies index build and compaction work, capacity reservations, monitoring targets, backups, version upgrades, and failure domains. Weaviate, for example, describes each tenant in a multi-tenant collection as a shard with its own vector index and notes that active tenant count is constrained by operating-system open-file limits. Weaviate data structure and tenancy Qdrant similarly warns that many collections or dedicated shards add overhead. Qdrant multitenancy

The operational goal is not to maximize the number of partitions. It is to create enough boundaries to meet access, availability, and tail-latency goals without turning every customer into a separate fleet of databases.

Performance tests that answer the real question

Benchmark with a corpus that preserves tenant-size skew, document types, ACL selectivity, update churn, and real query language. A synthetic uniform distribution can make a shared index look much better than production.

• Establish a relevance set per tenant. Measure authorized recall at k, result count, and answer-grounding quality, not only raw vector latency.
• Run filtered and unfiltered searches separately. Confirm whether the selected engine applies the predicate during ANN search, after ANN search, or by exact search on a filtered subset.
• Sweep context-independent retrieval parameters such as candidate count, graph search depth, IVF probe count, filter selectivity, and k. Track the cost required to maintain authorized recall.
• Apply a mixed workload of foreground queries, ingestion, deletes, re-embedding, compaction, backup, and one deliberately hot tenant. Report per-tenant p50, p95, and p99, not only fleet averages.
• Test failure and lifecycle paths: permission revocation, tenant deletion, route rollback, region failover, restore, promotion, and stale-cache eviction. A query from tenant A must never return a tenant B chunk in the test harness.

Use an exact search over a small sampled tenant corpus as a reference for ANN recall. This is reproducible and more informative than comparing approximate methods only to each other. For highly selective filters, include the product's exact fallback or brute-force behavior in the test. Elasticsearch documents this kind of optimization when the filtered document count is small relative to candidates. Elasticsearch filtered kNN behavior

Observability that supports placement decisions

Make the routing decision measurable. At a minimum, collect these dimensions by region, shared pool, dedicated destination, tenant tier, embedding generation, and authorization-policy revision:

• query count, concurrent requests, rate-limit rejects, p50, p95, p99, timeout rate, and result count;
• vector count, bytes, graph or index memory, disk use, cache hit rate, build and compaction backlog, replication lag, and backup age;
• filter selectivity, candidate count, ANN traversal or probe work where the engine exposes it, exact-fallback rate, and authorized recall evaluation results;
• ingest, update, delete, and re-embedding throughput, plus age of unprocessed deletes and permission changes;
• route changes, dual-write lag, checksum or count mismatches, failed migrations, and time spent in each migration state;
• cross-tenant authorization test failures, denied requests, anomalous query rates, and retrieval audit completeness.

Use these metrics to drive promotion and demotion. For example, a tenant that repeatedly drives shared-pool p99 above the SLO during its morning ingest window may merit a dedicated partition even if its vector count is modest. A formerly large but inactive tenant may be eligible for a colder logical tier if its security and recovery requirements still permit it.

Keep partitioning separate from chunking and embeddings

Document chunking determines how many vectors a document produces, how much context each retrieved item carries, and what metadata must be inherited. Smaller chunks can increase vector count, index cost, and candidate diversity. Larger chunks can reduce vector count but may dilute relevance or exceed a generator's context budget. Chunking should be evaluated per content type and must preserve the tenant, source, ACL, and lifecycle metadata on every resulting chunk.

The embedding model determines vector dimension, distance metric, language coverage, and relevance behavior. It should be versioned in routing and index metadata. An embedding upgrade may require a new index generation, but it does not automatically require a new tenant partition. The decision to split a tenant is about isolation, workload, and independent lifecycle needs. The decision to change chunking or embeddings is about retrieval quality and schema compatibility. Conflating them makes migrations harder to reason about.

Implementation checklist

• Define tenant, in-tenant authorization, region, encryption, retention, and recovery requirements before choosing an index primitive.
• Create a server-controlled routing catalog with tier, destination, region, schema generation, policy revision, quota, and migration state.
• Put tenant and current ACL metadata on every chunk and enforce those predicates in the vector retrieval request.
• Verify the chosen database's filtered-ANN semantics with a restrictive-filter test. Do not infer behavior from API syntax.
• Start compatible small tenants in regional shared pools with quotas, admission control, and a documented promotion policy.
• Put hard-isolation, region-bound, custom-key, incompatible-schema, very large, or very hot tenants in dedicated placements.
• Version embedding schemas and use generation-based rebuilds with shadow evaluation and routing rollback.
• Build tested promotion, demotion, backup, restore, deletion, revocation, and rebalancing runbooks.
• Instrument authorized recall, result count, tail latency, filter selectivity, index resources, migration lag, and cross-tenant test failures.
• Review logs, rerankers, caches, observability exports, and backups as part of the same tenant-boundary design.

Limits and viable alternatives

No partitioning strategy makes a weak authorization layer safe. A dedicated index can still be queried by the wrong caller if the route is not checked. A shared index can be safe, but only if authorization-aware retrieval is implemented and tested. Retrieval layout also does not solve prompt injection, source quality, or answer correctness, although it prevents one class of cross-tenant exposure.

For very small filtered populations, exact vector search may be cheaper and more accurate than maintaining an ANN graph per tenant. For a product with truly hard isolation needs, a separate database deployment or cloud account may be simpler to audit than a shared cluster with complex filters. For tenant-specific retrieval quality requirements, use separate embedding or reranking experiments behind a versioned route, not an untracked mixture of incompatible vectors in one index.

The correct final design is often a small number of regional cells, each with a shared long-tail pool and a controlled set of dedicated tenant destinations. That provides a clear path from efficient consolidation to stronger isolation without a disruptive rewrite.