Naming a vector store in a system design interview usually belongs after the design work. Once the requirements are clear, choosing among products that meet them is largely a procurement decision. Chunk policy, metadata, filtering, and rebuild strategy establish those requirements.

A vector index is a derived artifact, built from an embedding model version, a chunk policy, and source documents and permissions. The useful questions concern write propagation, lag, rebuild cost, and what a query must filter on before its answer is correct.

Assume a support knowledge base with 2 million documents, retrieval-augmented answers for agents, and a pending embedding model upgrade.

The index is derived; the documents are the system of record

The articles already live somewhere, in a CMS, an object store, or a relational table with versions and permissions. That is the system of record. The index should be reproducible from it and the recorded transformation configuration. An article correction can replace only its affected chunks; a model or global chunk-policy change can require regenerating the whole corpus.

This makes the index a replication problem, with lag and recovery concerns similar to a read replica. A freshly published article is invisible to retrieval until it has been chunked, embedded, and upserted. A correction can keep serving old text until replacement chunks become active and stale ones are excluded. State the lag the product can tolerate: minutes may be acceptable for routine articles, while a changed refund rule may need a tighter bound.

A model migration can proceed gradually, but incompatible embedding versions must remain in separate search spaces. One workable pattern is to build a shadow index from source, run the same query set against old and new using their respective embedding models, and compare retrieval and answer quality. Catch up intervening writes and deletions before cutting over reads; retire the old index after validation.

Concurrent index storage is the sum of the old and new footprints, with additional capacity for build overhead. Version cached query embeddings by model too, so the query vector matches the searched index.

Chunk policy multiplies the vector count and the prompt bill

Small chunks can retrieve precisely while losing surrounding context. Larger chunks preserve context but may represent several topics less distinctly in one embedding. This resembles index granularity, with two cost consequences that can pull in opposite directions.

For sizing, suppose the 2 million documents average 2,048 tokens, with no chunk overlap or boundary-rounding overhead, and the embedding model produces 1,536 float32 dimensions. Each vector is 1,536 × 4 bytes, about 6 KB, before any index structure.

Chunk policy	Chunks per document	Vectors	Raw vector storage
512 tokens	~4	~8M	~49 GB
256 tokens	~8	~16M	~98 GB

Those figures cover vectors only. An HNSW graph adds neighbor lists per node; metadata and stored text add more. RAM can become a binding resource when the deployment needs the working set resident to meet latency. Disk-backed indexes also make I/O part of the capacity model.

Now the query side. If all five retrieved chunks enter the prompt, 512-token chunks contribute roughly 2,500 context tokens and 256-token chunks roughly 1,250. At fixed k, larger chunks increase input tokens, which can increase prefill latency and input-token charges. The actual penalty depends on the model service, caching, and context trimming. Smaller chunks may need a larger k to supply equivalent evidence, so compare answer quality alongside token counts.

Support articles with headings per step also offer structural chunk boundaries rather than fixed token counts. A global chunk-policy change can require a full rebuild. Run the experiment against a labeled query set before committing to a production build.

Metadata is the filter predicate and the isolation control

For this knowledge base, store tenant ID, document ACL or an authorization reference, document version, timestamp, and source identifier alongside each vector. These support the question: "What is similar that this user is allowed to see, from the current version, in their workspace?"

Returning a slightly less relevant chunk is a quality regression. Returning another tenant's private runbook is a data leak.

Isolation controls can combine:

• A metadata filter on tenant ID and ACL, evaluated per query, restricts eligible rows. Fine-grained authorization can also be enforced outside the vector engine.
• A namespace or partition per tenant scopes search within a collection. Bind that scope to the authenticated tenant and reject missing or invalid scope.
• A separate collection or database per tenant or sensitivity tier can strengthen separation when paired with independent access controls.

One possible layout is a collection per region, a namespace per tenant, and ACL filtering within the namespace. Each control changes a different boundary; neither a namespace nor a collection automatically makes an omitted filter safe. Start with server-enforced tenant scoping and document authorization, adding stronger separation when the security requirements warrant it.

Replicated ACL metadata can lag a revocation. If immediate revocation is required, validate against current authoritative permissions before exposing retrieved text. Version and timestamp metadata support stale-chunk exclusion and tracing an answer to its source version.

Filtered search: pre-filter, post-filter, and measured recall

Once metadata exists, the question is when the filter runs relative to approximate nearest neighbor search. Placement affects candidate coverage and cost. Privacy depends on enforcing authorization before text reaches an unauthorized user, model service, or log sink.

Post-filtering runs ANN search first, then removes candidates the predicate rejects. The candidate set was chosen without knowing the filter. If a tenant owns 1 percent of the corpus, the top 40 neighbors overall may contain none of its documents, returning fewer than k results even when enough eligible documents exist. Exploring more candidates or continuing the scan can help, at additional latency. Post-filtering inside a trusted retrieval boundary can be safe; filtering after unauthorized exposure is too late.

Pre-filtering restricts eligibility before or during search. If fewer than k documents qualify, no approach can return k distinct eligible results. With enough eligible data, a naive graph traversal that excludes ineligible nodes can lose useful paths. An exact scan of a small eligible set may be preferable. Filter-aware traversal is another approach, whose behavior depends on the index implementation.

Verify these semantics for the engine and index type you plan to use. For pgvector, the filtering documentation is the place to check scan behavior and available controls rather than assuming all index types behave alike.

Search-width settings such as ef_search tune candidate exploration; they do not guarantee k eligible results. Measure filtered ANN recall against exact nearest neighbors within the same eligible set, using the same embeddings and distance metric. Separately assess relevance and answer quality on labeled queries. For the support knowledge base, include small tenants and restrictive ACLs, where an unfiltered candidate set is particularly liable to miss eligible documents.

When the store choice actually matters

The store name becomes informative when it expresses a constraint established by the design. ANN support alone does not establish suitability; filtering behavior, deployment limits, and operating requirements can make product selection architectural.

The first condition is scale. Once chunk arithmetic puts the index at tens or hundreds of gigabytes, plus migration capacity, storage architecture and sharding become operational differences with a budget attached. Whether the working set needs to fit in RAM depends on the index, workload, and latency target.

For the pending model upgrade, steady-state query performance is only part of the capacity question. A build that fits alongside the serving index may still compete for CPU, memory bandwidth, or storage I/O. Test the expected query mix while the replacement index is being populated, including the small-tenant and restrictive-ACL searches. If the shared deployment misses its latency target, consider throttling the build or isolating build resources. Throttling trades lower interference for a longer migration window; isolation can increase operating cost.

The second is filtering semantics. If retrieval requires k results, verify that the engine can return them when enough eligible documents exist, while meeting recall and latency targets under tenant and ACL filters. Use representative tests rather than a feature checkbox.

The third is co-location. If the knowledge base already lives in Postgres, pgvector can keep vectors, metadata, and ACL tables in the same database. Source changes and derived writes share a transaction boundary only when the application commits them together. An asynchronous embedding pipeline still introduces lag, and co-location alone does not enforce permissions.

Extending an existing database can reduce separate replication machinery, but vector queries and index builds compete with transactional work for resources. A dedicated store can isolate that workload at the cost of another propagation and operations boundary. For a vendor perspective, see Qdrant’s article on dedicated vector search. Base the choice on measured filtered-search performance, workload interference, and operating cost, not a fixed rule that vectors must fit one instance's RAM.

Spend the interview time on chunk policy, metadata, filter semantics, and rebuild strategy. Name the store as the consequence of those decisions.

Formation publishes this blog. To practice this design, try Formation’s sub-challenge.