An embedding is derived state: the output of a particular embedding model version applied to a particular representation of source data. When that model changes, the safe default is to treat the result as a new vector space. Completing the upgrade therefore requires re-embedding every searchable item in that namespace and moving retrieval to a compatible index.

That makes an embedding model upgrade a data migration, not a routine application rollout. A design that only changes the model used for new documents leaves the system with a mixed-version index whose similarity scores have no established interpretation unless the model provider explicitly documents compatibility.

In a senior system-design interview, this distinction connects model selection to data lineage, deployment, recovery, and freshness.

Document staleness and model staleness are different

A stale vector index can describe two separate problems.

Document staleness occurs when source content changes but its indexed representation does not. A policy is revised, for example, while retrieval still returns chunks from the previous revision. The repair is to process the current source through the existing chunking and embedding pipeline, then replace or delete the affected index entries.

Model-version staleness occurs when the system adopts a different embedding model but the index still contains vectors produced by the previous model. The source document may be current.

The first problem can usually be repaired one document at a time within the existing embedding space. The second changes the compatibility contract for the entire retrieval namespace.

A sound RAG design tracks both states rather than compressing them into one indexed_at timestamp.

A matching vector dimension does not establish compatibility

An embedding model maps an input into coordinates whose geometry is learned during that model's training. Retrieval works because the document vectors and query vector are produced under the same compatible mapping, then compared with the index's configured distance function and normalization assumptions.

Changing the model can change that mapping. Different vector dimensions make the incompatibility immediately visible, but equal dimensions do not prove that two spaces are aligned.

A vector database may still calculate cosine similarity, dot product, or another numerical score between same-length vectors from different models. Without documented compatibility or a validated alignment layer, the system has no sound basis for ranking cross-model comparisons.

That rules out a rollout in which new documents receive version two embeddings while existing documents keep version one embeddings in the same search namespace. A new-model query can rank old-model chunks arbitrarily, producing irrelevant or missing context and potentially incorrect or incomplete answers.

Compatibility must be explicit rather than inferred from an unchanged API or vector size.

Build the target index as a separate version

For an online migration, the practical baseline is a parallel target index. Create a new collection, namespace, or equivalent isolation boundary for vectors produced by the target model.

Define the target contract before starting the backfill: embedding model and version, dimensionality, distance metric, text normalization and chunking versions, source revision semantics, index and filter schema, and build identifier.

At fixed per-unit cost and pipeline throughput, a full-corpus re-embed consumes provider or compute spend and wall-clock time proportional to corpus size, potentially hours to days at meaningful volume. That cost and elapsed time can reasonably justify deferring an upgrade.

Dual-write changes while the backfill runs

Once the target index exists, new ingestion events should update both versions using each destination's preprocessing and embedding contract.

Dual writing must cover updates and deletions, not just newly created documents. Consider a policy that is revised while an older revision is waiting in the backfill queue. If the backfill later writes that older revision without an ordering check, it can overwrite the newer vector in the target index.

Carry a stable document identifier and comparable source revision through the pipeline. Use idempotent, revision-ordered upserts where supported, or an equivalent reconciliation rule. Durable tombstones or an equivalent deletion record prevent backfill retries from restoring removed content.

Record each destination's write outcome independently so failures can be retried without overwriting newer revisions. A successful old-index write does not imply a successful new-index write.

Backfill from source records, not old vectors

The corpus backfill should begin with the source content and its authoritative revision. It must repeat the target preprocessing pipeline and call the target embedding model. Converting old vectors is not a substitute unless a supported and validated transformation exists.

Process the corpus in restartable units with durable checkpoints that record completed source work, not attempted embedding requests. Retries must remain safe after worker failures, rate limits, timeouts, or partial index writes.

Reconciliation should confirm more than a total vector count. Chunking changes can alter the number of vectors per document, and deleted or superseded revisions can make raw totals misleading. Validate each in-scope source revision's expected representation and removal of stale chunks. Track failures by document class so a large archive does not hide missing current policies.

Shadow reads must keep each query and index compatible

After enough of the target corpus is available, run shadow retrieval against both versions. The production path embeds the query with the old model and searches the old index. The shadow path embeds the same query with the target model and searches the target index. Do not reuse one query vector across both indexes.

Compare retrieval using application-specific signals: judged relevance, known-answer coverage, filter behavior, downstream answer quality, or specific failure cases. Do not directly compare similarity scores across spaces unless their interpretation has been validated.

Shadowing can reveal migration defects as well as model-quality differences. Missing target results may come from failed dual writes, an incomplete backfill, changed chunk boundaries, inconsistent filters, or stale source revisions. The evaluation should separate those operational defects from the question of whether the target model is appropriate.

Cut over the query model and index together

The serving configuration should select a compatible pair: query embedding contract plus index version. Cutover changes that pair together through a controlled configuration change.

A staged cutover can route selected tenants or traffic partitions to the target pair, provided each individual search remains within one compatible namespace. Do not rank a combined candidate set by raw similarity across spaces.

Before expanding the cutover, verify ingestion health, coverage for the routed corpus, query latency, filter correctness, and retrieval quality under representative traffic against the system's service objectives and evaluation set. An embedding-model release note is not evidence that the application has improved.

Retain the old index for rollback

Rollback requires more than changing the query model back. The old index must still contain current enough document state to serve traffic. Continue dual writing during the initial target-serving period, or define the source changes that would have to be replayed before rollback.

Keep the previous query model available as well. An old index without access to its compatible query embedder is not a usable rollback target. Retention should end only after the team has accepted the new retrieval behavior and the remaining rollback path is understood.

Parallel migration is not mandatory for every system. An offline rebuild may be appropriate when retrieval can be paused or regenerated before the next serving window. What is unsafe is an accidental in-place migration whose recovery properties were never decided.

Store the full embedding lineage

The embedding model version belongs in index metadata and, where useful for diagnosis, in each vector record. Retain the build contract, normalization assumptions, and source checkpoint; each vector should also identify its source revision and indexing timestamp.

This lineage supports diagnosis, rebuilds, and safe migration restarts.

Provider-hosted embeddings add an ownership boundary. Model availability, aliases, and retirement schedules may be controlled outside the application team. The design should therefore identify pinned dependencies where the provider supports them, monitor published lifecycle information, and retain enough source and lineage data to rebuild elsewhere if required. Do not assume that an endpoint name is a permanent compatibility guarantee.

Assign freshness requirements by document class

RAG data freshness is not one corpus-wide number. Policies may need reprocessing whenever an approved revision changes. Product manuals may only change with a release. Historical records may be immutable, while pricing or availability content may require a much shorter path from source update to searchable vector.

Assign freshness objectives to document classes, then connect each class to an ingestion trigger, retry policy, reconciliation process, and alert. This avoids repeatedly scanning stable content while high-change content waits for a global reindex job.

Keep document freshness separate from model migration status. A policy can be current in both indexes during a migration while an archived manual has not yet been backfilled to the target. Routing decisions should reflect whether the target index has the required coverage for the traffic being moved.

One reindexing strategy for the entire vector database is worth questioning for the same reason one cache TTL for every object is worth questioning: the underlying data has different change patterns and consequences when stale.

State the migration contract in the interview

In a senior system-design interview, the important move is to identify embeddings as versioned derived state before discussing the upgrade procedure. A concise answer is enough:

Unless the embedding versions have documented compatibility, I would build a parallel index, dual-write source changes, backfill from authoritative content, shadow retrieval with matching query models, and cut over the model-index pair together. I would retain the old pair until rollback is no longer required.

The useful follow-up is to name model and preprocessing lineage in metadata, then separate document freshness objectives by corpus class. That shows the design includes the state transitions and failure boundaries created by model changes, rather than treating the embedding API as a stateless dependency.

Formation publishes this blog. If you want to practice explaining this level of migration reasoning under interview constraints, ask about the Fellowship and AI-era system-design mock interviews.