LLM inference splits into compute-heavy prefill and memory-bandwidth-sensitive decode. Learn why batching helps, how long prompts disrupt streaming, and which latency metrics to measure.
Formation Blog
An embedding-model upgrade creates a new vector space, not a routine deployment. Learn how to re-embed, dual-write, validate, cut over, and preserve rollback for RAG retrieval.
In a vector database system design interview, focus on chunk policy, metadata, filtered search, and rebuild strategy—not just the store you name.
Learn how to design a RAG system for interviews by separating retrieval from generation, with distinct freshness, latency, failure, and quality decisions.
When an AI assistant can write working code in seconds, interviewers stop testing typing speed and start testing whether you can clarify requirements, own the design, and verify what the model produced.
When interviewers ask you to design an AI-powered feature, they're testing system design judgment, not ML knowledge — this post shows how to treat the model as an unreliable dependency and reason about its latency, cost, and failure modes.
A partition key doesn't just distribute data; it decides which queries stay cheap and which get expensive. This piece compares sharding an ad platform by tenant ID, entity ID, and time, and explains why hot keys and jumbo tenants are failure risks, while resharding is an operational cost to plan for.
Failover isn't just "promote a follower" — done carelessly, it produces split-brain and zombie-leader failures worse than the outage it was meant to prevent. Here's what fencing tokens, quorum election, and STONITH actually protect against.
Read replicas add read capacity and failover copies, but every write still funnels through one leader—so replicas alone won't fix a write bottleneck or guarantee fresh reads.
