AISeptember 15, 2026by Formation

KV Cache Capacity: Why Long Context Lowers Concurrency

Long-context capacity is bounded by the KV cache: every resident token consumes finite GPU memory. Learn how to estimate the concurrency ceiling and monitor the latency signals that reveal cache pressure.

AISeptember 15, 2026by Formation

TTFT, TPOT, and SLOs for Streamed LLM Responses

Streamed LLM responses need separate targets for time to first token and time per output token. Learn how workload shape, batching, deadlines, and cost determine the right SLOs.

AISeptember 14, 2026by Formation

Prefill vs. Decode: Why LLM Inference Slows Under Load

LLM inference splits into compute-heavy prefill and memory-bandwidth-sensitive decode. Learn why batching helps, how long prompts disrupt streaming, and which latency metrics to measure.

Engineering ResourcesSeptember 10, 2026by Formation

Embedding Model Upgrades Are Data Migrations, Not Rollouts

An embedding-model upgrade creates a new vector space, not a routine deployment. Learn how to re-embed, dual-write, validate, cut over, and preserve rollback for RAG retrieval.

Interview PreparationSeptember 10, 2026by Formation

Vector Database System Design: What Matters More

In a vector database system design interview, focus on chunk policy, metadata, filtered search, and rebuild strategy—not just the store you name.

AISeptember 9, 2026by Formation

RAG System Design Interviews: Retrieval and Generation

Learn how to design a RAG system for interviews by separating retrieval from generation, with distinct freshness, latency, failure, and quality decisions.

AIAugust 27, 2026by Formation

The Three-Step Loop for Reviewing AI-Generated Code

Interview PreparationAugust 26, 2026by Formation

What AI-Assisted Coding Interviews Actually Score

When an AI assistant can write working code in seconds, interviewers stop testing typing speed and start testing whether you can clarify requirements, own the design, and verify what the model produced.

Interview PreparationAugust 25, 2026by Formation

Design an AI Feature? Treat the LLM as a Dependency

When interviewers ask you to design an AI-powered feature, they're testing system design judgment, not ML knowledge — this post shows how to treat the model as an unreliable dependency and reason about its latency, cost, and failure modes.

Engineering ResourcesAugust 18, 2026by Formation

"Shard by ID" Is the Wrong Default

A partition key doesn't just distribute data; it decides which queries stay cheap and which get expensive. This piece compares sharding an ad platform by tenant ID, entity ID, and time, and explains why hot keys and jumbo tenants are failure risks, while resharding is an operational cost to plan for.