Advanced|21.5 hours|37 lessons

Production RAG Infrastructure on Kubernetes

Operating the retrieval tier behind a RAG system on Kubernetes: index internals and the memory wall, sharding and what it does to recall, embedding pipelines and corpus backfills, re-embedding migrations without downtime, and measuring retrieval quality when nobody labelled the corpus. Built for corpora measured in millions of documents. Lifetime updates included.

Text-based, no videos

9 modules, 37 lessons

Lifetime access

What you'll learn

The arithmetic that turns a corpus size into a memory budget and a node count

Index internals: the recall, memory and latency triangle, and the build parameters that move it

Why filtered search collapses recall, and what to do instead

When one node stops working, and what sharding does to your results

Embedding as a GPU serving workload, and backfills that survive failure at hour forty

Deletes, tombstones and index decay, and when to compact rather than rebuild

Executing a re-embedding migration without downtime, and validating it before you trust it

Retrieval inside the request latency budget, and degrading when the vector store is gone

Measuring recall in production when nobody labelled the corpus

A debugging method for retrieval failures, and a cost model whose dominant term is memory

Curriculum

9 modules · 37 lessons

The Retrieval Path in Production

What runs between a user question and the tokens a model generates, the arithmetic that decides the architecture, and the failure that returns 200 and a wrong answer.

4 lessons

The Retrieval Request Path30 minFREE The Arithmetic of a Corpus40 minFREE How Retrieval Fails30 minFREE Metrics for a Retrieval Tier30 minFREE

The Index as a Data Structure You Operate

Index families and the tradeoff triangle, build parameters and their real cost, filtered search, and the quantization decision that stops being optional.

5 lessons

Index Types and the Tradeoff Triangle40 min Build Parameters and What They Cost35 min Index Build Jobs at Scale30 min Filtered Search and the Recall Collapse35 min Vector Quantization35 min

Running the Vector Store on Kubernetes

The sizing decision, sharding and what it does to recall, restore against rebuild, tenant isolation, and upgrading a stateful tier whose index takes hours to build.

6 lessons

When One Node Is Not Enough35 min Sharding and Replication35 min What Sharding Does to Recall35 min Backup, Restore, and the Rebuild Clock40 min Multi-Tenant Index Isolation30 min Upgrades Without Losing the Index30 min

The Embedding Pipeline as Infrastructure

Embedding as a GPU serving workload, backfills that survive failure, freshness lag, and what chunk size costs you downstream.

4 lessons

Embedding Is Inference35 min Corpus Backfill40 min Incremental Ingestion and Freshness Lag35 min What Chunking Costs You Downstream25 min

Index Lifecycle: Mutation and Re-Embedding

What deletes do to a graph index, when to compact against rebuild, and executing a re-embedding migration without downtime.

6 lessons

Deletes, Tombstones, and Index Decay35 min Compaction and the Rebuild Decision35 min When Vectors Expire30 min Dual-Write, Shadow Index, Blue-Green40 min Executing the Cutover40 min Validating the New Index35 min

Retrieval in the Request Path

The latency budget shared with generation, caching, degrading when retrieval is unavailable, and the reranker as a second serving tier.

4 lessons

The Latency Budget30 min Caching Retrieval Results30 min Degrading Without Retrieval25 min Reranking as a Second Serving Tier35 min

Retrieval Quality in Production

Measuring recall without labels, catching silent decay, building evaluation sets from real traffic, and attributing a bad answer to the right layer.

4 lessons

Measuring Recall Without Ground Truth40 min Drift: Silent Quality Decay35 min Evaluation Sets from Production Traffic35 min Tracing a Bad Answer Back to a Bad Retrieval35 min

Debugging and Capacity

A method for retrieval failures, tail latency in a vector search, and a cost model whose dominant term is memory.

3 lessons

A Systematic Method for Retrieval Failures40 min Debugging Latency in the Retrieval Tier35 min Capacity and Cost Modelling for Retrieval35 min

Capstone

A full retrieval tier designed end to end, with the tradeoffs stated and a re-embedding you already know is coming.

1 lessons

Capstone: Design the Retrieval Tier for a Corpus of Millions60 min

About the Author

Sharon Sahadevan

AI Infrastructure Engineer

Building production GPU clusters on Kubernetes. H100s, large-scale model serving, and end-to-end ML infrastructure across Azure and AWS.

10+ years designing cloud-native platforms with deep expertise in Kubernetes orchestration, GitOps (Argo CD), Terraform, and MLOps pipelines for LLM deployment.

Author of KubeNatives, a weekly newsletter read by 3,000+ DevOps and ML engineers for production insights on K8s internals, GPU scheduling, and model-serving patterns.

Ready to master this topic?

Start with the free preview lesson and see for yourself.