Advanced|18 hours|35 lessons

LLM Observability in Production

Operating the observability stack for an LLM platform: why a span carrying a prompt and a completion is orders of magnitude heavier than an ordinary one, the volume arithmetic that decides whether you can self host, running Langfuse on Kubernetes across its four datastores, ingestion and sampling when the payload is the point, retention and cardinality, prompt data as customer data, evaluation pipelines as infrastructure, and debugging a slow request across six layers that each have their own telemetry. Lifetime updates included.

Text-based, no videos

9 modules, 35 lessons

Lifetime access

What you'll learn

Why an LLM span is orders of magnitude heavier than an ordinary one, and what that does to your architecture

The volume arithmetic that decides whether self hosting a trace store is viable at all

Running Langfuse on Kubernetes across Postgres, ClickHouse, Redis and blob storage

Ingestion, batching, backpressure and the traces that drop silently

Sampling and retention when the payload is the thing you came for

Prompt data as customer data: redaction, access control and provable deletion

Operating an evaluation pipeline, including the judge as a second inference workload

Attributing a slow request across gateway, queue, engine, GPU and retrieval

Curriculum

9 modules · 35 lessons

What Makes LLM Observability Different

Why the telemetry for an LLM system is a different size and shape from anything else you instrument.

4 lessons

The Payload Is the Telemetry26 minFREE The Volume Arithmetic30 minFREE What Metrics Alone Cannot Tell You26 minFREE Which Signal Answers Which Question28 minFREE

The Trace Shape

Designing the trace before you emit it, because the shape decides which questions you can ask later.

4 lessons

What a Span Should Be in an LLM Call28 min Nesting: Chains, Tools and Retrieval32 min Sessions, Users and Threading a Conversation28 min Semantic Conventions and a Moving Target30 min

Running Langfuse on Kubernetes

The trace store as a stateful system: four datastores, two services, and the capacity that decides everything.

5 lessons

Four Datastores and Two Services30 min ClickHouse Decides Your Capacity35 min Postgres, Redis and Blob in This Stack30 min Sizing From the Volume Arithmetic32 min Upgrades, Backups and What Loss Means Here30 min

Ingestion at Volume

The path from an SDK call to a stored trace, and every place it silently drops.

4 lessons

From SDK to Stored Trace28 min Batching, Backpressure and Dropped Traces33 min Sampling When the Payload Is the Point32 min The Worker That Falls Behind30 min

Cost, Retention and Cardinality

What a trace costs to keep, how long to keep it, and the labels that multiply the bill.

4 lessons

What a Trace Actually Costs28 min Retention Tiers for Prompt Data30 min Cardinality in a Trace Store30 min Truncation and What You Give Up27 min

Prompts, Privacy and Compliance

A trace store full of prompts is a store full of customer data, with all that implies.

4 lessons

Prompt Data Is Customer Data28 min Redaction at the Right Layer32 min Access Control on a Trace Store28 min Deletion Requests Against a Trace Store30 min

Evaluation as Infrastructure

Operating an evaluation pipeline at volume, rather than designing the evaluations themselves.

4 lessons

Datasets From Production Traces30 min Running the Judge at Volume32 min Online Evaluation Without Doubling Cost30 min Regression Detection and Quality Drift28 min

Debugging With Traces

Using the thing you built, under pressure, across layers that each have their own telemetry.

5 lessons

One Slow Request, Six Candidate Layers32 min Correlating Traces With Engine and GPU Telemetry33 min Tracing a Bad Answer to Its Cause30 min The Cost Spike and the Trace That Explains It28 min When the Observability Stack Is the Outage30 min

Capstone

A complete LLM observability platform design combining every module, with explicit tradeoffs.

1 lessons

Capstone: Design the LLM Observability Platform60 min

About the Author

Sharon Sahadevan

AI Infrastructure Engineer

Building production GPU clusters on Kubernetes. H100s, large-scale model serving, and end-to-end ML infrastructure across Azure and AWS.

10+ years designing cloud-native platforms with deep expertise in Kubernetes orchestration, GitOps (Argo CD), Terraform, and MLOps pipelines for LLM deployment.

Author of KubeNatives, a weekly newsletter read by 3,000+ DevOps and ML engineers for production insights on K8s internals, GPU scheduling, and model-serving patterns.

Ready to master this topic?

Start with the free preview lesson and see for yourself.