LLM Observability in Production

What Metrics Alone Cannot Tell You

The dashboard is excellent. Latency percentiles, tokens per second, error rate, cost per request, all flat for three weeks. Support has fourteen tickets saying the assistant has got worse, and the dashboard has no opinion about that at all.


The Problem at Scale

Metrics are the right first instrument for almost every system you have operated. For an LLM system they are necessary and structurally unable to answer the questions you will actually be asked, for three reasons that compound.

  the failure is not an error         a wrong answer is a 200
  the population is not homogeneous   requests differ 100x in cost
  individual events matter            aggregates exist to hide those

None of these is a gap in your dashboard. They are properties of the workload, and no additional metric fixes them.

KEY CONCEPT

In ordinary systems failure correlates with status codes, so error rate is a good proxy for whether users are suffering. In an LLM system the most important failure mode is a successful response containing the wrong content, which is indistinguishable from success at every layer that metrics observe. Error rate cannot see quality, and quality is what people complain about, which is why a flat dashboard and unhappy users is the normal condition rather than a contradiction.


How It Works

The failure is not an error

A hallucinated citation, an ignored instruction, a refusal to answer something reasonable, a response in the wrong language, a summary that inverts the meaning of the source. Every one of those is:

  HTTP 200
  normal latency
  normal token counts
  zero errors
  zero exceptions

There is nothing in the shape of the response that distinguishes a good answer from a confidently wrong one. That is not true of a database query or an API call, where wrongness usually announces itself somewhere.

So the entire class of failure that matters most to your users is invisible to the instrument you would normally reach for first.

The population is not homogeneous

A p95 latency over all requests is only meaningful if requests are comparable. In an LLM system they differ enormously:

  a 40 token intent classification       ~200 ms
  a 2,000 token question with retrieval   ~3 s
  a 12,000 token document summarisation   ~40 s

A single p95 across that mixture describes none of them. Worse, it moves for reasons that have nothing to do with performance: shift the traffic mix slightly toward summarisation and your p95 rises with no system change at all, which produces investigations into a regression that never happened.

You have to segment before you aggregate, and the segmentation keys are model, prompt version, task type and tenant. Which is the cardinality problem in Module 5, arriving immediately: the segmentation that makes metrics meaningful is the same thing that makes them expensive.

Individual events matter

An aggregate is a device for making individual events invisible, which is usually what you want. One slow request out of a million is noise.

Here it frequently is not. One catastrophically wrong answer to an important customer is an incident, and it is one event. A metric system is the wrong instrument for a question about one event, and no percentile will ever contain it.

That is the structural argument for traces being mandatory rather than a nice addition: you will be asked about a specific response, by name, and only a trace can answer.

The trap inside quality metrics

The response to all this is usually to add evaluation and put the score on the dashboard. Worth doing, and it introduces a specific statistical trap.

Quality distributions are frequently bimodal. A system does not degrade by every answer getting slightly worse. It degrades by a subset of answers collapsing while the rest stay fine.

  90% of answers score 0.95
  10% of answers score 0.10
  mean                  0.865   <- looks healthy

  degrade to:
  80% of answers score 0.95
  20% of answers score 0.10
  mean                  0.78    <- a small move on the dashboard

The failing population doubled and the headline moved by less than a tenth. A mean is the wrong statistic for a bimodal distribution, and it is the statistic everybody picks.

Track the share below a threshold rather than the mean. That number doubled in the example above, which is what happened.

What metrics are genuinely good for

Being fair to them, because the conclusion is not to skip them:

  capacity and cost            tokens/sec, spend, utilisation
  availability and latency     against an SLO, per segment
  detecting that something changed, without saying what
  the trigger for looking at traces

That last line is the useful framing and it is the next lesson. Metrics are an excellent detector and a poor explainer, and in this domain the gap between those two roles is wider than anywhere else you have worked.


Building and Operating It

Segment before aggregating, or the number describes nobody.

# A single p95 across all requests is a fiction. These are the four
# dimensions that make it mean something, and Module 5 is about
# affording them.
histogram_quantile(0.95,
  sum by (le, model, task_type) (rate(llm_request_duration_seconds_bucket[5m])))

Watch the distribution rather than the mean, for anything about quality.

# Share of scored responses below the acceptable threshold. This is
# the number that doubles when a subset collapses, which a mean
# barely registers.
sum(rate(llm_eval_score_bucket{le="0.5"}[1h]))
  / sum(rate(llm_eval_score_count[1h]))

Detect the traffic mix shift that moves your latency without a regression.

# If this moves, your latency percentiles move too, and nothing is
# wrong. Put it next to the latency panel so the two are read together.
sum by (task_type) (rate(llm_requests_total[1h]))
  / ignoring(task_type) group_left sum(rate(llm_requests_total[1h]))

And make the metric lead to a trace, which is the whole point.

# Exemplars attach trace identifiers to metric samples, so a spike on
# a dashboard becomes a click through to a specific request rather
# than the start of a search. Module 8 uses this constantly.
histogram_quantile(0.99,
  sum by (le) (rate(llm_request_duration_seconds_bucket[5m])))
# with exemplar storage enabled on the histogram
WAR STORY

A team investigated a rising p95 for most of a week. Nothing had been deployed, the models were unchanged, GPU utilisation was flat, and the inference engine metrics were healthy. The cause was a new customer whose workload was document summarisation rather than short questions, so the traffic mix had shifted toward requests that were legitimately ten times longer. Latency per task type had not moved at all. The dashboard had one latency panel for all traffic and no panel showing the mix, so a change in who was using the product presented as a performance regression, and the only way anybody found out was by opening individual traces and noticing they were all from the same tenant.


Tradeoffs and Decision Framework

QuestionMetricsTracesEvaluations
Is it up?YesNoNo
Is it slow?Yes, if segmentedYes, and whereNo
What does it cost?YesPer requestNo
Why was this answer wrong?NoYesPartly
Is quality falling?Only as a distributionNoYes
What happened to this one request?NoYesNo

Three questions when the dashboard is flat and users are unhappy. Is the failure mode an error at all, since a wrong answer is a 200 and error rate cannot see it. Is the aggregate mixing incomparable populations, because an unsegmented percentile describes nobody. And has the traffic mix moved, as that changes latency without anything being wrong.

Default: segment every aggregate by model and task type at minimum, track quality as the share below a threshold rather than as a mean, put the traffic mix panel next to the latency panel, and attach exemplars so a metric leads to a trace.


Failure Modes and Common Mistakes

Using error rate as a proxy for user experience. The failure that matters returns 200.

One latency percentile across all traffic. It describes none of the populations and moves when the mix moves.

Averaging quality scores. Distributions are bimodal, so a subset collapsing barely moves a mean.

Adding more metrics in response to an unanswerable question. The question was about one event, and no aggregate contains one event.

Investigating a latency regression before checking the traffic mix. A new customer with longer requests looks identical to a regression.

Metrics with no path to a trace. The detection works and the explanation still starts from nothing.

KNOWLEDGE CHECK

Latency p95 has risen steadily for a week. Nothing was deployed, models are unchanged, GPU utilisation is flat and inference engine metrics are healthy. What should you check before investigating performance?

INTERVIEW QUESTION

Your dashboards are flat and users say the product has got worse. How do you approach that?