Distributed Systems Design

Reliability

A system can be available but unreliable, returning wrong answers, losing data, corrupting state. Reliability is a distinct property and interviewers probe whether you understand the difference.


The Concept Explained

Availability asks whether the system responded. Reliability asks whether the response was correct, and whether the system did what it promised with the data you gave it.

They are separate properties and they fail separately. A service that returns HTTP 200 with a stale, empty, or wrong body is perfectly available and completely unreliable. Every dashboard is green. Every health check passes. Users are being lied to at full speed, which is worse than an outage, because an outage is at least honest about its state.

The reason this distinction is an interview favourite is that it exposes whether a candidate thinks about correctness at all, or only about uptime. Uptime is easy to measure, so it is what most monitoring measures, and so it becomes what many engineers optimize by default. The failures that damage a business most are usually not downtime. They are silent corruption, lost writes, and duplicated side effects, all of which happen while availability metrics look excellent.

KEY CONCEPT

Availability is "did it answer?" Reliability is "was the answer right, and is my data still intact?" A system can score perfectly on the first while failing catastrophically on the second, and the second failure is the one that is hard to detect and hard to undo.

Reliability also has a time dimension that availability lacks. Availability is a ratio you can compute for any window. Reliability is about the probability that the system performs correctly over a period, which is why it degrades with load, with time, and with the accumulation of small unhandled edge cases.


How It Works

Reliability is built from a small set of properties, and it is worth being able to name them separately because they fail separately.

Available but Unreliable vs Reliable but Unavailable

Available, unreliable

Answers every request, answers wrongly

Health checksAll green
HTTP status200, consistently
Response bodyStale, partial, or wrong
Data effectSilent loss or corruption
DetectionHard, often via customer report
ExampleCache serving evicted data as fresh
Blast radiusGrows silently over time
Unavailable, reliable

Refuses to answer rather than answer wrongly

Health checksFailing loudly
HTTP status503, explicitly
Response bodyNone, by design
Data effectNothing written, nothing lost
DetectionImmediate and obvious
ExampleDatabase refusing writes without quorum
Blast radiusBounded, visible, recoverable

The right-hand column is a deliberate design choice, and a good one for anything holding money or state. Refusing service is recoverable. Corrupting data is often not.

The Components of Reliability

Fault tolerance is the ability to keep producing correct results while some component is broken. Note the word correct. A system that stays up during a failure but starts serving stale reads has traded reliability for availability, which may be right or wrong depending on what the data is.

Durability is the promise that acknowledged writes survive. This is the property users assume without being told and the one whose violation is least forgivable. If you return success on a write, that write must survive the failure of the node that accepted it, which means it must have been replicated or committed to stable storage before you answered.

Correctness under concurrency is where most subtle unreliability lives. Two requests arriving at the same moment, a retry arriving after a timeout, a message delivered twice. Each is individually handled and collectively catastrophic if the operations are not idempotent.

Graceful degradation is the ability to shed functionality rather than accuracy. A recommendation panel that disappears when its service is down is graceful. A recommendation panel that shows another user's recommendations is not degradation, it is a correctness failure wearing degradation's clothes.

Why Retries Are a Reliability Problem

The classic trap: a request times out, so the client retries. The original request had already succeeded, the response was simply lost on the way back. The retry now applies the operation a second time. Availability tooling created a reliability failure.

This is why idempotency is not an optimization but a precondition for safe retries, and why any interview answer that reaches for retries should mention it in the same breath. Without idempotency, every retry mechanism you add is a duplicate-execution mechanism.

WARNING

Redundancy improves availability but does nothing for reliability on its own, and can actively hurt it. Adding replicas means more copies that can disagree. Without a defined way to resolve that disagreement, you have simply increased the number of places a wrong answer can come from.


System Design Implications

The design question is always the same: when this component fails, would you rather be wrong or be down? Answer it per data type, not per system, because the answer genuinely differs.

For a bank balance, an inventory count, or a permission check, being wrong is unacceptable and being down is merely bad. These paths should fail closed: refuse the operation, return an error, make the failure loud. For a recommendation feed, a view counter, or a "people also bought" panel, being briefly wrong is fine and being down is a worse user experience. These should fail open: serve stale, serve empty, serve a default.

Getting this backwards is a serious design error, and interviewers construct scenarios specifically to catch it. Serving a stale permission check because the auth service is slow is exactly the failure mode that turns a minor outage into a security incident.

The mechanisms that buy reliability are worth naming explicitly:

Write acknowledgement discipline. Do not return success until the write is durable to the standard you have promised. If you acknowledge after writing to one node's memory, you have promised durability you cannot deliver.

Idempotency keys. Give every mutating operation a client-supplied identifier so a duplicate arrival is recognized and discarded rather than applied twice. This is what makes retries safe, and retries are what make availability recoverable.

End-to-end checksums. Verify data at the boundary it is read, not only where it was written. Silent corruption between the two is exactly the failure mode that availability monitoring cannot see.

Failing closed on the paths that matter. When the system cannot establish the truth, returning an error is a correct answer. Guessing is not.

Reconciliation. Accept that divergence will happen and build the process that detects and repairs it. Any system holding money has one of these, usually running nightly, and its existence is an admission that reliability is maintained rather than achieved once.


Tradeoffs and Decision Framework

ChoiceFavoursCost
Fail closed on errorReliabilityLower availability, visible errors
Fail open, serve staleAvailabilityUsers act on wrong data
Synchronous replication before ackDurabilityHigher write latency
Asynchronous replicationWrite latencyAcknowledged writes can be lost on failover
Aggressive client retriesRecovery from transient faultsDuplicate execution without idempotency
Strong validation at boundariesCatches corruption earlyThroughput cost, more rejected requests

The framework has three steps. Classify each data path by what a wrong answer costs. Choose fail-open or fail-closed per path from that cost, not as a system-wide default. Then make sure your monitoring can actually see the reliability failures you have chosen to risk, because the defining property of unreliability is that uptime dashboards do not show it.

PRO TIP

A strong interview move is to name the correctness metric alongside the availability metric. "99.9% availability, and separately, zero acknowledged writes lost" tells the interviewer you understand these are different promises requiring different mechanisms. Most candidates only offer the first.


Common Mistakes

Using the terms interchangeably. They are different properties with different mechanisms, and conflating them is the exact thing the question is designed to detect.

Adding retries without idempotency. This converts a transient failure into duplicate execution, which is a reliability failure created by an availability mechanism.

Acknowledging writes before they are durable. A success response the system cannot honour after a node loss is the most damaging lie a system can tell.

Failing open on paths that hold truth. Serving stale permissions, stale balances, or stale inventory to preserve uptime trades a small outage for a large incident.

Assuming replicas improve correctness. More copies means more opportunities to disagree unless the reconciliation rule is defined.

Monitoring only uptime. If nothing measures correctness, silent corruption runs for as long as it takes a customer to complain.

Calling any degradation graceful. Removing a feature is graceful. Showing wrong data in place of right data is a correctness failure with better marketing.


INTERVIEW QUESTION

Explain the difference between availability and reliability. Give an example of a system that is highly available but unreliable.