Domain Name System (DNS)
Your service is unreachable. Everything is healthy except users can't resolve your domain. DNS is invisible until it breaks, and it's a common system design and reliability topic.
The Concept Explained
DNS maps names to addresses. In system design it is more interesting than that, because it is the first decision point in every request: before a client connects to anything, it asks DNS where to go, and whoever answers that question controls where the traffic lands.
That makes DNS a traffic control mechanism, and it is the only one that operates before a connection exists.
user types example.com
│
├─ DNS answers: 203.0.113.10 <-- you chose this, per user, per region
│
└─ client connects to 203.0.113.10 <-- every later decision happens afterDNS is the only load balancing layer that works before the client has connected to anything. A load balancer can distribute across servers it fronts; it cannot send a user in Frankfurt to a different continent's load balancer. DNS can, which is why every global architecture uses it for geographic routing, and why its caching behavior is the constraint you design around.
Resolution mechanics with dig, plus CoreDNS in Kubernetes, are covered hands-on in the free Networking Fundamentals course. This lesson is about using DNS as an architectural component and living with its failure modes.
How It Works
Resolution, briefly
client -> stub resolver -> recursive resolver (ISP, 8.8.8.8, corporate)
│ cache hit? answer immediately
├─> root servers "ask the .com servers"
├─> TLD servers "ask ns1.example.com"
└─> authoritative "203.0.113.10, TTL 300"The critical structural fact: the recursive resolver caches, and you do not control it. Your authoritative server returns a TTL as a request, and resolvers mostly honor it, sometimes clamp it, and occasionally ignore it. Some client libraries and JVMs cache indefinitely regardless of what you say.
Everything difficult about DNS in production follows from that: you are publishing a hint, not issuing a command.
The records that matter
| Record | Maps | Design note |
|---|---|---|
A | Name to IPv4 | The common case |
AAAA | Name to IPv6 | Needed for dual-stack (Lesson 1.2) |
CNAME | Name to another name | Cannot exist at a zone apex |
MX | Mail routing | Priority-ordered |
TXT | Arbitrary text | Domain verification, SPF, DKIM |
NS | Delegation | Which servers are authoritative |
SRV | Service, port, priority | Service discovery |
The CNAME apex restriction catches people regularly: you cannot put a CNAME on example.com itself, only on www.example.com. Providers work around this with ALIAS or ANAME records that resolve server-side and behave like an apex CNAME. If a design points a root domain at a load balancer's hostname, this is the mechanism doing it.
TTL: the central tradeoff
The TTL determines how long resolvers cache an answer, and it trades speed of change against query volume and resilience.
TTL 60s change propagates in ~1 minute high query load, fast failover
TTL 3600s change propagates in ~1 hour low query load, slow failover
TTL 86400s change propagates in ~1 day minimal load, effectively unchangeableThe design pattern worth stating in an interview: lower the TTL before a planned change, not during an incident.
T-48h drop TTL from 3600 to 60 old cached entries expire over the next hour
T-0 make the change propagates in ~60 seconds
T+24h raise TTL back to 3600 query volume returns to normalLowering the TTL at the moment you need to move traffic does not help, because resolvers are still holding the old record with the old long TTL. They will not see your new short TTL until the old one expires. This is a genuinely counterintuitive property and a good thing to say out loud.
DNS-based failover is not fast failover, and treating it as such is a common design error. Between resolver caches that ignore your TTL, client libraries that cache for the process lifetime, and browsers with their own caches, a portion of traffic will keep hitting a dead address well past the TTL. For fast failover use a load balancer with health checks, or anycast, and treat DNS as the coarse layer that eventually converges.
DNS as a routing mechanism
Because you control the answer, you can answer differently per client:
GeoDNS. Answer based on the resolver's approximate location. A user in Europe gets the Frankfurt address, one in Asia gets Singapore. This is how global systems cut latency, and its accuracy is limited by the fact that you see the resolver's location, not the user's. A user on a public resolver may be geolocated to the wrong continent, which is what EDNS Client Subnet partially addresses by forwarding a truncated client subnet to the authoritative server.
Latency-based routing. Route to whichever region measures fastest for that resolver, rather than whichever is geographically nearest. Usually better, since network topology and geography disagree more often than people expect.
Weighted routing. Return different answers in different proportions, which supports canary releases and gradual migrations at DNS granularity.
Failover routing. Health-check endpoints and stop returning addresses that fail, subject to the caching caveat above.
Round-robin. Return multiple addresses and let clients pick. Crude distribution with no health awareness, and clients often just take the first one.
The alternative worth comparing against is anycast: the same IP address announced from many locations, with internet routing delivering packets to the nearest one. No caching problem, near-instant failover, and it requires operating network infrastructure at that level, which is why CDNs use it and most application teams use DNS instead.
Service discovery
Inside a system, DNS is how services find each other. In Kubernetes, orders-service.default.svc.cluster.local resolves to a stable virtual IP, so callers never hold pod addresses that change constantly.
The design implication is important and often missed: internal DNS becomes a hard dependency on the request path. If cluster DNS is slow or degraded, every service-to-service call slows down with it, and the symptom presents as a broad, confusing slowdown rather than a DNS error. Client-side caching and connection reuse (Lesson 1.4) reduce lookup frequency substantially, which is a reliability argument for pooling, not only a latency one.
The failure modes
DNS causes outages in a small number of recognizable ways, and knowing them is worth points:
- Expired domain registration. Total outage, entirely self-inflicted, and it happens to large companies.
- Authoritative nameserver failure. If all your nameservers are with one provider and that provider has an incident, you disappear from the internet regardless of how healthy your servers are. Using two independent DNS providers is the mitigation, and it is cheap.
- Misconfigured records. Deleting or mistyping a record propagates as fast as your TTL allows, which is a good argument for keeping DNS in version control and reviewed like code.
- DNS as an outage amplifier. Because everything depends on resolution, a DNS problem presents as "everything is broken," which lengthens diagnosis. Several of the largest internet outages on record were DNS or routing problems rather than application failures.
- Negative caching. Failures are cached too. An
NXDOMAINreturned during a bad deploy is cached per the zone's negative TTL, so the fix takes effect later than you expect.
System Design Implications
- It is the first layer of global traffic management. Any multi-region design should mention how users reach the right region, and the answer is GeoDNS or latency-based routing, or anycast.
- It is a single point of failure worth designing against. Multiple providers, monitored expiry, and version-controlled zones are cheap insurance against a total outage.
- TTL is a lever you set in advance. Pre-lowering before migrations is the practice that makes DNS-based changes usable.
- It is not a health-aware load balancer. Use it for coarse geographic and failover routing, and a real load balancer for per-request distribution and health checking.
- Internal DNS is on the critical path. Resolution latency is added to every uncached call; caching and connection reuse mitigate it.
Tradeoffs and Decision Framework
Low versus high TTL. Low gives agility and more queries and more resolver dependency. High gives resilience and stability at the cost of being unable to move quickly.
DNS routing versus anycast. DNS is accessible to everyone and converges slowly. Anycast fails over in seconds and requires network-level capability.
GeoDNS versus latency-based. Geography is intuitive and sometimes wrong. Latency measurement reflects the actual network and is more complex to operate.
Single versus multiple DNS providers. One is simpler and correlates your availability with theirs. Two is a modest amount of configuration work for meaningful protection against a total outage.
Common Mistakes
Treating DNS as fast failover. Caches you do not control mean stragglers persist well past the TTL.
Lowering the TTL during an incident. Too late: resolvers still hold the old record under the old TTL.
All nameservers at one provider. Their outage is your total outage.
Assuming clients honor TTLs. Some libraries and runtimes cache for the process lifetime.
Forgetting negative caching. Failed lookups are cached too, so fixes appear to take longer than they should.
Unmonitored domain and certificate expiry. Both cause abrupt, total outages with known dates.
A CNAME at the zone apex. Not permitted; use ALIAS or ANAME.
Ignoring internal DNS as a dependency. Cluster DNS degradation looks like everything being slow at once.
How would you use DNS as part of a global load balancing strategy? What are the limitations?