Networking and API Design for System Design

IP Address

Your service needs to talk to another service across regions. Understanding IP addressing, subnets, and routing is the difference between designing this correctly and hand-waving through it.


The Concept Explained

An IP address identifies a network interface so packets can be routed to it. That much is obvious. What matters in system design is the consequences of addressing choices, and there are three that come up repeatedly:

  • Reachability is a design decision. Whether service A can reach service B is determined by addressing and routing, not by application code. Most "the service cannot connect" incidents are addressing problems.
  • Address space is finite and must be planned. Overlapping ranges chosen carelessly become impossible to merge later, and this is the single most common source of permanent networking pain in growing companies.
  • Addresses are identity to your infrastructure. Firewall rules, allow-lists, and audit logs key on them, and that identity is weak and shifting in any dynamic environment.
KEY CONCEPT

The interview-relevant framing: public versus private addressing is a security and topology boundary, not a technical detail. Private addresses are unreachable from the internet by design, so putting a database in a private subnet is not defense in depth, it is the primary control. Everything else (security groups, auth) layers on top of a resource that could not be dialed in the first place.


How It Works

CIDR, and reading a range correctly

An address plus a prefix length describes a block:

10.0.0.0/16     first 16 bits fixed, 16 bits free    65,536 addresses
10.0.1.0/24     first 24 bits fixed,  8 bits free       256 addresses
10.0.1.0/28     first 28 bits fixed,  4 bits free        16 addresses

The rule to internalize: a smaller prefix number means a larger block. A /16 contains 256 /24s. Interviewers occasionally check this directly because it is a fast signal.

Two practical adjustments to the arithmetic. Every subnet loses a few addresses to the network address, broadcast, and (in cloud providers) reserved gateway and DNS addresses, typically five in AWS. And usable capacity is what matters: a /28 gives you eleven usable addresses in AWS, not sixteen.

Private ranges and why the boundary matters

Three ranges are reserved for private use and are not routable on the public internet:

10.0.0.0/8         16.7M addresses    large cloud VPCs
172.16.0.0/12       1.0M addresses    the range Docker often uses by default
192.168.0.0/16     65.5K addresses    home and small office networks

A packet with a private destination address will not cross the public internet, and that unreachability is the property you are designing around. The standard cloud topology follows directly:

VPC 10.0.0.0/16
  public subnets   10.0.0.0/24, 10.0.1.0/24     load balancers, NAT gateways
  private subnets  10.0.10.0/24, 10.0.11.0/24   application services
  data subnets     10.0.20.0/24, 10.0.21.0/24   databases, caches

Traffic enters at the load balancer in a public subnet, reaches services in private subnets, which reach data in subnets that route nowhere outbound at all. Compromising an application server still does not put the attacker on a network segment that can be reached directly from outside.

WARNING

Plan the address space before you need it. Two teams that independently pick 10.0.0.0/16 for their VPCs cannot ever peer those VPCs, because the ranges overlap and routing becomes ambiguous. The usual fixes are re-addressing an entire environment or running NAT between them, and both are expensive and permanent sources of confusion. Allocate non-overlapping ranges per environment and region from the start; it costs nothing to do early and cannot be retrofitted cheaply.

NAT, and what it implies

Network Address Translation rewrites addresses in transit, most often letting many private hosts share one public address for outbound traffic:

10.0.10.5:51234  ──> NAT gateway ──> 203.0.113.10:62145  ──> internet
                                     (the internet sees only this)

Three design consequences follow, and they show up constantly:

Outbound works, inbound does not. A NATed host can initiate connections and cannot receive unsolicited ones, because there is no mapping until it sends something. This is why servers behind NAT need explicit port forwarding or a load balancer, and it is the same property that makes WebRTC need STUN and TURN (Lesson 5.5).

All your traffic appears to come from one address. Partners allow-listing your IP see a single stable source, which is convenient. It also means IP-based rate limiting on the receiving side treats your entire fleet as one client (Lesson 3.2), and the reverse: rate limiting by IP punishes every user behind a corporate NAT equally.

NAT gateways are a cost and a bottleneck. In cloud environments they are billed per gigabyte processed, and a chatty service pulling large volumes from an external API through NAT is a line item people are surprised by. Traffic to cloud services within the same region can usually bypass it entirely through private endpoints, which is both faster and cheaper.

IPv4 exhaustion and IPv6

IPv4 has about 4.3 billion addresses and ran out years ago, which is why NAT is everywhere. IPv6's 128-bit space is large enough that every device can have a globally unique address, which removes the need for NAT entirely.

Adoption is mixed, and the interview-relevant points are:

  • Dual-stack (running both) is the common transition, and it means two sets of firewall rules, two DNS record types (A and AAAA), and two paths to test. A rule applied only to IPv4 leaves an open door on IPv6.
  • IPv6 removes NAT, which removes accidental protection. Every host being globally addressable means firewall policy becomes the only control, where NAT was previously doing part of the job by accident.
  • Address exhaustion is a real constraint inside clusters. Kubernetes assigns an address per pod, so a large cluster in a /24 per node runs out quickly. Cluster sizing is partly an address planning exercise, and "we ran out of IPs" is a genuine scaling limit teams hit.

Addressing across regions

The scenario from this lesson's opener. Two services in different regions have several ways to reach each other, and they differ in exactly the ways that matter:

ApproachPathEncryptionNotes
Public internet + TLSOver the internetTLSSimplest, variable latency, egress costs
VPC peeringProvider backboneNot by defaultNon-overlapping CIDRs required
Transit gateway / hubProvider backboneNot by defaultScales past pairwise peering
VPN tunnelEncrypted tunnelIPsecCommon for hybrid and on-prem
Private interconnectDedicated linkVariesPredictable latency, higher fixed cost

The point to make in an interview is that cross-region latency is physics (Lesson 1.1), so no addressing choice fixes it. What these choices affect is whether traffic traverses the public internet, whether it is encrypted by default, whether egress is billed, and whether your address plan even permits the connection. That last one is why the CIDR planning point above is not pedantry.

Addresses are weak identity

Firewall rules and allow-lists key on addresses, and in any dynamic environment addresses move constantly: containers get new ones on every restart, autoscaling adds and removes hosts, and a NATed fleet shares one.

That is why service-to-service authentication moved to cryptographic identity (mTLS certificates, signed tokens) rather than network location, which is the core argument of zero trust: the network position of a caller is evidence, not proof. Identity and Trust covers this in The Service Identity Problem.

Use network controls as a coarse outer boundary, and authenticate the caller properly regardless (Lesson 4.1).


System Design Implications

  • Subnet layout is the skeleton of your security model. Public, private, and data tiers with routing between them is the expected answer for "how do you protect the database," and it is stronger than any application-level control.
  • Address planning is irreversible. Overlapping ranges block future peering, acquisitions, and hybrid connectivity. Allocate deliberately.
  • NAT shapes both cost and observability. Egress charges, a shared source address, and the loss of per-host attribution in downstream logs all follow from it.
  • Address exhaustion is a real scaling ceiling, particularly in Kubernetes where every pod consumes one.
  • Never treat a source address as authentication. It is spoofable, shared, and constantly reassigned.

Tradeoffs and Decision Framework

Large versus small subnets. Large avoids exhaustion and wastes space you may want for other environments. Small conserves space and risks running out during a scale-up, which is a painful failure to hit under load.

Public internet versus private connectivity. Internet plus TLS is simple, universally available, and variable in latency. Private paths give predictable performance and cost more and require address planning.

IPv4 with NAT versus dual-stack. IPv4 is universally supported and constrained. Dual-stack is future-proof and doubles your firewall surface, which is a genuine security risk if you only harden one.

Network controls versus identity controls. Network boundaries are coarse, cheap, and effective as an outer layer. Identity is precise and correct in dynamic environments. Use both; rely on identity.


Common Mistakes

Reusing default CIDR ranges across environments. Guarantees an overlap problem the first time anything needs to connect.

Sizing subnets by current need only. Autoscaling stops when addresses run out, at exactly the worst moment.

Assuming private means secure. It means unreachable from the internet. Lateral movement inside the network is unaffected.

Treating source IP as identity. Shared by NAT, reassigned constantly, and spoofable.

Ignoring NAT gateway cost. A surprising bill for high-egress services, often avoidable with private endpoints.

Hardening IPv4 only in a dual-stack deployment. Leaves an equivalent path open.

Forgetting cloud-reserved addresses. A /28 does not give you sixteen usable addresses.


INTERVIEW QUESTION

Explain the difference between a public and private IP address, and why NAT matters in a system design.