Networking Fundamentals for DevOps Engineers

How Packets Actually Flow

You type https://devopsbeast.com into Chrome and hit Enter. In roughly 300 milliseconds, the page loads. Text appears. Images render. It feels instant.

But in those 300 milliseconds, your request crossed seven OSI layers, triggered a DNS lookup, completed a three-way TCP handshake, negotiated a TLS session, sent an HTTP request, and received a response — possibly routed through a CDN, a load balancer, and a reverse proxy before reaching the actual application server.

Understanding this journey is not academic. When something is slow or broken, knowing which hop in this chain is the bottleneck is the difference between debugging for two minutes and debugging for two hours.


The Complete Journey: Browser to Server

Let us trace the entire path of a single HTTPS request. We will go step by step, layer by layer, and show you what is actually happening on the wire at each stage.

Step 1: DNS Resolution — "What IP Is This Hostname?"

Before your browser can send a single byte to devopsbeast.com, it needs an IP address. Hostnames are a human convenience. The network operates on IP addresses.

Your browser checks its DNS cache first. If it has a recent answer for devopsbeast.com, it skips DNS entirely. If not, it asks the operating system resolver, which checks its own cache. If that misses too, a DNS query goes out to your configured DNS resolver (usually your ISP, or 8.8.8.8 / 1.1.1.1 if you changed it).

# What your system actually does behind the scenes:
# 1. Check browser DNS cache (Chrome: chrome://net-internals/#dns)
# 2. Check OS DNS cache
#    macOS: sudo dscacheutil -flushcache  (to clear it)
#    Linux: systemd-resolve --statistics  (to check it)
# 3. Query configured resolver
dig devopsbeast.com A +short
# 104.21.45.67

# The full query with timing:
dig devopsbeast.com A +stats
# ;; Query time: 12 msec   <-- This is the DNS latency
# ;; SERVER: 1.1.1.1#53    <-- Which resolver answered

The DNS query travels as a UDP packet on port 53. It is small (usually under 512 bytes) and does not require a TCP handshake, which is why DNS uses UDP — speed matters here.

KEY CONCEPT

DNS resolution adds latency to every new connection. If your DNS resolver is slow (50-100ms), every first request to a new domain is 50-100ms slower. This is why using fast resolvers like Cloudflare (1.1.1.1) or Google (8.8.8.8) matters, and why DNS caching at every layer (browser, OS, pod, node) is critical for performance.

The resolver returns: devopsbeast.com → 104.21.45.67. Your browser now has an IP address to connect to.

Step 2: TCP Three-Way Handshake — Establishing the Connection

With an IP address in hand, your browser initiates a TCP connection to 104.21.45.67 on port 443 (HTTPS). TCP requires a handshake before any data can flow.

TCP Three-Way Handshake

Click each step to explore

This handshake takes exactly one round-trip time (RTT). If the server is 50ms away, the handshake takes 50ms. If the server is on another continent (150ms RTT), the handshake alone costs 150ms. This is why CDNs (Content Delivery Networks) place servers close to users — to reduce RTT.

# Measure RTT to a server
ping -c 5 104.21.45.67
# round-trip min/avg/max = 5.2/6.1/8.3 ms

# Watch the handshake with tcpdump
sudo tcpdump -i eth0 -nn host 104.21.45.67 and port 443
# 10:00:00.001 IP 192.168.1.50.54321 > 104.21.45.67.443: Flags [S], seq 12345
# 10:00:00.007 IP 104.21.45.67.443 > 192.168.1.50.54321: Flags [S.], seq 67890, ack 12346
# 10:00:00.007 IP 192.168.1.50.54321 > 104.21.45.67.443: Flags [.], ack 67891
# S = SYN, S. = SYN-ACK, . = ACK
PRO TIP

If nc -zv host port shows a long delay before "connected," the bottleneck is RTT during the TCP handshake. If it says "connection refused" instantly, the server is reachable but nothing is listening on that port. If it times out completely, a firewall is silently dropping SYN packets. These three outcomes tell you completely different things.

Step 3: TLS Handshake — Encrypting the Connection

With TCP established, HTTPS requires a TLS handshake before any HTTP data flows. This is where encryption is negotiated. TLS 1.3 (the current standard) requires one additional round-trip.

The TLS handshake exchanges:

  • ClientHello: your browser tells the server which TLS versions and cipher suites it supports
  • ServerHello: the server picks a version and cipher, sends its certificate
  • Key exchange: both sides derive shared encryption keys using Diffie-Hellman
  • Finished: both sides confirm the handshake succeeded
# Watch the TLS handshake
openssl s_client -connect devopsbeast.com:443 -servername devopsbeast.com

# Key output to look for:
# Protocol  : TLSv1.3
# Cipher    : TLS_AES_256_GCM_SHA384
# Server certificate:
#   subject=CN = devopsbeast.com
#   issuer=C = US, O = Let's Encrypt, CN = R3
#   notAfter=Jul 15 00:00:00 2026 GMT
WARNING

TLS certificate errors are one of the most common Layer 6 issues. The top three: (1) certificate expired, (2) certificate does not match the hostname (CN or SAN mismatch), (3) incomplete certificate chain (missing intermediate CA). Always check with openssl s_client — it gives you the exact error, unlike a browser which just says "your connection is not private."

At this point, you have spent two round-trips (one for TCP, one for TLS) and zero bytes of application data have been exchanged. With TLS 1.3 session resumption and TCP Fast Open, subsequent connections can reduce this to zero round-trips — but the first connection always pays this cost.

Step 4: HTTP Request — Sending Your Actual Data

Now your browser sends the HTTP GET request over the encrypted TLS connection:

GET / HTTP/2
Host: devopsbeast.com
User-Agent: Mozilla/5.0 ...
Accept: text/html,application/xhtml+xml
Accept-Encoding: gzip, br

This request is encrypted by TLS, segmented by TCP, packaged into IP packets, and transmitted as Ethernet frames. Each layer adds its own header.

Step 5: Encapsulation — How Layers Wrap Each Other

This is where the OSI model becomes tangible. Each layer takes the data from the layer above and wraps it with its own header. This process is called encapsulation.

Packet Encapsulation — Layer by Layer

Layer 7: HTTP Data

The actual payload: GET / HTTP/2, headers, and body. This is what your application sees. Typically 200 bytes to several MB.

Layer 4: TCP Header (20 bytes)

Source port (54321), destination port (443), sequence number, acknowledgment number, flags (SYN/ACK/FIN), window size. TCP handles reliable delivery and ordering.

Layer 3: IP Header (20 bytes)

Source IP (192.168.1.50), destination IP (104.21.45.67), TTL (Time to Live), protocol (TCP=6), total length. IP handles routing across networks.

Layer 2: Ethernet Header (14 bytes) + Trailer (4 bytes)

Source MAC, destination MAC, EtherType (0x0800 = IPv4). The Ethernet frame carries the IP packet across the local network segment to the next hop (usually your router).

Hover to expand each layer

A typical HTTP request in a TCP/IP packet has roughly 54 bytes of overhead (14 Ethernet + 20 IP + 20 TCP) before a single byte of your application data. This is why MTU (Maximum Transmission Unit) matters.

Step 6: MTU and Fragmentation — When Packets Are Too Big

The MTU is the maximum size of a single Layer 2 frame. Standard Ethernet MTU is 1500 bytes. This means the maximum IP packet that can fit in one Ethernet frame is 1500 bytes (including IP and TCP headers), leaving 1460 bytes for application data (this is called MSS — Maximum Segment Size).

If your application tries to send more than 1460 bytes (and it will — web pages are much bigger), TCP splits the data into multiple segments, each fitting within the MTU.

# Check your interface MTU
ip link show eth0
# mtu 1500

# Test path MTU to a destination
ping -M do -s 1472 10.0.1.50
# 1472 + 8 (ICMP header) + 20 (IP header) = 1500 = exactly MTU
# If this works: MTU is at least 1500 on the entire path
# If "message too long": a hop has a smaller MTU

# Common MTU values:
# 1500 — standard Ethernet
# 9000 — jumbo frames (data center internal)
# 1450 — VXLAN overlay networks (50 bytes VXLAN overhead)
# 1400 — common in VPN tunnels
WAR STORY

A classic production issue: "curl works but the application is broken." The application was sending large POST bodies. Curl sends small requests that fit in one packet. The application sends 8KB payloads that require fragmentation. A misconfigured firewall was dropping fragmented packets. The fix: set iptables -A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu to enable Path MTU Discovery. Or simply reduce MTU on the interface.

Step 7: NAT — Translating Private to Public Addresses

Your machine likely has a private IP address like 192.168.1.50. The internet does not route private addresses. Your router performs NAT (Network Address Translation) to replace your private IP with its public IP before the packet leaves your network.

Before NAT (inside your network):
  Source: 192.168.1.50:54321 → Dest: 104.21.45.67:443

After NAT (on the internet):
  Source: 203.0.113.10:32768 → Dest: 104.21.45.67:443

The router keeps a NAT table mapping the internal source (192.168.1.50:54321) to the external port it assigned (203.0.113.10:32768). When the response comes back to 203.0.113.10:32768, the router translates it back and forwards to 192.168.1.50:54321.

KEY CONCEPT

NAT is everywhere. Your home router does it. Your cloud VPC does it (private subnets with NAT gateways). Kubernetes does it (kube-proxy SNAT for traffic leaving the cluster). Understanding NAT is essential because it affects connection tracking, port exhaustion, and debugging — the source IP in your server logs is the NAT gateway, not the actual client.

Step 8: The Response Journey

The server processes your request and sends back an HTTP response. This response takes the reverse path:

  1. Server application generates HTTP 200 OK with HTML body (Layer 7)
  2. Response is encrypted by TLS
  3. TCP segments the response (might be hundreds of KB)
  4. Each segment gets an IP header with source=104.21.45.67, dest=203.0.113.10
  5. Ethernet frames carry each packet to the next hop
  6. Your router performs reverse NAT (dest 203.0.113.10:32768 becomes 192.168.1.50:54321)
  7. TCP reassembles all segments in order
  8. TLS decrypts the data
  9. Browser receives the HTML and starts rendering

The Full Picture: Timing Breakdown

Let us put it all together with realistic timing for a request from New York to a server in Virginia (roughly 10ms RTT):

StepTimeCumulativeWhat Happens
DNS lookup5ms5msCached in local resolver
TCP handshake10ms15msOne round-trip (SYN, SYN-ACK, ACK)
TLS handshake10ms25msOne round-trip (TLS 1.3)
HTTP request1ms26msSmall GET request
Server processing50ms76msApplication generates response
HTTP response10ms86msResponse travels back
Total~86ms

With a CDN edge server nearby (2ms RTT), the same flow takes roughly 20ms. This is why CDNs matter for web performance — they collapse the RTT for TLS and TCP handshakes from tens of milliseconds to single digits.

Complete Packet Journey: Browser to Server

Click each step to explore


Where Things Break: Common Bottlenecks

Now that you understand the full flow, here is where things typically go wrong:

DNS Latency

# Measure DNS resolution time
dig devopsbeast.com A +stats | grep "Query time"
# ;; Query time: 87 msec   <-- This is too slow

# Compare with a fast resolver
dig @1.1.1.1 devopsbeast.com A +stats | grep "Query time"
# ;; Query time: 3 msec    <-- Much better

If DNS takes 50-100ms on every new connection, your application feels sluggish. Fix: use faster resolvers, enable DNS caching (NodeLocal DNSCache in K8s), reduce TTLs carefully.

TCP Connection Overhead

Every new TCP connection costs one RTT for the handshake. If your application makes many short-lived connections to different backends, you are paying this cost repeatedly.

# Check how many connections are being established
ss -s
# TCP:   2500 (estab 1800, closed 200, timewait 500)
#                                       ^^^^^^^^
# High TIME_WAIT means many short-lived connections
PRO TIP

HTTP/2 and HTTP/3 multiplex many requests over a single TCP connection, eliminating the per-request handshake cost. If your services are still using HTTP/1.1 with new connections per request, migrating to HTTP/2 can dramatically reduce latency. Most reverse proxies (nginx, envoy) support HTTP/2 out of the box.

TLS Handshake Cost

TLS 1.2 requires two round-trips. TLS 1.3 requires one. On high-latency links (cross-continent), this difference is significant.

# Check which TLS version a server supports
openssl s_client -connect devopsbeast.com:443 2>/dev/null | grep Protocol
# Protocol  : TLSv1.3

MTU/Fragmentation Issues

The classic symptom: small requests (ping, small curl) work, but large data transfers fail or stall.

# Test if fragmentation is causing issues
# Send a packet at exactly MTU size with "don't fragment" flag
ping -M do -s 1472 10.0.1.50
# PING 10.0.1.50: 1472 data bytes
# If "Frag needed and DF set": a hop has a smaller MTU

NAT Port Exhaustion

A NAT gateway has a limited number of ephemeral ports (roughly 64,000). If your application makes thousands of outbound connections per second through a single NAT gateway, you can exhaust the port pool.

# Check connection tracking table size (Linux NAT)
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max
# If count approaches max, new connections will be dropped
WARNING

In AWS, a single NAT Gateway supports 55,000 simultaneous connections to a single destination IP and port. If your pods make heavy outbound connections (scraping, API calls, database connections) through a NAT Gateway, you can hit this limit. The symptom is intermittent connection timeouts with no clear pattern. Fix: use multiple NAT Gateways, or give pods public IPs.


Key Concepts Summary

  • Every HTTPS request involves at minimum: DNS lookup, TCP handshake, TLS handshake, and HTTP exchange — understanding each step lets you pinpoint where latency lives
  • DNS uses UDP port 53 and adds 1-100ms depending on cache status — slow DNS affects every new connection
  • TCP three-way handshake costs exactly one RTT — this is why CDNs and connection reuse matter
  • TLS 1.3 adds one RTT, TLS 1.2 adds two — always prefer TLS 1.3
  • Encapsulation adds 54 bytes of overhead per packet (Ethernet + IP + TCP headers) — this matters for MTU calculations
  • MTU (1500 bytes standard) limits frame size — VPN and overlay networks reduce effective MTU, causing fragmentation issues
  • NAT translates private IPs to public IPs — it happens at your router, cloud NAT gateways, and in Kubernetes (kube-proxy SNAT)
  • NAT port exhaustion is a real production issue — 55,000 connection limit per destination on AWS NAT Gateway
  • HTTP/2 multiplexing eliminates per-request TCP overhead — upgrade from HTTP/1.1 for significant latency reduction

Common Mistakes

  • Assuming DNS is instant — uncached DNS lookups can add 50-100ms to every new connection
  • Ignoring TCP handshake cost for short-lived connections — connection pooling and HTTP/2 eliminate this overhead
  • Using TLS 1.2 when TLS 1.3 is available — you are paying an extra round-trip for no benefit
  • Not testing with large payloads — small pings and curl requests may work while large transfers fail due to MTU issues
  • Forgetting about NAT when reading server-side logs — the source IP is the NAT gateway, not the actual client
  • Not accounting for connection tracking limits — nf_conntrack_max defaults to 65536, which busy servers can exceed easily

KNOWLEDGE CHECK

During an HTTPS request, how many round-trips are required before the first byte of application data is sent (assuming TLS 1.3 and no prior connection)?