TCP/IP Fundamentals for CDN Engineers
TCP/IP (Transmission Control Protocol / Internet Protocol) is the foundational protocol suite that enables all communication on the internet — every email, web page, video stream, and API call relies on it. Developed in the 1970s by Vint Cerf and Bob Kahn as part of a U.S. Department of Defense research project, TCP/IP replaced the original ARPANET protocols and became the universal standard for networked communication. For CDN engineers, understanding TCP/IP is essential because CDN TCP optimization directly impacts content delivery performance.
How Was TCP/IP Created?
The story of TCP/IP begins with ARPANET, the world's first packet-switched network, which went live on October 29, 1969 — connecting UCLA, Stanford, UC Santa Barbara, and the University of Utah. ARPANET used a protocol called NCP (Network Control Protocol) that worked well for a single, homogeneous network but couldn't interconnect different types of networks.
In 1973, Vint Cerf (then at Stanford) and Bob Kahn (at DARPA) published "A Protocol for Packet Network Intercommunication," describing a new protocol that could connect disparate networks into an "internetwork." This protocol evolved into TCP/IP through several iterations:
- 1974: First TCP specification published (RFC 675), combining transport and internet layers
- 1978: TCP split into TCP (transport) and IP (internet) layers — creating the dual-protocol architecture we use today
- 1981: IPv4 specification finalized (RFC 791)
- January 1, 1983 ("Flag Day"): ARPANET switched from NCP to TCP/IP in a single coordinated cutover. Every connected host had to upgrade simultaneously — one of the most remarkable engineering feats in internet history
- 1991: The World Wide Web launched on top of TCP/IP, using HTTP as an application-layer protocol
Cerf and Kahn received the Presidential Medal of Freedom in 2005 and the Turing Award in 2004 for their work — recognition that TCP/IP is arguably the most impactful engineering achievement of the late 20th century.
What Are the Four Layers of TCP/IP?
TCP/IP organizes network communication into four layers, each with specific responsibilities:
| Metric | Edge Processing | Origin Processing | Improvement |
|---|---|---|---|
| Latency (p50) | 8–15 ms | 120–300 ms | 15–20x faster |
| Cache Hit Ratio | 85–95% | N/A | Reduces origin load |
| TLS Handshake | 1–5 ms (nearby PoP) | 50–150 ms | 10–30x faster |
| Bandwidth Cost | $0.01–0.04/GB | $0.05–0.12/GB | 3–5x cheaper |
| Availability | 99.99% (multi-PoP) | 99.9% (single DC) | 10x fewer outages |
1. Link Layer (Network Access): Handles the physical transmission of data over a specific medium — Ethernet cables, Wi-Fi radio waves, fiber optic light pulses, or cellular signals. This layer manages MAC addresses, frame formatting, and error detection at the hardware level. CDN edge servers typically connect via high-speed Ethernet (10/25/100 Gbps) to data center switches.
2. Internet Layer (IP): Responsible for addressing and routing packets across networks. The Internet Protocol (IP) assigns each device a unique address and determines the best path for packets to reach their destination. IPv4 uses 32-bit addresses (4.3 billion possible), while IPv6 uses 128-bit addresses (340 undecillion possible). CDNs use BGP routing at this layer for anycast addressing.
3. Transport Layer (TCP/UDP): Provides end-to-end communication between applications. TCP guarantees reliable, ordered delivery through acknowledgments and retransmissions — essential for web pages and API calls. UDP prioritizes speed over reliability — used for DNS queries, video streaming, and gaming. QUIC, the protocol underlying HTTP/3, operates at this layer as a UDP-based transport with TCP-like reliability.
4. Application Layer: The protocols that applications directly interact with. HTTP/HTTPS for web content, DNS for name resolution, SMTP for email, and SSH for remote access all operate here. CDN edge servers primarily handle HTTP/HTTPS traffic, with Cache-Control headers controlling caching behavior.
How Does the TCP Three-Way Handshake Work?
Before any data can flow over TCP, the client and server must establish a connection through a three-step process called the "three-way handshake":
- SYN: The client sends a SYN (synchronize) packet to the server, proposing initial sequence numbers and connection parameters. This packet contains no application data.
- SYN-ACK: The server responds with SYN-ACK, acknowledging the client's SYN and sending its own synchronization data. The server allocates resources for the connection at this point.
- ACK: The client acknowledges the server's SYN-ACK. With this final packet, the connection is established, and data transfer can begin.
This handshake takes one full round-trip time (RTT). For a user in Sydney connecting to a server in New York (RTT ~250ms), the handshake alone adds 250ms before any content loads. With TLS on top, add another 1-2 RTTs for the TLS handshake — potentially 500-750ms of latency before the first byte of content arrives.
This is precisely why CDNs exist: by placing edge servers close to users, they reduce RTT from hundreds of milliseconds to single digits, dramatically accelerating the TCP handshake and subsequent data transfer.
How Do CDNs Optimize TCP Performance?
CDN providers employ several TCP optimizations that go beyond simple geographic proximity:
- TCP Fast Open (TFO): Allows data to be sent during the SYN handshake itself, eliminating one RTT for repeat connections. The server issues a cryptographic cookie on first connection; subsequent connections include this cookie in the SYN packet along with application data.
- Congestion Control Algorithms: CDNs use optimized algorithms like BBR (developed by Google) instead of traditional CUBIC. BBR models the network path to achieve higher throughput with lower latency, particularly effective on long-distance or lossy connections. Understanding CDN latency optimization requires familiarity with these algorithms.
- Connection Reuse: CDN edge servers maintain persistent TCP connections to origin servers (called "origin shields" or "connection pooling"), avoiding repeated handshakes. A single persistent connection can serve thousands of different user requests. This is a core aspect of connection coalescing.
- Window Scaling: TCP's default 64KB receive window limits throughput on high-latency connections. CDN servers use TCP window scaling (RFC 7323) with windows up to 1GB, enabling full utilization of high-bandwidth links.
What Is the Difference Between IPv4 and IPv6?
| Feature | IPv4 | IPv6 |
|---|---|---|
| Address Size | 32 bits (4 bytes) | 128 bits (16 bytes) |
| Address Format | 192.168.1.1 | 2001:0db8:85a3::8a2e:0370:7334 |
| Total Addresses | ~4.3 billion | ~340 undecillion |
| Header Size | 20-60 bytes (variable) | 40 bytes (fixed) |
| NAT Required | Yes (address exhaustion) | No (enough addresses) |
| IPsec | Optional | Built-in support |
| CDN Support | Universal | All major CDNs support dual-stack |
When Should You Use TCP vs UDP?
The choice between TCP and UDP depends on whether your application prioritizes reliability or speed:
- Use TCP when: Data must arrive completely and in order — web pages (HTTP), file transfers, email, database queries, API calls. Most CDN traffic uses TCP (via HTTP/HTTPS).
- Use UDP when: Speed matters more than perfect delivery — DNS queries (small, single-packet exchanges), live video streaming (a dropped frame is better than a delayed one), online gaming (stale position data is worthless), and VoIP calls.
- QUIC (HTTP/3): The newest option combines UDP's speed with TCP's reliability. QUIC runs over UDP but provides TCP-like ordering, flow control, and congestion management — plus built-in TLS 1.3 encryption. CDNs are rapidly adopting QUIC for HTTP/3 support.