Web Servers: Apache, Nginx, and Beyond
Web servers are the software programs that power every website on the internet — listening for incoming HTTP requests, processing them, and returning the appropriate response. From the first web server at CERN in 1990 to today's high-performance event-driven architectures handling millions of concurrent connections, web server technology has evolved dramatically. Understanding how web servers work is fundamental to understanding how CDN architecture delivers content at scale, because every CDN edge server is, at its core, a specialized web server.
How Did Web Servers Originate?
The history of web servers is the history of the World Wide Web itself:
- CERN httpd (1990): Tim Berners-Lee wrote the first web server on a NeXT computer at CERN. It ran on port 80 and could serve HTML files from the local filesystem. This same machine also ran the first web browser (WorldWideWeb) — Berners-Lee was both the first web developer and the first web server administrator.
- NCSA HTTPd (1993): Developed by Rob McCool at the National Center for Supercomputing Applications (NCSA), this was the first widely deployed web server. It introduced the concept of CGI (Common Gateway Interface), allowing web servers to execute external programs to generate dynamic content.
- Apache HTTP Server (1995): When Rob McCool left NCSA and development stalled, a group of webmasters began sharing patches for NCSA HTTPd — creating "a patchy server" that became Apache. The Apache Software Foundation formed around this project, and Apache dominated the web server market for over a decade, peaking at 70% market share in 2005.
- Nginx (2004): Igor Sysoev, a Russian system administrator, created Nginx (pronounced "engine-x") to solve the C10K problem — the challenge of handling 10,000 concurrent connections on a single server. Nginx's event-driven architecture consumed far less memory than Apache's process-per-connection model, making it the preferred choice for high-traffic websites. By 2019, Nginx surpassed Apache as the most-used web server.
What Is the Difference Between Apache and Nginx?
The Apache vs Nginx comparison illustrates a fundamental architectural choice in server design:
| Metric | Edge Processing | Origin Processing | Improvement |
|---|---|---|---|
| Latency (p50) | 8–15 ms | 120–300 ms | 15–20x faster |
| Cache Hit Ratio | 85–95% | N/A | Reduces origin load |
| TLS Handshake | 1–5 ms (nearby PoP) | 50–150 ms | 10–30x faster |
| Bandwidth Cost | $0.01–0.04/GB | $0.05–0.12/GB | 3–5x cheaper |
| Availability | 99.99% (multi-PoP) | 99.9% (single DC) | 10x fewer outages |
| Feature | Apache (httpd) | Nginx |
|---|---|---|
| Architecture | Process/Thread per connection | Event-driven, asynchronous |
| Memory per Connection | ~10-15 MB | ~2.5 KB |
| Concurrent Connections | Hundreds to low thousands | Tens of thousands to millions |
| Static File Serving | Good | Excellent (2-3x faster) |
| Dynamic Content | mod_php, mod_perl (in-process) | FastCGI, proxy_pass (separate process) |
| Configuration | .htaccess (per-directory) | Centralized config only |
| Module Loading | Dynamic (can load at runtime) | Compiled in (mostly) |
| Best For | Shared hosting, PHP apps | High-traffic sites, reverse proxy, CDN origin |
Most CDN and high-traffic web architectures now use Nginx (or its commercial version, Nginx Plus) as a reverse proxy in front of application servers. Nginx handles TLS termination, static file serving, load balancing, and connection management, while application servers (Node.js, Python/Gunicorn, PHP-FPM, Go) handle business logic.
What Modern Web Servers Are Available Today?
Beyond Apache and Nginx, several modern web servers have emerged:
- Caddy (2015): Written in Go, Caddy provides automatic HTTPS via Let's Encrypt with zero configuration. Its Caddyfile configuration format is dramatically simpler than Nginx or Apache. Caddy is increasingly popular for development environments and small to mid-size production deployments.
- LiteSpeed (2003): A commercial web server compatible with Apache's .htaccess files but with Nginx-like event-driven performance. Its open-source variant, OpenLiteSpeed, gained popularity for WordPress hosting due to its built-in page cache (LSCache).
- Traefik (2016): A cloud-native reverse proxy and load balancer designed for microservices. Traefik automatically discovers services in container orchestration platforms (Docker, Kubernetes) and configures routing, TLS, and load balancing dynamically.
- HAProxy: While technically a load balancer rather than a web server, HAProxy is widely used in CDN and high-traffic architectures for TCP/HTTP load balancing, with sub-millisecond latency overhead.
How Do Web Servers Function as CDN Origins?
When a website uses a CDN, its web server becomes the "origin" — the authoritative source of content that the CDN caches and distributes. Proper origin server configuration is critical for CDN performance:
- Cache Headers: The origin must send proper Cache-Control headers telling the CDN how long to cache each resource. Static assets (images, CSS, JS) typically get
max-age=31536000(1 year) with content-hash filenames, while HTML pages get shorter TTLs. - Health Check Endpoints: CDN providers periodically probe the origin to verify it's healthy. The origin should expose a lightweight health endpoint that returns 200 OK without expensive processing.
- Connection Limits: CDN edge servers maintain persistent connections to the origin. Nginx's
keepalivedirective and appropriateworker_connectionssettings ensure the origin can handle the CDN's connection pool without exhausting resources. - Compression: Origins should serve content with Brotli or gzip compression. CDN edge servers can also apply compression, but origin-level compression reduces bandwidth between origin and edge.
What Are CDN Edge Servers Made Of?
CDN edge servers are specialized web servers optimized for caching and proxying at massive scale. Major CDN providers use custom server stacks:
- Varnish Cache: An HTTP reverse proxy designed specifically for caching. Varnish uses VCL (Varnish Configuration Language) for fine-grained cache control and can serve cached content at wire speed. Fastly's edge platform is built on a heavily customized version of Varnish.
- Apache Traffic Server (ATS): Originally developed at Yahoo and donated to Apache. ATS is used by Akamai, Apple, and Comcast as the foundation of their CDN edge infrastructure. It handles caching, proxying, and connection management at scale.
- Custom Stacks: Cloudflare uses a custom Nginx-based stack with Lua scripting (OpenResty) and, increasingly, Rust-based components. Google Cloud CDN uses Google's internal web serving infrastructure. These custom stacks integrate caching, TLS, WAF, and edge compute into unified systems.