WebSockets break the classic CDN model. Instead of short-lived, cacheable HTTP requests, they create long-lived, stateful TCP connections that must stay open for minutes to hours. Scaling them at the edge requires a fundamentally different approach to connection management, load balancing, and failure handling.
When your real-time chat, live dashboard, multiplayer game, or collaborative editor goes from 1,000 to 500,000 concurrent users, the bottleneck is rarely bandwidth. It is file descriptors, memory per connection, origin fan-out, and the load balancer's ability to route persistent sessions without dropping frames. This guide covers production-tested patterns for scaling WebSockets through a CDN without sacrificing latency or reliability.
Why Persistent Connections Challenge Traditional CDNs
A standard CDN is optimized for the request-response cycle: accept a TCP connection, serve a cached asset, close or keep-alive briefly, and reuse the socket. WebSockets invert this. After the initial HTTP Upgrade handshake (101 Switching Protocols), the connection becomes a full-duplex message stream with no inherent request boundaries. Understanding the CDN fundamentals guide helps clarify why this stateful model requires dedicated edge logic.
Three constraints dominate at scale:
- Memory footprint: Each idle WebSocket holds a file descriptor, kernel buffers (typically 16-64KB per direction), and user-space state. At 1 million concurrent connections, that is 32-128GB of buffer memory alone, before application logic.
- Origin connection amplification: Without edge termination, every client connects directly to origin. With 200 PoPs and 500k clients, your origin would need to handle 500k sockets. With edge aggregation, the CDN can multiplex 500k client connections into 2k-5k persistent connections to origin using connection pooling and pub/sub fan-out.
- State affinity: Unlike stateless HTTP, WebSocket frames often depend on server-side session state. A naive round-robin load balancer that sends frame 1 to Origin-A and frame 2 to Origin-B will break the application unless you externalize state.
Edge termination solves the first two problems by moving the cost of idle connections to the CDN's globally distributed, highly tuned edge nodes. The third problem — state — is solved through architecture, not just infrastructure.
Connection Management at the Edge
Effective edge management starts at the TCP and TLS layer, long before the WebSocket handshake. Modern edge stacks use kernel-bypass or io_uring, TCP Fast Open, and TLS 1.3 0-RTT to reduce handshake latency to under 50ms globally.
| Metric | Edge Processing | Origin Processing | Improvement |
|---|---|---|---|
| Latency (p50) | 8–15 ms | 120–300 ms | 15–20x faster |
| Cache Hit Ratio | 85–95% | N/A | Reduces origin load |
| TLS Handshake | 1–5 ms (nearby PoP) | 50–150 ms | 10–30x faster |
| Bandwidth Cost | $0.01–0.04/GB | $0.05–0.12/GB | 3–5x cheaper |
| Availability | 99.99% (multi-PoP) | 99.9% (single DC) | 10x fewer outages |
Once terminated, the edge must manage the lifecycle efficiently. Key tunings include:
1. Idle Timeouts and Heartbeats
Intermediaries — NAT gateways, corporate proxies, and mobile carriers — silently drop idle TCP connections after 60-300 seconds. If your application only sends data sporadically, you must implement application-level ping/pong. Configure the edge to send ping frames every 25 seconds and expect pong within 10 seconds. This keeps the connection alive and provides early detection of dead peers without waiting for TCP retransmission timeouts, which can take 15 minutes.
2. Backpressure and Message Buffering
A slow client should not block the edge. Implement per-connection write buffers with high-water marks. When a client cannot consume fast enough, queue up to 1MB per connection, then apply backpressure: either drop non-critical messages (e.g., presence updates) or disconnect with 1013 Try Again Later. At the edge, use SO_SNDBUF tuning and non-blocking I/O to prevent one slow consumer from starving others.
3. Graceful Drain on Deploy
Edge nodes roll frequently. Never kill WebSockets abruptly. On shutdown, send a GOAWAY-style signal, stop accepting new upgrades, and allow existing connections 30-60 seconds to migrate via client-side reconnection with exponential backoff and jitter. Pair this with robust monitoring and observability to track drain success and reconnection storms.
Load Balancing Strategies for Stateful Connections
Choosing a load balancing algorithm for WebSockets is not the same as for HTTP. Because connections are long-lived, the initial routing decision persists for the entire session. A poor choice leads to hot shards and cascading failures.
| Strategy | Best For | Stickiness | Trade-off |
|---|---|---|---|
| Consistent Hash (IP / Session ID) | Stateful servers with in-memory rooms | Strong | Uneven load if key distribution is skewed; needs virtual nodes |
| Least Connections | Stateless origins + external pub/sub | None | Optimal balance, but requires shared state layer |
| Round Robin + Cookie Affinity | Hybrid apps (HTTP + WS on same domain) | Strong | Simple, but cookie handling adds overhead at edge |
| Anycast + Geographic | Global low-latency fan-out | Weak | Lowest latency; requires stateless design or CRDTs |
| Weighted / Latency-Aware | Heterogeneous origin pools | Configurable | Adapts to capacity; needs health-aware control plane |
For most modern architectures, we recommend least-connections with an external backplane over sticky sessions. It decouples routing from state, allowing any edge node to serve any client and enabling seamless failover. If you must maintain affinity, prefer consistent hashing over naive IP hash — it minimizes rebalancing when you add or remove origins. Our deep dive on CDN load balancing algorithms compares these approaches with benchmarks.
At the CDN layer, Anycast is your first load balancer: BGP routes the client to the nearest healthy PoP automatically. Within the PoP, a Layer 4 load balancer (e.g., Maglev, Katran) distributes to edge workers using consistent hashing on the 4-tuple. Finally, the edge worker proxies to origin using a Layer 7 pool with active health checks every 2 seconds and passive circuit breaking after 5 consecutive failures.
Scaling Patterns in Production
Scaling WebSockets is not vertical — you scale by sharding and decoupling. Three patterns dominate production deployments:
Pattern 1: Stateless Edge with Pub/Sub Fan-Out
The edge holds no business logic. It authenticates the Upgrade, subscribes the connection to channels (e.g., room:1234), and bridges frames to a central pub/sub system. Origins publish once; the backplane fans out to all subscribed edge nodes. This is how Slack, Discord, and live sports tickers scale to millions. The origin never sees 500k connections — it sees one publish per message.
Example edge worker logic for connection pooling and authentication:
export default {
async fetch(req, env) {
if (req.headers.get("Upgrade") !== "websocket") {
return fetch(req); // pass through normal HTTP
}
// Validate JWT before accepting upgrade
const token = new URL(req.url).searchParams.get("token");
const user = await verifyJWT(token, env.JWT_SECRET);
if (!user) return new Response("Unauthorized", { status: 401 });
const pair = new WebSocketPair();
const [client, server] = Object.values(pair);
server.accept();
// Subscribe to pub/sub backplane (Redis/NATS)
const sub = await env.PUBSUB.subscribe(`room:${user.roomId}`);
sub.on("message", (msg) => {
try { server.send(msg); } catch (e) { sub.unsubscribe(); }
});
server.addEventListener("message", async (event) => {
// Publish client message to backplane, not directly to origin
await env.PUBSUB.publish(`room:${user.roomId}`, event.data);
});
// Keepalive: ping every 25s
const keepalive = setInterval(() => {
try { server.ping(); } catch { clearInterval(keepalive); }
}, 25000);
return new Response(null, { status: 101, webSocket: client });
}
}
Pattern 2: Origin Connection Pooling and Multiplexing
When you cannot introduce a pub/sub layer, pool connections from edge to origin. Instead of 1:1 proxying, maintain 50-100 persistent WebSockets per edge node to origin and multiplex client frames over them using a lightweight framing protocol (e.g., prepend channel ID). This reduces origin file descriptors by 100x and allows TCP congestion windows to stay warm.
# nginx.conf — edge to origin
upstream websocket_origin {
least_conn;
server origin1.internal:8080 max_fails=3 fail_timeout=10s;
server origin2.internal:8080 max_fails=3 fail_timeout=10s;
keepalive 100; # pooled idle connections to origin
}
server {
listen 443 ssl http2;
location /ws/ {
proxy_pass http://websocket_origin;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Critical timeouts for long-lived conns
proxy_connect_timeout 5s;
proxy_read_timeout 3600s; # do not close idle WS quickly
proxy_send_timeout 3600s;
proxy_buffering off; # disable buffering for real-time frames
# Keepalive to origin pool
proxy_set_header Host $host;
}
}
Pattern 3: Regional Sharding with Presence Service
For global scale, shard by geography or by room ID. A presence service tracks which PoP holds which connection. When a message is published in us-east, the presence service routes it only to PoPs that have subscribers for that room, avoiding global broadcast. This pattern, combined with CDN performance and security benefits like DDoS shielding at the edge, protects your origin from both traffic spikes and attack traffic that often targets WebSocket endpoints.
Regardless of pattern, implement client-side resilience: exponential backoff with full jitter (base 1s, cap 30s), resume tokens to replay missed messages, and a fallback to long-polling or SSE if the Upgrade fails due to an enterprise proxy. Test reconnection storms by draining a PoP in staging and measuring time to steady state — it should be under 60 seconds for 100k clients.
Observability, Timeouts, and Failure Modes
WebSocket failures are silent. Unlike HTTP 5xx, a dropped WebSocket surfaces as a 1006 Abnormal Closure with no payload. You need metrics at four layers:
- Edge L4: Active connections, accept queue drops, TCP retransmits, TLS handshakes per second.
- Edge L7: Upgrade success rate (target >99.9%), message rate per second, p50/p99 frame latency, backpressure drops, ping/pong round-trip time.
- Origin pool: Pooled connection health, publish latency to backplane, consumer lag.
- Client: Time to first frame, reconnection rate, abnormal closure codes (1006, 1011, 1013).
The most common production incident is a timeout mismatch. Example: the CDN edge has a 60s idle timeout, the Layer 7 load balancer has a 300s timeout, and the origin has no timeout. The edge closes the connection after 60s of silence, the client sees 1006, and the origin still thinks the client is connected, leaking memory. Align all timeouts and enforce keepalives end-to-end. Alert on any increase in 1006 closures — it is your early warning for network or tuning issues.
Also watch for file descriptor exhaustion. Set ulimit -n to at least 1M, tune net.core.somaxconn, net.ipv4.tcp_max_syn_backlog, and enable reuseport to distribute accepts across workers. Use CDN monitoring best practices to correlate connection count with memory and CPU, and autoscale edge workers before you hit 70% of per-node capacity.
Finally, secure the Upgrade. Validate Origin headers, enforce JWT or signed cookies during the handshake, and rate-limit upgrades per IP (e.g., 20 upgrades/minute). WebSocket endpoints are a favorite for credential stuffing because they bypass traditional WAF rules that only inspect HTTP bodies. Terminate TLS at the edge and apply WAF policies to the initial HTTP request before it upgrades.
Checklist: Production-Ready CDN WebSockets
- Terminate TLS and WebSockets at the edge; multiplex to origin via pooled connections or pub/sub.
- Choose least-connections + stateless backplane over sticky sessions where possible.
- Implement 25s ping/pong keepalives and align proxy_read_timeout to 3600s+ across all hops.
- Set per-connection buffer limits and backpressure handling; never block the event loop.
- Enable graceful drain with 30-60s migration window and client-side jittered reconnection.
- Monitor upgrade success rate, 1006 closures, and file descriptor usage; alert on anomalies.
- Secure the handshake with JWT, Origin validation, and per-IP upgrade rate limiting.
- Load test with realistic idle + burst patterns, not just connection count. See our optimization guides for related performance tuning strategies.
Frequently Asked Questions
Can CDNs cache WebSocket messages?
No. WebSockets are bidirectional persistent connections, not cacheable HTTP responses. CDNs accelerate WebSockets by terminating connections at the edge, maintaining persistent TCP sessions, and proxying frames to origin efficiently, reducing latency and origin connection overhead rather than caching payloads.
How many concurrent WebSocket connections can a single edge node handle?
Capacity depends on memory, file descriptors, and kernel tuning. A well-tuned edge node with 32GB RAM can typically handle 500k to 1M idle connections, but active message throughput reduces this significantly. Most providers scale horizontally and limit per-node connections to 50k-200k for headroom.
Should WebSocket load balancing use sticky sessions?
Yes for stateful architectures. If your origin server holds in-memory session state, use IP hash or consistent hashing with cookie-based affinity. For stateless designs with a pub/sub backplane like Redis or NATS, anycast or least-connections without stickiness provides better resilience and failover.
What timeout settings are critical for CDN WebSockets?
Configure three timeouts: idle timeout (60-300s), proxy read timeout (3600s+ for long-lived connections), and keepalive interval (25-30s ping/pong). Mismatched timeouts between CDN edge, load balancer, and origin are the most common cause of unexpected 1006 abnormal closures.
Scaling WebSockets at the CDN edge is less about raw capacity and more about architectural decoupling. Terminate close to the user, keep connections warm with heartbeats, route intelligently based on your state model, and decouple fan-out from business logic with a pub/sub backplane. Done right, your origin will handle 10x the concurrent users with 100x fewer connections — and your users will feel the difference in every real-time interaction.