CDN Congestion Control: BBR vs CUBIC vs Reno for Edge Delivery

Congestion control determines how fast your CDN edge can send without collapsing the network. For edge delivery, the choice between Reno, CUBIC, and BBR directly impacts time-to-first-byte, throughput on lossy mobile links, and origin shielding efficiency. This comparison breaks down how each algorithm behaves under real-world CDN conditions.

Every TCP connection from a CDN PoP to an eyeball network traverses a different bottleneck: a congested peering link, a buffer-bloated home router, or a 5G radio scheduler. If you rely on default settings, you leave 20-40% performance on the table. Understanding the evolution from Reno to CUBIC to BBR is essential for tuning edge stacks, especially as CDN fundamentals shift toward high-bandwidth, high-latency global delivery.

Why Congestion Control Matters at the Edge

Unlike origin servers in controlled datacenters, CDN edge nodes face extreme path heterogeneity. A single PoP may serve a fiber user with 5 ms RTT and a rural LTE user with 120 ms RTT and 2% random loss simultaneously. The congestion controller decides the sending rate for each connection.

Poor control manifests as:

• Bufferbloat: Loss-based algorithms fill deep buffers, spiking latency for all flows on that bottleneck. This hurts LCP and interactive responsiveness.
• Underutilization: Overly conservative backoff after random loss leaves bottleneck bandwidth idle, slowing large asset delivery like video segments and images.
• Slow start convergence: On short CDN objects (10-200 KB), the connection may never leave slow start. How quickly you ramp matters more than steady-state throughput.

For a primer on the full request path, see how a CDN works — congestion control sits between TCP handshake and HTTP delivery, shaping every byte after the initial window.

Reno: The Loss-Based Baseline

TCP Reno (1990) is the classic Additive Increase Multiplicative Decrease (AIMD) algorithm. It probes for bandwidth by increasing its congestion window (cwnd) by one MSS per RTT until packet loss occurs, then cuts cwnd by 50%.

How Reno Works

Reno maintains two phases: slow start (exponential growth) and congestion avoidance (linear growth). It interprets any loss — whether from true congestion or random wireless interference — as a signal to halve its rate. Fast retransmit and fast recovery allow it to recover without a timeout, but the core logic remains loss-driven.

CDN Limitations

Reno performs poorly for modern edge delivery. On high bandwidth-delay product (BDP) paths, its linear increase is too slow to reclaim bandwidth after loss. A 100 Mbps link with 50 ms RTT has a BDP of ~625 KB. After a single loss event, Reno needs dozens of RTTs to recover, during which throughput collapses. On lossy mobile networks, Reno misinterprets radio loss as congestion and chronically underperforms. It is rarely used on production CDN edges today except as a fallback for legacy kernels, but understanding it clarifies why successors were needed.

CUBIC: The Current Linux Default

CUBIC (2008) replaced Reno as the default in Linux, Windows, and macOS. It is still the default on most CDN nodes unless explicitly changed to BBR. CUBIC improves high-BDP performance by making window growth a cubic function of time since the last loss, not RTT.

The Cubic Window Function

After a loss, CUBIC reduces cwnd by 30% (beta=0.7) rather than 50%, then grows according to W(t) = C*(t-K)³ + W_max, where K is the time to reach W_max again. This creates a concave-then-convex curve: rapid growth toward the previous maximum, then cautious probing beyond it. Crucially, growth is RTT-independent, so flows with different RTTs share bandwidth more fairly than Reno.

Strengths for CDN Use

CUBIC excels on stable, low-loss backbone paths — ideal for CDN midgress, origin fetch, and cache fill between PoPs. It fully utilizes high-BDP links and is well-tested at scale. When paired with effective caching strategies, CUBIC efficiently pulls large objects from origin over reliable transit.

Weaknesses at the Last Mile

CUBIC is still loss-based. On last-mile links with 0.5-2% non-congestion loss, it repeatedly cuts cwnd and never sustains the bottleneck rate. It also fills buffers until loss occurs, which increases queuing delay and jitter. For video and image delivery to mobile users, this buffer-filling behavior inflates 95th percentile latency even when average throughput looks acceptable.

BBR: Model-Based Congestion Control

Bottleneck Bandwidth and Round-trip propagation time (BBR), developed by Google, rethinks congestion control. Instead of reacting to loss, BBR builds an explicit model of the path: bottleneck bandwidth (BtlBw) and round-trip propagation time (RTprop). Introduced in 2016 (BBRv1) and refined in BBRv2 (2020) and BBRv3 (2023), it is now widely deployed on major CDNs including Google, Cloudflare, and Fastly.

How BBR Works

BBR continuously estimates BtlBw by measuring delivery rate and RTprop by tracking minimum RTT over a window. It then paces packets to send at BtlBw while keeping inflight data at BDP = BtlBw * RTprop — just enough to fill the pipe without queuing. It cycles through states:

• Startup: Doubles sending rate like slow start but exits early when bandwidth plateaus, avoiding massive loss.
• Drain: Drains queue created during Startup.
• ProbeBW: Steady state, cycling pacing gain (1.25, 0.75, 1, 1, 1, 1, 1, 1) to probe for more bandwidth and drain periodically.
• ProbeRTT: Briefly reduces inflight to 4 packets to re-measure RTprop if RTT hasn't been refreshed.

BBRv2/v3 add loss and ECN sensitivity, fairness improvements, and better coexistence with CUBIC flows — fixing BBRv1's tendency to be too aggressive against loss-based flows.

Advantages for Edge Delivery

Because BBR paces and limits inflight to ~1x BDP, it avoids bufferbloat. Queuing delay stays low, which improves TTFB and tail latency. On lossy links, BBR ignores random loss and maintains high throughput; tests on 1% loss links show BBR delivering 2-4x the throughput of CUBIC. For small CDN objects, BBR's Startup is faster and less lossy than CUBIC's slow start, reducing retransmits. This directly lowers egress cost and improves cache hit performance, a key factor in CDN cost optimization.

Head-to-Head

In lab benchmarks across 100 Mbps / 40 ms paths, BBR delivers 15-25% lower median RTT and 30% lower p95 RTT than CUBIC while maintaining equal or higher goodput. The gap widens under loss. However, CUBIC can outperform BBRv1 when competing with many CUBIC flows on shallow-buffered links — a reason to prefer BBRv2/v3, which incorporate explicit fairness and loss response.

Tuning for CDN Edge: Practical Configurations

Choosing an algorithm is only half the work. Kernel tuning and pacing are critical.

1. Enable BBR on Linux Edge Nodes

# Check available algorithms
sysctl net.ipv4.tcp_available_congestion_control
# net.ipv4.tcp_available_congestion_control = reno cubic bbr

# Enable BBR + fq pacing (required)
sysctl -w net.core.default_qdisc=fq
sysctl -w net.ipv4.tcp_congestion_control=bbr

# Persist
cat >> /etc/sysctl.conf <<EOF
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_notsent_lowat = 16384
EOF
sysctl -p

# Verify per-socket
ss -ti | grep bbr

2. Per-Route Selection (Hybrid Deployment)

Run BBR toward eyeballs and CUBIC toward origin:

# Egress to users via BBR
ip route change 0.0.0.0/0 via 192.0.2.1 congctl bbr

# Origin fetch via CUBIC over private backbone
ip route add 203.0.113.0/24 via 10.0.0.1 congctl cubic

# Or per-socket in application (e.g., NGINX/OpenResty)
setsockopt(fd, IPPROTO_TCP, TCP_CONGESTION, "bbr", 3);

3. Complementary Tuning

• Initial window (IW10-32): Increase tcp_init_cwnd to 32-64 for small object acceleration; validate with packet captures.
• Pacing: Always use fq qdisc with BBR; without pacing, BBR bursts and loses its latency benefit.
• ECN: Enable tcp_ecn=1 for BBRv2/v3 to get explicit congestion signals without loss.
• Metrics: Export tcp_info fields (delivery rate, min_rtt, bbr_bw) via Prometheus/node_exporter to detect bottleneck shifts.

Choosing the Right Algorithm for Your Workload

Use BBRv2/v3 if: You serve global users over diverse access networks, deliver video, images, or large JS bundles, or see high retransmit rates (>0.5%) and bufferbloat on edge. It is the best default for modern CDN egress.

Use CUBIC if: Your workload is primarily cache fill over clean, low-loss transit, you run kernels without BBRv2 support, or you need strict fairness with legacy infrastructure. Many CDNs keep CUBIC for inter-PoP replication.

Avoid Reno if: You are on Linux 3.x+ — there is no performance reason to prefer Reno today. Migrate to CUBIC or BBR.

The most performant CDN architectures are hybrid: BBR at the edge facing users, CUBIC (or BBR) in the middle mile based on measured loss and buffer depth. A/B test with real user monitoring (RUM) — compare Speed Index, LCP, and rebuffer rate, not just iperf.

Reviewed by the FlyFish NWA Performance Team. For implementation help, explore our guides on edge optimization and global delivery architecture.