Segment Routing for CDN Traffic Engineering: Source-Routed Path Selection

Back in 2019 I watched a single fiber cut between Marseille and Milan push 40% of our southern European cache fill traffic onto a transit link we hadn't intended to use — for 11 minutes our origin shield collapsed under retransmits. That incident taught me that shortest-path IGP alone can't protect a CDN. Segment routing gave us a way to define the exact path from edge to origin, per traffic class, without signaling a mesh of RSVP tunnels across every router.

Why Source Routing Finally Solved CDN Traffic Engineering at Scale

For years we tried to engineer CDN backbones with RSVP-TE. It worked, technically. But operationally it was a nightmare. Each explicit path needed state on every midpoint router, which meant thousands of LSPs to maintain across a Tier-1 footprint. When we added a new PoP in Warsaw, we'd spend days pruning and re-optimizing tunnels. I've seen midnight maintenance windows blow up because one headend router ran out of RSVP state and silently refused new paths. That's not sustainable when you're moving 200+ Tbps of egress during a live sports event.

Segment routing flips the model. The ingress node imposes the path as a stack of segments — basically MPLS labels or IPv6 SIDs — and midpoints just forward based on the top segment. There's no per-flow state in the core. This decouples path complexity from network size, which is exactly what a CDN needs. We can now compute diverse, latency-optimized or cost-optimized paths centrally and push them to edge routers via PCEP or BGP SR-TE, without touching the core. If you're still running traditional MPLS in CDN Backbone Networks: Label Switching for Predictable Performance, this is the cleanest migration path — same label forwarding, but controller-driven instead of LDP-signaled.

Compared to ECMP Routing in CDN Networks: Equal-Cost Multi-Path Traffic Distribution, which hashes flows across equal-cost paths, segment routing lets us be opinionated. ECMP is great for uniform capacity, but it can't steer 4K video segments away from a congested yet equal-cost link. With SR-TE we define candidate paths with explicit constraints: avoid this SRLG, stay under 45ms, prefer private backbone over transit. The controller in our case is a PCE that ingests telemetry, latency probes, and link utilization every 30 seconds. For advanced transport behavior, the HTTP Working Group Specifications remain useful for understanding how end-to-end prioritization interacts with underlay path selection.

# Cisco IOS-XR SR-TE policy for cache-fill traffic
segment-routing
 traffic-eng
  policy CACHE-FILL-TO-MILAN
   color 100 end-point ipv4 198.51.100.5
   candidate-paths
    preference 100
     dynamic
      pcep
      metric type latency margin 10
     constraints
      segments
       sid-absolute 16102  # Paris P-router Node-SID
       sid-absolute 16203  # Turin Adj-SID (low-latency lambda)
       sid-absolute 16005  # Milan egress Node-SID
      affinity
       exclude-any red  # avoid links marked for maintenance
      disjoint-path group 1 type link
   candidate-paths
    preference 80
     explicit segment-list LOW-COST-BACKUP
      index 10 sid-absolute 16101
      index 20 sid-absolute 16304

# Verification
show segment-routing traffic-eng policy color 100
show pce ipv4 peer detail | i SR-TE

SR-MPLS vs SRv6: Picking the Data Plane That Won't Haunt You Later

I get asked constantly whether to build on SR-MPLS or jump straight to SRv6. There isn't a single right answer, but I've deployed both in production CDN fabrics and they feel very different. SR-MPLS is the pragmatic choice if you already have an MPLS core. You reuse your label switch infrastructure, your existing QoS policers, and your operations team doesn't need to retrain. The SID is just a label — typically 16000-range Node-SIDs plus Adj-SIDs. We've got PoPs where SR-MPLS has been running for four years without a hardware refresh, simply by upgrading software to support SR-TE.

SRv6 is a different beast. It encodes the segment list in an IPv6 Segment Routing Header (SRH), which means you can run traffic engineering over a pure IPv6 fabric with no MPLS data plane at all. That's attractive for cloud-native PoPs built on white-box switches or SmartNICs, and for integrating with Network Function Virtualization in CDN Infrastructure: Virtual Appliances at Scale where virtual routers struggle with MPLS label stacks. The downside I've hit repeatedly is MTU and overhead. An SRH with 5 segments adds 88 bytes. If your cache servers are pushing 9000-byte jumbo frames for origin fetch, you won't notice. If you're serving small objects and QUIC packets over the backbone, that overhead and potential fragmentation can hurt. Plus not all merchant silicon handles SRH processing at line rate yet — we learned that the hard way on an early Tofino deployment.

My rule of thumb today: if your backbone is MPLS and your hardware is less than six years old, go SR-MPLS. You'll get source-routed path selection with minimal risk and excellent ECMP compatibility underneath. If you're greenfield, IPv6-native, or pushing toward programmable data planes with eBPF and XDP, SRv6 gives you more flexibility for service chaining — for example, steering via a scrubbing center or an L7 firewall via a Service SID. For front-end delivery tuning, I still reference web.dev Articles when correlating last-mile performance with the backbone path changes we make. Don't try to run both in the same domain unless you have a strong controller to abstract the difference.

# Linux + iproute2 SRv6 encapsulation for CDN midgress steering
# Add SRv6 policy: steer 203.0.113.0/24 via Lyon -> Geneva -> Milan

ip -6 route add 203.0.113.0/24 encap seg6 mode encap \
  segs 2001:db8:ff:2::2,2001:db8:ff:3::3,2001:db8:ff:5::5 dev eth0

# Show SRv6 local SIDs and SRH stats
ip -6 route show type local | grep seg6
ip -6 seg6 show

# VPP alternative for high-throughput cache nodes ( ~100G per core )
# vppctl sr policy add bsid 2001:db8:aa::1 \
#   next 2001:db8:ff:2::2 next 2001:db8:ff:5::5 encap
# vppctl sr steer l3 203.0.113.0/24 via bsid 2001:db8:aa::1

How We Steer Real CDN Traffic With SR Policies Without Melting the Network

Policy definition is the easy part. The hard part is deciding *what* to steer and when. We don't put all traffic into SR-TE — that'd be wasteful. About 70% of our eyeball traffic stays on IGP shortest path with ECMP. We use SR policies for three classes: cache fill (origin to edge), inter-PoP replication for live, and high-priority API/video manifest traffic that can't tolerate jitter. Each class gets a color (as in the config above), and BGP routes are colored via extended communities. A route for our origin shield prefix might have color 100, which automatically steers it onto the low-latency policy above. If that policy flaps, it falls back to preference 80 or ultimately to best-effort IGP.

Telemetry is what makes this operational. We run continuous TWAMP and STAMP probes between PoPs and export per-adjacency latency into our PCE. When the PCE sees latency on the primary segment-list breach a threshold, it recomputes and pushes a new candidate path via PCEP — no CLI changes. Convergence is where segment routing shines. With TI-LFA (Topology-Independent Loop-Free Alternate) enabled, local protection happens in under 50ms without waiting for the controller. I can't overstate how much that mattered during the 2022 Frankfurt fiber cut. Our RSVP network would have taken 600-900ms to reconverge; SR with TI-LFA was sub-50ms and viewers didn't see a rebuffer.

Capacity planning changes too. Because paths are explicit, we can model utilization deterministically. Before pushing a major live event policy, we simulate it in our offline PCE using actual demand matrices. That's impossible with ECMP hashing where split ratios are probabilistic. The table below shows numbers we measured during a six-month A/B between legacy RSVP-TE and SR-TE on the same European backbone — same hardware, same fiber, different control plane.

Putting It Together: Building a PCE-Driven SR Policy Pipeline That Actually Scales

Here's where most CDN teams stall: you've got BGP-LS feeding topology, you've picked your data plane, but you still need a controller that can compute and push policies without turning your backbone into a science project. We've learned the hard way — don't build a monolithic PCE. Our current stack splits responsibilities: BGP-LS speakers on two route reflectors per region stream the entire IS-IS topology (including TE metrics, SRLGs, and SID info) to a pair of PCEs, and those PCEs compute constrained paths every 30 seconds. If you're still running Software-Defined Networking for CDN: Programmable Traffic Control with OpenFlow or legacy PCEP baggage, this is a clean break — you're moving to a model where the headend router makes the final decision, not the controller.

We use color-aware steering and it’s the single best feature for CDN traffic engineering. Instead of per-flow states, you tag your service routes with a color (e.g., color 100 = low-latency, color 200 = low-cost transit, color 300 = scrubbed for DDoS). The PCE then instantiates an SR Policy for (color, endpoint) and BGP automatically steers matching traffic. A typical edge PoP in our CDN Backbone Topology Design: Ring, Mesh, and Hierarchical Architectures mesh has only 40-60 active SR Policies at any time, even while carrying 2.8 Tbps — that's 3 orders of magnitude less state than per-flow ECMP hacks we used to run. Don't try to micromanage with hundreds of policies; you'll drown in churn during reconvergence.

! IOS-XR Headend - SR Policy with explicit SID-list pushed via PCEP
segment-routing
 traffic-eng
  policy POLICY-COLOR100-TO-CHI
   color 100 end-point ipv4 192.0.2.5
   candidate-paths
    preference 100
     dynamic
      pcep
      metric
       type latency
       bound 4200 ! 42ms cap for live video rebuffer avoidance
    preference 90
     explicit segment-list SL-BACKUP-LOWCOST
      weight 1
     !
    constraints
     affinity exclude-any RED  ! avoid fiber shares with SRLG RED
!
! BGP steering - anything with color 100 auto-steers
router bgp 65001
 address-family ipv4 unicast
  segment-routing traffic-eng
 !
 neighbor 203.0.113.1
  address-family ipv4 unicast
   extended-community color:100:0.0.0.0
!

Advanced Patterns and Troubleshooting When Segments Go Sideways

Once you're live, you'll hit two gotchas within the first 90 days. First is MTU. With SR-MPLS, we add 4-6 labels (16-24 bytes) — not terrible, but if your CDN PoP Interconnection Design: Building Reliable Edge-to-Core Links still has 1500-byte interfaces from 2018, you'll see silent drops on large cache-fill objects. With SRv6 it's worse: 2 SIDs is already +32 bytes plus the SRH overhead. We standardized on 9000-byte jumbo on all core and DCI links and clamped the SRH to 3 SIDs max via Flex-Algo. Second is ECMP blindness — your latency optimization can accidentally pin traffic to a single path and leave parallel 100G links at 12% while the chosen path is at 88%. Fix it by computing SID-lists that explicitly leverage ECMP-aware Adj-SIDs where you *want* load sharing.

For resilience, stop relying on RSVP FRR and enable TI-LFA — it's the reason we can promise <50ms failover without a single RSVP reservation. In our last fiber cut between Ashburn and Chicago, TI-LFA rerouted 1.1 Tbps of origin fetch in 32ms before the PCE even recomputed. Pair it with Flex-Algo if you run a large CDN: we run Algo 128 for low-latency (optimized on delay metric, not IGP cost) and Algo 129 for low-cost (optimized on TE metric that penalizes expensive transit). Then your SR Policy is just one SID — the Flex-Algo Prefix-SID — instead of a 5-label stack. It's elegant and it'll save you hours of debugging malformed segment lists at 2 a.m.

When it breaks — and it will — don't start with the controller. `show segment-routing traffic-eng policy color 100` on the headend is your ground truth. If the policy is down, check PCEP delegation state first (`show pce peer`), then BGP-LS. Nine times out of ten, it's a stale BGP-LS update that made the PCE think a link still has 40G free when it's actually congested. We added streaming telemetry on SID-list byte counters and per-policy latency probes (TWAMP over the SR Policy every 10s). If probe latency spikes 8-10ms above the computed bound, we automatically de-preference that candidate path. It’s caught 23 brownout events this quarter that SNMP polling would have missed entirely.

Frequently Asked Questions