CDN Network Automation with Ansible and Nornir: Infrastructure as Code for Networking

It was 2:17 AM on a Saturday when our config push to 40 edge PoPs went sideways. A junior engineer had pasted a slightly wrong BGP community string into half our cache nodes, and suddenly traffic that should've stayed local was hairpinning through our origin shield. We spent the next six hours logging into routers one by one over SSH, fixing typos by hand. That's the night I decided we were done doing networking like it was 2005. If you're still touching boxes manually, you can't scale a modern CDN.

Why Infrastructure as Code Is Non-Negotiable for CDN Scale

When you've got 50 nodes, you can get away with a wiki page and some copy-paste. When you've got 500 or 2,000 nodes spread across six continents, that approach collapses. Every manual change is a chance for drift, where one PoP ends up with a subtly different QoS policy or ACL than the rest. I've seen drift cause cache-fill storms that melted a transit link because one site didn't get the updated rate-limiter. With infrastructure as code, your entire network state lives in Git. You review it, you test it, you version it.

For CDN teams, this isn't just about convenience — it's about predictability. Our edge configs handle ECMP Routing in CDN Networks: Equal-Cost Multi-Path Traffic Distribution hashes, Anycast advertisements, and TLS cert mappings that must be identical everywhere. We treat our Junos and IOS-XR configs the same way our developers treat application code. A pull request gets linted, rendered in a lab, and validated before it ever touches production. That discipline is what HTTP Working Group Specifications expect for consistent behavior, and it's what our customers pay for. You can't promise 99.99% hit ratio if your PoPs don't agree on what they're supposed to do.

# group_vars/cdn_edge.yml - Single source of truth for all edge caches
cdn_edge_config:
  anycast_prefix: "192.0.2.0/24"
  bgp_communities:
    - "65000:100"  # edge:cache
    - "65000:200"  # region:us-east
  ecmp_hash_fields: [src_ip, dst_ip, src_port, dst_port]
  cache_fill_rate_limit_mbps: 5000

# In your playbook, this single variable drives 300+ devices
# Change it once, test once, deploy everywhere.

Ansible Networking: Our Workhorse for Declarative Edge Automation

I've used Ansible to manage CDN infrastructure for the last seven years, and I still reach for it first when the task is declarative. Ansible's strength is that you describe what the device should look like, not how to get there. You don't write "enter these 20 CLI commands" — you say "this interface should have this description and this MTU," and the ios_config or junos_config module figures out the diff. For teams that aren't full-time Python developers, that's a huge win. Your network engineers can write YAML without needing to become software engineers overnight.

Where Ansible really shines for us is day-zero provisioning and compliance. We have a role called cdn_edge_base that sets up everything from NTP and SNMP to our standard MPLS in CDN Backbone Networks: Label Switching for Predictable Performance baseline. When we bring up a new PoP in, say, Johannesburg, we just add its inventory entry and run the site playbook. In about 12 minutes, a bare-metal switch goes from zero to a fully peered, monitored CDN cache node. And on a nightly cron, we run the same playbooks in check mode to catch anyone who made a manual "quick fix" — the playbook reports drift and we revert it automatically. That loop has saved us countless times.

---
- name: Configure CDN Edge Routers
  hosts: cdn_edge
  gather_facts: no
  vars_files:
    - group_vars/cdn_edge.yml

  tasks:
    - name: Ensure BGP Anycast is advertised
      junos_config:
        lines:
          - set policy-options policy-statement ANYCAST term 1 from route-filter {{ anycast_prefix }} exact
          - set policy-options policy-statement ANYCAST term 1 then community add {{ bgp_communities | join(' ') }}
          - set policy-options policy-statement ANYCAST term 1 then accept
        comment: "Managed by Ansible - CDN anycast policy"

    - name: Push ECMP hash configuration
      ios_config:
        lines:
          - ip cef load-sharing algorithm include-ports source destination
        save_when: modified

Nornir: Python-Native Control When Ansible Hits Its Limits

Don't get me wrong, I love Ansible, but it wasn't built for speed at CDN scale. Ansible's agentless SSH model runs tasks serially or with limited forks, and for 2,000 devices that means your global purge or emergency ACL push can take 45 minutes. Nornir flips that model. It's pure Python, it runs threaded by default, and it gives you direct access to Netmiko and NAPALM without the abstraction layer. When we needed to push a new WAF rule to block a zero-day and couldn't wait an hour, we rewrote that job in Nornir and cut the rollout to under 4 minutes.

The other reason we've adopted Nornir is flexibility for complex logic. With Ansible, if you need to do something like "query our PeeringDB, calculate the optimal Segment Routing for CDN Traffic Engineering: Source-Routed Path Selection policy per PoP, and then push a different SR-TE color to each region," you're fighting with Jinja filters and custom modules. In Nornir, it's just Python. You write a function, you loop, you use libraries like requests and ipaddress natively. Our team follows patterns documented in web.dev Articles for handling concurrent data fetching, which translates surprisingly well to network inventory processing. You're not learning a DSL — you're just writing code your developers already understand.

Putting It Together: A Hybrid Pipeline for 500+ Edge Nodes

Here's where theory meets reality. We don't choose between Ansible and Nornir — we use both, and we've learned the hard way where each one shines. Ansible owns the declarative baseline: OS versions, base BGP peering, ACLs, and interface descriptions across the entire fleet. Nornir handles the surgical, state-aware work: selective cache purges, live traffic draining, and anything where you need custom Python logic mid-run. In our production setup, this hybrid approach cut our full edge rollout time from 4.5 hours to under 38 minutes for 650 PoPs, with a 95% reduction in manual touchpoints.

The trick is letting Ansible do the inventory and idempotency heavy lifting, then handing off to Nornir for orchestration that Ansible simply can't do cleanly. We keep a single source of truth — NetBox backed by Git — and both tools read from it. Ansible runs nightly to enforce desired state, while Nornir is triggered by CI/CD for change-driven events. For example, when we roll a new rate-limiting policy or spin up Network Function Virtualization in CDN Infrastructure: Virtual Appliances at Scale instances like virtual WAFs and load balancers, Ansible ensures the hypervisor and vSwitch config is correct, but Nornir verifies service health, shifts 5% of traffic, and only then promotes the change fleet-wide. You don't want Ansible blindly templating 600 virtual appliances without that feedback loop.

This setup also plays nicely with our underlay. Whether you're running a partial mesh between super-PoPs or a hierarchical regional model — which we break down in our guide to CDN Backbone Topology Design: Ring, Mesh, and Hierarchical Architectures — your automation has to understand topology. Our inventory groups encode that. A single `group_vars` change to a backbone role propagates correctly without us rewriting 200 host files. Here's a simplified version of how we chain them together in a GitLab CI job that actually runs in production:

# .gitlab-ci.yml - Hybrid CDN Deploy Pipeline
# Stage 1: Ansible enforces baseline (idempotent)
ansible-playbook -i netbox_inventory.yml site.yml --limit edge:datacenter:eu-west --diff --check
ansible-playbook -i netbox_inventory.yml site.yml --limit edge:datacenter:eu-west

# Stage 2: Nornir does state-aware canary + validation
python nornir_orchestrator.py --target-group eu-west --canary 5 --validate

# nornir_orchestrator.py (excerpt)
from nornir import InitNornir
from nornir_napalm.plugins.tasks import napalm_get, napalm_configure
from nornir_utils.plugins.functions import print_result

nr = InitNornir(config_file="config.yaml")
# Filter to canary nodes first
canary = nr.filter(role="edge_cache", region="eu-west")[:5]

result = canary.run(
    task=napalm_configure,
    configuration="set firewall filter RATE-LIMIT term 1 policer 500m",
    dry_run=False
)
# Custom Python: check BGP sessions + HTTP 200s before proceeding
health = canary.run(task=napalm_get, getters=["bgp_neighbors"])
if all(n["is_up"] for r in health.values() for n in r.result.values()):
    print("Canary healthy, promoting to full fleet...")
    nr.filter(role="edge_cache", region="eu-west").run(task=napalm_configure, configuration="...")
else:
    raise Exception("Canary failed - aborting rollout in 12 seconds")

Advanced Patterns: Drift Detection, Canaries, and Surviving a Bad Push

If you're not doing drift detection, you don't have IaC — you have a suggestion. We run Ansible in check mode every 6 hours against the entire fleet and compare the live config (pulled via `napalm_get: getters=["config"]`) against what's in Git. Last quarter alone, we caught 47 manual hotfixes that never got committed — mostly emergency ACL tweaks during DDoS events. Without that loop, those changes get nuked on the next deploy and you're suddenly exposed again. We feed the diff into Slack and auto-create a Jira ticket if drift exceeds 15 lines. It’s noisy for the first two weeks, then it goes quiet, which is exactly what you want.

Canary deployments are non-negotiable when you're touching anything that touches traffic. Don't push to 600 nodes at once just because you can. Our rule: 5 nodes for 10 minutes, then 10% of a region for 30 minutes, then full region. Nornir makes this trivial because you can embed real health checks — we're not just checking if the config applied, we're checking if the cache hit ratio stayed above 92%, if origin offload didn't drop, and if p95 TTFB in that PoP stayed under 85ms. If any of those regress by more than 5%, the pipeline auto-rolls back using the previous Git-tagged config. We had a bad QoS policer push in March that would have blackholed 22% of our EU traffic — the canary caught it on node 3 because origin fetch latency spiked to 1.2s. Rolled back in 19 seconds, no customer impact.

And when you do need to troubleshoot, stop trying to debug Ansible's 400-line error output. Nornir's Python-native failure objects are a lifesaver. You get structured exceptions per host, which you can log to Elasticsearch and alert on. The other pattern that saved us is integrating with your Software-Defined Networking for CDN: Programmable Traffic Control controller. Instead of pushing static ACLs to every edge, we now have Nornir call the SDN controller's API to programmatically drain traffic via BGP community (like 64999:0) *before* we touch the node. That 3-second API call prevents the 2-3% error rate you'd otherwise see during a rolling restart. It’s the difference between customers noticing and customers not noticing.

Frequently Asked Questions