Origin Push CDN Workflows: Preloading Content Before Users Request It

I've sat through too many major launches where the first wave of users in a new region all hit the edge at once and triggered a thundering herd back to origin. I remember one global trailer drop where we watched origin p95 latency jump from 45ms to over 2.1 seconds in under a minute because every PoP was pulling the same 80MB file simultaneously. That's the exact problem origin push solves — you don't wait for the request to arrive, you place the content on the edge before anyone asks for it.

Why Pull Alone Can't Handle Predictable Surges

Most CDNs default to a pull model and it works beautifully for long-tail content. A user requests /videos/trailer-v2.mp4, the edge misses, it fetches from origin, caches, and the next user gets a hit. It's simple and keeps storage efficient. But when you know a million people are about to request the same asset within a ten-minute window — a game patch, a product launch video, a news homepage takeover — that first-request penalty multiplies across every PoP. You end up with dozens of identical origin fetches, wasted bandwidth, and a terrible experience for the users who happen to be first.

I've run the numbers on this more times than I can count. With a 60-PoP footprint and a pull-only workflow, a single 50MB asset requested globally at launch will generate roughly 3GB of redundant origin egress just to fill caches. It also puts brutal pressure on your origin connection layer. If your Origin Connection Pooling Optimization: Reducing TCP Overhead at CDN Scale isn't dialed in perfectly, you'll see connection queueing and TLS handshake overhead stack up right when you can least afford it. Push flips that model. You publish once, and the origin actively distributes the object to the edges you've targeted.

That distinction matters because not all CDNs implement push the same way. Some offer a true FTP/S3-style storage push where the file lives on edge disks persistently until you delete or expire it. Others use a pre-warming API that still relies on pull under the hood — the control plane just triggers a pull on your behalf. The latter is often called proactive caching, and while it's better than nothing, it doesn't give you the same guarantee. For guidance on how cache directives interact with these workflows, the HTTP Working Group Specifications remain the definitive reference. Here's how I typically distinguish the two when auditing a setup:

# Checking if content is truly pushed vs pre-warmed (pull)
# True push: object exists on edge storage without origin validation

curl -I -H "Pragma: cdn-debug" https://cdn.example.com/assets/launch-v2.zip

# Response for true push (no origin trip)
HTTP/2 200
x-cache: HIT
x-cache-source: EDGE_STORAGE  # Served directly from pushed storage
age: 86400
cache-control: public, max-age=31536000, immutable

# Response for pre-warm (still requires revalidation)
HTTP/2 200
x-cache: HIT
x-cache-source: ORIGIN_PULL_REVALIDATED
age: 120
cache-control: public, max-age=3600

Building a Push Pipeline That Won't Fail at 2 AM

A reliable push workflow isn't just a single API call. In production, it's a pipeline: CMS publish -> origin packaging -> distribution service -> edge storage -> verification. I've seen teams try to push straight from their CMS and wonder why it times out. Your origin shouldn't be holding open hundreds of HTTP connections to PoPs while also trying to serve user traffic. We always put a dedicated distribution layer in front — usually a small service that takes the publish event, chunks the asset if needed, and handles retries and checksum validation per PoP.

That distribution service needs to be defensive. Origins go down, edges get drained for maintenance, and networks blip. If you don't build retry logic with jitter and a clear budget, a single failed push can cascade into a partial rollout where half your users get the new file and half get a 404. I've learned to treat push operations like any other critical origin interaction and apply the same patterns we use for fetch resilience. The principles in Origin Retry Strategies for CDN Resilience: Backoff, Jitter, and Budget Limits apply directly here, and your Origin Health Monitoring Patterns: Active Probes, Passive Checks, and Hybrid Approaches should cover the push endpoints themselves, not just user-facing origin.

Here's a simplified version of the Python worker we use to push content after a CMS publish. It handles chunked upload, per-PoP verification, and respects the Cache-Control headers we want on the edge. We base our header strategy on recommendations from web.dev Articles for long-lived immutable assets, so we don't accidentally push something that'll be revalidated every few seconds.

import hashlib
import requests
from tenacity import retry, wait_exponential_jitter, stop_after_attempt

CDN_PUSH_API = "https://api.cdn-provider.com/v1/push"
EDGE_POPS = ["lax", "iad", "fra", "sin", "nrt", "gru"]

def push_asset(file_path, cdn_path, ttl=31536000):
    with open(file_path, "rb") as f:
        content = f.read()
        checksum = hashlib.sha256(content).hexdigest()

    headers = {
        "Authorization": "Bearer $CDN_API_TOKEN",
        "Content-SHA256": checksum,
        "Cache-Control": f"public, max-age={ttl}, immutable",
        "X-Purge-Previous": "true"  # Remove stale version on push
    }

    for pop in EDGE_POPS:
        _push_to_pop(pop, cdn_path, content, headers, checksum)

@retry(wait=wait_exponential_jitter(initial=1, max=10),
       stop=stop_after_attempt(5))
def _push_to_pop(pop, cdn_path, content, headers, checksum):
    url = f"{CDN_PUSH_API}/{pop}/objects{cdn_path}"
    resp = requests.put(url, data=content, headers=headers, timeout=30)
    resp.raise_for_status()
    # Verify distribution succeeded
    verify = requests.head(f"https://{pop}.cdn.example.com{cdn_path}",
                           headers={"Pragma": "cdn-verify-checksum"})
    assert verify.headers.get("x-content-sha256") == checksum
    print(f"[{pop}] pushed {cdn_path} verified {checksum[:8]}")

What Successful Preloading Looks Like in Production

You can't just push and hope. You need to measure whether the effort is actually paying off. I look at three things immediately after a push: distribution time to 95% of PoPs, cache hit ratio in the first hour after launch, and origin offload percentage. If distribution takes longer than five minutes for a 100MB file, your chunking or concurrency needs work. If hit ratio isn't above 95% in that first hour, you didn't push to the right PoPs — or your TTL is too short and edges are evicting early.

Last year we switched a streaming client from on-demand pull to a scheduled push for their weekly episode drops. Before the change, their origin was serving around 40% of requests in the first 30 minutes because cache fills were racing user traffic. After we moved to a push workflow triggered 15 minutes before publish time, we could preload all 60 PoPs and the origin barely registered the launch. The big win wasn't just bandwidth; it was time-to-first-byte for users in distant regions like Johannesburg and Sydney, who went from 300-400ms misses to consistent 20-30ms hits. We scheduled the push to complete well before the embargo lifted, so there was no last-second scramble.

The table below shows the before-and-after from that exact rollout. We're talking about a 45-minute episode file, ~1.2GB, pushed to 60 PoPs globally. These numbers are typical for what a well-tuned proactive caching setup can deliver when you get the workflow right.

Implementing the Push: From API Call to Edge-Verified Cache Fill

Most teams start their origin push journey with a bash loop and a prayer. They've got a list of URLs, they fire off a bunch of curl -X PURGE and curl -X POST calls, and hope the CDN's API doesn't rate-limit them into oblivion at 2:05 AM before a launch. Don't do that. A production-grade push implementation needs idempotency, authentication that won't expire mid-workflow, and explicit cache-control semantics on every object you upload.

Your pipeline should treat the CDN as a storage target, not just a proxy. That means authenticating with scoped tokens — not your master account key — and pushing with a manifest that defines the cache key, TTL, and surrogate keys upfront. We've found that pushing without explicit Surrogate-Key headers is the number one reason preloading fails to pay off. If you push /assets/app-v2.4.1.js with a 7-day TTL but no way to purge it by tag, you're stuck waiting for expiration when v2.4.2 drops an hour later. Always push with a content hash and set your Cache-Control: public, max-age=604800, stale-while-revalidate=86400 explicitly. Don't rely on origin defaults — edges will honor what you send at push time.

The other non-negotiable is verification at the edge. After a push, your workflow must confirm the object isn't just accepted (202 Accepted) but actually retrievable and valid. This means checking the X-Cache-Status or equivalent header and validating the checksum the edge computed. If you're pushing over verified TLS — and you absolutely should be — make sure your automation validates the edge certificate chain. Skipping that step opens you to silent failures where your content lands on a spoofed endpoint. For anything handling signed assets, review your Origin SSL Verification at CDN Edges: Preventing Man-in-the-Middle Attacks setup before you automate pushes at scale.

#!/usr/bin/env python3
import hashlib
import requests
import json

CDN_API = "https://api.cdn-provider.example/v1/push"
TOKEN = "cdn_push_token_with_edge_write_scope"
MANIFEST = "preload-manifest.json"  # [{ "url": "/video/launch-trailer-4k.mp4", "file": "./dist/trailer.mp4" }]

def push_with_verification(manifest_path):
    with open(manifest_path) as f:
        assets = json.load(f)

    headers = {
        "Authorization": f"Bearer {TOKEN}",
        "Content-Type": "application/octet-stream"
    }

    for asset in assets:
        with open(asset["file"], "rb") as fh:
            data = fh.read()
            sha256 = hashlib.sha256(data).hexdigest()

        # Push with explicit cache semantics
        push_headers = {
            **headers,
            "X-Cache-Key": asset["url"],
            "Cache-Control": "public, max-age=86400",
            "Surrogate-Key": asset.get("tag", "preload-batch-042"),
            "Content-SHA256": sha256
        }

        resp = requests.put(
            f"{CDN_API}{asset['url']}",
            data=data,
            headers=push_headers,
            timeout=(3.05, 30)  # connect=3.05s, read=30s - tune per your origin
        )
        resp.raise_for_status()

        # Verify edge has it and checksum matches
        verify = requests.head(f"https://edge.example.com{asset['url']}", timeout=5)
        assert verify.headers.get("X-Cache-Status") in ("HIT", "PUSHED")
        assert verify.headers.get("ETag").strip('"') == sha256[:32]
        print(f"Pushed {asset['url']} -> {verify.headers.get('X-Cache-Status')} verified")

if __name__ == "__main__":
    push_with_verification(MANIFEST)

Advanced Patterns: Staggering, Hedging, and Debugging When Push Goes Sideways

Pushing 50 assets for a product launch is easy. Pushing 75,000 segmented video chunks, images, and API responses across 45 PoPs without melting your origin or getting throttled is where it gets interesting. Never burst-push everything at once. The most reliable pattern we've used in production is staggered regional preloading with jitter. Instead of firing 10,000 PUTs concurrently, queue them at 150-300 requests per second per region, with 50-100ms of random jitter. That keeps you under the typical 500 req/s push API rate limit and prevents your origin's disk I/O from spiking to 100%. For global events, push to your tier-1 PoPs (Ashburn, Frankfurt, Singapore, São Paulo) first, then let tier-2 edges pull from tier-1 via shield — it cuts your origin egress by 60-70%.

When a push fails, it'll fail quietly. Your API returns 202, but the edge returns a TCP timeout or a 412 checksum mismatch three seconds later. We've seen two repeat offenders: mismatched timeouts and unhedged origin reads. If your push client times out at 10 seconds but the edge needs 14 seconds to write a 250MB file to SSD, you'll retry and double-push, wasting egress. Tune your timeouts deliberately — our rule is connect timeout at 3.05s, read timeout at 60s for large objects, and a total timeout of 90s. Check your Origin Timeout Configuration Guide: Connect, Read, and Total Timeout Tuning and log origin_response_time on every push attempt. And if you're preloading latency-sensitive data like personalization JSON that must be available in <50ms, don't rely on a single push path. Using Origin Request Hedging: Sending Parallel Requests to Cut Tail Latency — sending the same push validation request to two edges and taking the first success — has saved us from 4-5 second tail latencies during shield congestion.

For troubleshooting, stop looking at CDN dashboards and start tailing validation logs. Enable edge push logs and filter for push_status != 'committed'. The fields you care about are push_latency_ms, checksum_verified, and cache_write_status. If you see a spike in NO_CACHE_WRITE with 200 OKs, your Vary header is fragmenting cache keys — a push to /api/config with Vary: Accept-Encoding, Cookie will create 4-6 variants, and your preload only filled one. Another classic: you pushed with Cache-Control: max-age=3600 but your edge is configured to honor origin no-store from a previous pull — your push gets overwritten in 12 seconds. Fix it by setting Cache-Control at push time and locking it with Surrogate-Control or edge rules.

Frequently Asked Questions