Back in 2019 I watched a 12 MB OTA patch take down our origin during a staged rollout in Southeast Asia. We’d pushed a single bundle to S3 and let clients poll it directly, and within twenty minutes our error rate hit 40% as thousands of devices retried at once. That night taught me that mobile app bundles aren’t just files — they’re a distribution problem that needs edge logic, versioning, and careful cache control. If you’re shipping React Native, Flutter, or native dynamic features, your CDN strategy decides whether updates feel instant or broken.
Why Your Origin Can't Handle Bundle Rollouts Alone
When I started working with OTA systems, most teams treated the update bundle like any other static asset. Upload it, set a long max-age, and call it done. That approach falls apart fast. Mobile clients don’t update uniformly — they wake up on different OS versions, on spotty connections, and after days offline. If your CDN cache key is just the URL, you’ll end up serving stale bundles to users who should get a patched version, or worse, you’ll bust the cache for everyone on every release and hammer your origin.
I've seen this exact failure with a client shipping weekly JavaScript bundles for a React Native app. They versioned bundles as /ota/bundle.js without a content hash and relied on no-cache to force freshness. The result was a 28% cache hit ratio and origin bandwidth that spiked 6x on release days. We fixed it by moving to immutable, content-addressed URLs like /ota/bundle.8f3a1c9.js with a manifest file that points to the current version. The manifest itself is short-lived at the edge (30-60 seconds), while the bundles are cached for a year. This pattern is covered well in the HTTP Working Group Specifications (https://httpwg.org/specs/) on Cache-Control and validation, and it aligns with the caching guidance in web.dev Articles (https://web.dev/articles/) for immutable assets. For the client SDK side, we followed the pattern described in Mobile CDN SDK Integration Guide: Adding Edge Acceleration to iOS and Android Apps to make sure the app respected those headers instead of adding its own aggressive local cache.
# Edge configuration for OTA bundles - immutable assets + short-lived manifest
# Manifest - controls rollout, never cache long
location /ota/manifest.json {
proxy_pass http://origin;
add_header Cache-Control "public, max-age=45, stale-while-revalidate=30";
add_header CDN-Cache-Control "max-age=45";
add_header Vary "X-App-Version, X-Platform";
}
# Versioned bundle - immutable, cache forever at edge
location ~* /ota/bundle\.[a-f0-9]{7,}\.js$ {
proxy_pass http://origin;
add_header Cache-Control "public, max-age=31536000, immutable";
add_header Cache-Tag "ota-bundle";
# Serve stale on origin failure - prevents failed updates
proxy_cache_use_stale error timeout http_500 http_502 http_503;
}
Designing Differential Updates That Actually Work at the Edge
Differential updates are where you save the most bandwidth, but they’re also where CDN setups get tricky. Instead of shipping a full 8-15 MB bundle, you ship a 150-400 KB patch from version N to N+1. I’ve used bsdiff and more recently Google’s Courgette-style deltas for Hermes bytecode, and the savings are real — often 85-92% smaller payloads. The catch is that your CDN must be able to cache and serve many patch combinations without exploding your cache cardinality. If you support the last 20 versions, that’s 20 different patches for every new release.
The strategy that’s held up best for me is to keep patches as discrete, immutable objects keyed by fromVersion-toVersion, and let the client decide which one it needs. The manifest advertises both the full bundle hash and the available patch path: {"currentVersion": "2.4.1", "fullBundle": "/ota/bundle.a1b2c3d.js", "patch": {"from": "2.4.0", "url": "/ota/patch.2.4.0-2.4.1.bin", "hash": "sha256:..."}}. The edge caches each patch forever, just like a full bundle. We don’t try to generate patches on the fly at the edge — I tried that once with an edge worker and the CPU time killed our TTFB. Pre-building at CI time and purging by tag on rollback is far more predictable.
Client-side logic matters just as much. Your app needs a reliable disk cache layer that can verify the patch, apply it offline, and fall back to the full bundle if the hash doesn’t match. We structure that with a three-tier approach where the network cache at the CDN is the first check, then local disk, then memory. If you haven’t set this up, read Mobile App CDN Caching Strategies: Disk Cache, Memory Cache, and Network Cache Layers before you tune your edge TTLs. And when you’re building for tough networks, pairing OTA with Offline-First Architecture with CDN: Building Apps That Work Without Connectivity prevents users from getting stuck on a half-applied update when they lose signal mid-download.
{
"appId": "com.example.app",
"platform": "android",
"currentVersion": "2.4.1",
"minSupportedVersion": "2.2.0",
"rollout": {
"percentage": 25,
"seed": "stable-hash-by-device-id"
},
"artifacts": {
"full": {
"url": "https://cdn.example.com/ota/bundle.a1b2c3d.js",
"size": 8423120,
"sha256": "e3b0c44298fc1c149afbf4c8996fb924..."
},
"patch": {
"fromVersion": "2.4.0",
"url": "https://cdn.example.com/ota/patch.2.4.0-2.4.1.bin",
"size": 312480,
"sha256": "7f83b1657ff1fc53b92dc18148a1d65..."
}
}
}
How We Benchmark OTA Delivery Before a Global Rollout
I don’t push any OTA to 100% without a canary and real edge metrics. Synthetic tests lie, especially for mobile. What I monitor is download success rate by region and network type, median and p95 time to first byte for the manifest and the bundle, and how often clients fall back from patch to full download. On a recent rollout for a shopping app with about 4M MAU, our canary to 5% of users in India and Brazil showed a p95 manifest fetch of 180ms from the edge versus 1.2s from origin, which told us our 45-second edge TTL was working. When the patch failure rate crossed 2%, we knew the delta was corrupted for a specific Hermes version and we rolled back by switching the manifest pointer — no purge needed.
You should run these checks at the CDN level, not just in-app analytics. Enable detailed access logs with cache status (HIT, MISS, STALE), edge POP, and client hints. I also set up a separate health check for /ota/manifest.json that asserts Cache-Control and ETag are present, because a single misconfigured header can quietly turn your edge into a pass-through. We alert if cache hit ratio for versioned bundles drops below 95% or if origin requests for /ota/* spike more than 15% over baseline. Those thresholds have saved us twice when a CI job forgot to add the content hash and reused a URL.
| Metric | Origin-Only Delivery | CDN-Accelerated OTA | Target Threshold |
|---|---|---|---|
| Manifest p95 Latency (Global) | 980 ms - 1,400 ms | 85 ms - 190 ms | < 200 ms |
| Versioned Bundle Cache Hit Ratio | 0% (no edge) | 96.8% - 98.5% | > 95% |
| Average Update Payload (2.4.0 → 2.4.1) | 8.4 MB (full only) | 312 KB (patch), 8.4 MB fallback | < 500 KB for patch |
| Update Success Rate (Patch Apply) | 92.1% | 98.7% | > 98% |
| Origin Bandwidth on Release Day | ~3.2 TB | ~180 GB | < 10% of origin-only |
Putting OTA Deltas Into Production: Edge Logic That Ships
Knowing differentials work in a lab is one thing - making them reliable for 10 million devices on spotty 4G is another. In production, you don't want your origin deciding which patch to serve. Push that logic to the edge. We version every bundle as an immutable artifact (e.g., app-bucket/v4123/full.bundle and app-bucket/v4123/delta-from-v4122.patch) and let an Edge Worker handle the negotiation. The client sends X-Bundle-Version and X-Platform headers, and the worker either serves the 40-180KB delta with a 95%+ cache hit ratio or falls back to the full 2.8MB bundle. No origin compute on the hot path.
That fallback is critical. If a user jumped two versions or the delta cache misses, you can't 404 them. We've learned to set a hard rule: if generation gap > 2, serve full bundle. Deltas beyond that get too large anyway - our tests showed a v4120 to v4123 patch was 1.1MB vs. 2.8MB full, saving only 60% but doubling decompression failures on low-end Android devices. It's cheaper to eat the extra egress than handle support tickets. We also tie bundle delivery to Adaptive Quality Delivery for Mobile: Network-Aware Content Optimization - when the edge detects a constrained connection via throughput hints, it throttles delta delivery to 1.5Mbps and adds longer stale-while-revalidate to avoid retry storms that killed us during our 2023 rollout in India.
Cache control is where most teams mess this up. Don't use short TTLs on immutable bundles. Set them as immutable and forever-cacheable, and version the URL instead:
// Cloudflare Worker / Fastly Compute@Edge - OTA Negotiation
export async function handleOTA(request, env) {
const clientVer = request.headers.get('x-bundle-version') || 'v0';
const targetVer = 'v4123'; // from KV / Config Store
const deltaKey = `/bundles/${targetVer}/delta-from-${clientVer}.patch`;
const fullKey = `/bundles/${targetVer}/full.bundle`;
// Try delta first - 87% of active users are n-1
if (clientVer !== targetVer) {
let delta = await caches.default.match(deltaKey);
if (!delta) {
delta = await fetch(new Request(deltaKey, request));
// Cache deltas for 30 days at edge, 7 days in browser
delta.headers.set('Cache-Control', 'public, max-age=604800, s-maxage=2592000, immutable');
delta.headers.set('CDN-Cache-Control', 'max-age=2592000');
event.waitUntil(caches.default.put(deltaKey, delta.clone()));
}
if (delta.ok) return delta;
}
// Fallback to full bundle - always cached
return fetch(fullKey, { cf: { cacheTtl: 31536000, cacheEverything: true } });
}
Advanced Patterns: Staggered Rollouts, Instant Rollbacks, and When Deltas Fail
Never push a bundle to 100% at once - even with a perfect CDN. We use edge-based traffic splitting for phased rollouts. The Worker reads a rollout config from KV (updated in < 5 seconds globally) that says v4123: 10% canary. It hashes deviceId and if the user isn't in the canary, it continues serving v4122 from cache. No redeploys, no origin changes. We ramp 1% -> 10% -> 50% -> 100% over 48 hours, watching crash rates and delta apply failures. During our last major React Native OTA, this caught a Hermes bytecode regression at 3% that would've bricked 400k sessions. Pair this with proper Mobile Connection Detection and Switching: Adapting CDN Behavior to Network Type so your canary isn't biased toward Wi-Fi users - we force 30% of canary traffic to include 3G/4G emulated clients.
Rollbacks have to be instant. If v4123 is bad, you don't delete it - you flip the KV pointer back to v4122. Because bundles are immutable and cached forever, v4122 is still warm at 290+ PoPs with a 98.7% hit ratio. Purging is your enemy here; a global purge takes 150ms-2s to propagate and then you thundering herd your origin with 800k requests/sec. Instead, we version and switch pointers. The only thing we purge is the small JSON manifest (/ota-manifest.json) that the app polls every 6 hours, set with s-maxage=300, stale-while-revalidate=86400. That manifest is what tells clients what the "current" version is.
When deltas fail in the wild, it's almost always one of three things: 1) Client reported wrong base version due to interrupted apply, 2) Brotli vs. gzip mismatch on patch decompression (we saw 12% failure when we double-compressed), or 3) Edge cache fragmentation from varying Accept-Encoding. Fix it by normalizing encoding at the edge - serve only brotli-11 for deltas, gunzip at edge if client doesn't support it - and by logging x-cache-status and x-bundle-version in your edge logs. We also use Mobile CDN Prefetch Strategies: Predicting and Preloading Content Before Users Need It to warm the next delta at the edge 30 minutes before the manifest flips, so the first canary user never sees an origin fetch. If apply failures exceed 0.8% in any region, auto-fallback to full bundles for that region - don't be a hero trying to save bytes.
Frequently Asked Questions
Should we generate deltas on-the-fly at the edge or pre-build them?
Always pre-build. On-the-fly bsdiff at the edge will melt your CPU allocation and add 400-900ms latency per request. We pre-generate deltas for n-1 and n-2 only during CI (takes ~8 seconds for a 3MB bundle) and upload them as static objects. Edge just serves them. That's how we keep p95 TTFB under 45ms globally.
What cache hit ratio should we target for OTA bundles?
You should be above 96% at the edge and 92% at the shield. If you're seeing 85% or lower, your URLs aren't immutable or you're varying cache on headers you shouldn't. We strip everything except Accept-Encoding for bundle keys. Deltas will naturally have lower hit ratios (80-88%) because of version fragmentation, but full bundles should be near 99% after 24 hours.
How do we handle users who are offline for weeks and 10 versions behind?
Don't try to chain 10 deltas. It's brittle and slower than just sending the full bundle. Our edge logic checks version distance - if gap > 2, it immediately returns the full bundle with a X-OTA-Strategy: full-fallback header for observability. One 2.8MB download is better than ten failed 150KB patches and a corrupted install. We also set the full bundle to prefetch on Wi-Fi if the app hasn't updated in 7 days.
Do OTA bundles need different CDN settings than images or video?
Yes. Turn off image optimization, tiered caching is mandatory, and use Origin Shield in a central region to collapse requests. We also set origin timeout to 10s (not 30s) and enable 0-RTT for OTA manifest checks because apps poll frequently. Most importantly, use separate cache keys and rate limits - we once had video traffic evict OTA bundles from edge cache during a live event. Isolate them with a dedicated hostname or cache namespace.