CDN Tenant Isolation Patterns: Cache Separation, Rate Limits, and Access Controls

Back in 2019 I watched a single SaaS customer take down cache performance for 400 others. They’d deployed a misconfigured purge script that sent wildcard invalidations every 30 seconds, and because we’d keyed our cache only on URL path, every tenant sharing that edge PoP saw their hit ratio collapse from 94% to 31% in under ten minutes. That incident cost us a full post-mortem and a complete rewrite of how we think about CDN Multi-Tenant Architecture: Isolating Customer Traffic While Sharing Infrastructure — you can’t just put tenants on the same infrastructure and hope for the best.

Why Shared Cache Keys Will Poison Your Tenants

I’ve seen teams start with the simplest thing: `cache-key = host + uri`. It works until two tenants serve `/api/config.json` or `/assets/app.js` with completely different contents. Without explicit tenant scoping, you’ve built a cache poisoning primitive. One tenant’s stale or private data gets served to another, and you won’t catch it in staging because you never test with overlapping paths at load.

The fix isn’t just appending a tenant ID. You’ve got to decide where that ID comes from and how it composes with Vary semantics. If you’re doing White-Label CDN Delivery: Custom Branding for SaaS Customer Domains with custom domains, the `Host` header might be distinct (`customer-a.yourapp.com` vs `customer-b.yourapp.com`), but many SaaS platforms route by path prefix (`/t/acme/`) or by an `X-Tenant-ID` header injected at the origin. The HTTP Working Group Specifications are pretty clear on this — caches must respect `Vary` and any surrogate keys you define, but they can’t invent tenant boundaries you didn’t tell them about. I don’t trust `Vary: X-Tenant-ID` alone at the edge; I want it baked into the cache key itself.

We learned to treat cache separation as a namespacing problem. Every object gets a logical partition, and purges, tiered cache fills, and even cache analytics must respect that partition. If your purge API doesn’t require a tenant scope, you haven’t solved tenant isolation — you’ve just hidden it until the next incident. Our current pattern is to hash the tenant identifier into the key and enforce it at both edge and shield layers, so a shield miss doesn’t accidentally promote cross-tenant contamination.

// Cloudflare Worker / Fastly VCL-style cache key with tenant isolation
// Don't trust Host alone when tenants share domains via path routing

function generateCacheKey(request) {
  const tenantId = request.headers.get("x-tenant-id") 
    || extractTenantFromPath(request.url) // e.g., /t/:tenantId/...
    || "public";
  
  // Normalize to avoid key fragmentation
  const url = new URL(request.url);
  const normalizedPath = url.pathname.toLowerCase().replace(/\/$/, "");
  
  // Explicit namespace: tenant -> asset version -> normalized URL
  const version = request.headers.get("x-asset-version") || "v1";
  
  return `tenant:${tenantId}:ver:${version}:${normalizedPath}?${url.searchParams.toString()}`;
}

// Enforce Vary + surrogate keys for purge isolation
// Purge must include tenant scope or be rejected
async function handlePurge(purgeRequest) {
  const { tenantId, surrogateKeys } = await purgeRequest.json();
  if (!tenantId || !surrogateKeys) {
    return new Response("Missing tenant scope", { status: 400 });
  }
  // Only purge keys matching tenant namespace
  await cache.purgeBySurrogateKeys(
    surrogateKeys.map(k => `tenant:${tenantId}:${k}`)
  );
}

Per-Tenant Rate Limits That Don't Melt Your Edge Logic

Global rate limits are useless in multi-tenant SaaS. A 10k rps limit for the whole edge sounds healthy until one tenant’s scraper or buggy retry loop consumes 9.5k of it and starves everyone else. We’ve had to move to hierarchical limits: global, per-PoP, and per-tenant, evaluated in that order. The hardest part isn’t the algorithm — token bucket vs. leaky bucket — it’s doing it without adding 20ms of Redis lookup latency on every request.

I’m not a fan of doing synchronous origin calls for limit checks. What’s worked for us is a two-tier system: local in-memory counters at the edge with a 1-2 second sync to a centralized store. You sacrifice a little burst accuracy (a tenant might slip 5-10% over the limit for a second) but you keep p95 under 5ms. The guidance in web.dev Articles on limiting excess fetches and prioritizing critical requests lines up with this — if you block at the edge cheaply, you protect origin and keep good tenants fast. The other mistake I see? Using IP-based limits for tenant limits. In SaaS, a single tenant may have thousands of IPs behind a corporate NAT, and a single IP (like a shared egress) may host dozens of tenants.

For Custom Domain CDN for SaaS: Automated SSL and DNS for Customer Vanity Domains setups, you’ll also need to rate limit by custom domain *and* by underlying tenant ID, because attackers will try to bypass limits by rotating hostnames. We map every custom domain to its canonical tenant ID at the edge during TLS termination, then apply both: a lenient per-domain limit to absorb normal traffic spikes, and a stricter per-tenant aggregate limit that sums across all their domains. Here’s a simplified version of that logic running as an edge worker before cache lookup:

// Edge rate limiting: hierarchical + tenant-aware
// Uses Durable Object / Redis with local sliding window

const LIMITS = {
  perTenantRps: 500,      // SaaS isolation: hard aggregate
  perDomainRps: 300,      // vanity domain burst allowance
  perIpRps: 50            // abuse floor
};

async function isAllowed(request, env) {
  const tenantId = request.tenantId; // resolved during TLS + Host mapping
  const domain = new URL(request.url).hostname;
  const ip = request.headers.get("cf-connecting-ip");

  // Increment local counters first (0ms)
  const [tenantCount, domainCount, ipCount] = await Promise.all([
    env.LIMITER.increment(`tenant:${tenantId}`, 1000),
    env.LIMITER.increment(`domain:${domain}`, 1000),
    env.LIMITER.increment(`ip:${ip}`, 1000)
  ]);

  if (ipCount > LIMITS.perIpRps) return { allowed: false, reason: "ip" };
  if (domainCount > LIMITS.perDomainRps) return { allowed: false, reason: "domain" };
  if (tenantCount > LIMITS.perTenantRps) return { allowed: false, reason: "tenant" };

  // Async replicate to central store, don't block response
  env.ctx.waitUntil(env.CENTRAL.syncCounters({ tenantId, domain, ip }));

  return { allowed: true };
}

// Return 429 with Retry-After and tenant-scoped headers
function rateLimitResponse(reason) {
  return new Response("Too Many Requests", {
    status: 429,
    headers: {
      "Retry-After": "2",
      "X-RateLimit-Scope": reason,
      "Cache-Control": "no-store"
    }
  });
}

Enforcing Access Controls Without Cross-Tenant Leakage

Cache separation and per-tenant limits mean nothing if your access controls are flat. I audited a platform last year where every tenant’s signed URL used the same HMAC secret, just with a different `tenant` claim in the payload. The verification code checked the signature but never checked that the tenant in the URL matched the tenant owning the object. One customer could guess another’s object path and their valid signature would still pass. That’s not tenant isolation — that’s tenant hope.

We now require three checks on every private asset request: authentication (who are you), authorization (are you allowed for this tenant), and namespace binding (does this token’s tenant match the object’s tenant namespace). I like short-lived, tenant-scoped tokens issued by origin and verified at the edge with a per-tenant public key or secret. Secrets must be per-tenant, rotated independently, and stored in a way that edge nodes can fetch only the keys for tenants they’re currently serving — not a global secrets dump. If an edge node is compromised or misconfigured, the blast radius should be one tenant, not all of them.

The trade-off is operational complexity versus isolation strength. We’ve measured the patterns below across a fleet serving ~8,000 SaaS tenants at ~2.5M rps aggregate. There’s no perfect answer, but the numbers make the decision clearer when you’re pitching this to engineering leadership:

Putting It Together: A Production-Ready Tenant Isolation Worker

Talk is cheap — you'll only get real isolation when every request is tagged, checked, and keyed at the edge before it ever touches cache or origin. We've seen too many teams validate the tenant at the origin and assume the CDN will "just handle it." It won't. If your edge logic doesn't deterministically resolve the tenant from the first byte, you've already opened the door to cache poisoning and meter bleed.

Here's the flow we run in production for SaaS platforms doing 15k-40k RPS per PoP: 1) Extract tenant ID from the subdomain or verified JWT — never from a query string or custom header alone. 2) Validate it against an edge KV store with a 60s TTL, so you're not adding 25-40ms of origin latency on every hit. 3) Build a composite cache key and enforce a per-tenant token bucket *before* you do a cache lookup. That ordering matters — if you check cache first, a burst from one tenant can still thrash your cache hit ratio for everyone else.

This is also where your billing accuracy lives. If you're not tagging every request with a tenant ID at the edge, your usage aggregates are fiction. A solid implementation feeds directly into CDN Usage Tracking for SaaS: Per-Customer Bandwidth and Request Analytics and lets you push real-time counters for API Metering at CDN Edge: Real-Time Usage Tracking for SaaS Billing without reconciling logs 24 hours later. Don't batch this — emit it inline.

// Cloudflare Workers / Fastly Compute@Edge style tenant isolation
export default {
  async fetch(req, env, ctx) {
    const url = new URL(req.url);
    
    // 1. Deterministic tenant resolution - don't trust unauthenticated headers
    const tenantId = getTenantId(req); // from subdomain or verified JWT claim
    if (!tenantId || !await isValidTenant(tenantId, env.TENANT_KV)) {
      return new Response('Unknown tenant', { status: 401 });
    }

    // 2. Per-tenant rate limit check (token bucket in Workers KV / Redis)
    // 1000 req/min per tenant, burst 200 - tune per plan tier
    const rateLimitKey = `rl:${tenantId}:${Math.floor(Date.now()/60000)}`;
    const count = await env.RATE_KV.incr(rateLimitKey, 1);
    if (count === 1) await env.RATE_KV.expire(rateLimitKey, 60);
    if (count > 1000) {
      return new Response('Rate limit exceeded', { 
        status: 429, 
        headers: { 'Retry-After': '10', 'X-Tenant-RateLimit-Remaining': '0' }
      });
    }

    // 3. Composite cache key - THIS is your isolation boundary
    const cacheKey = `${tenantId}::${url.pathname}::${url.searchParams.get('v') || 'latest'}`;
    const cache = caches.default;
    let response = await cache.match(cacheKey);

    if (!response) {
      // Enforce origin access control - add tenant context for origin to re-validate
      const originReq = new Request(req, {
        headers: { ...req.headers, 'X-Tenant-ID': tenantId }
      });
      response = await fetch(originReq);
      // Only cache 200s, and add Vary to prevent downstream confusion
      if (response.ok) {
        response = new Response(response.body, response);
        response.headers.set('Cache-Control', 'public, max-age=300');
        response.headers.set('Vary', 'X-Tenant-ID, Authorization');
        ctx.waitUntil(cache.put(cacheKey, response.clone()));
      }
    }
    
    // Async metering - don't block the response
    ctx.waitUntil(emitMetering(tenantId, req, response));
    return response;
  }
}

function getTenantId(req) {
  const host = new URL(req.url).hostname; // acme.app.example.com
  return host.split('.')[0]; // or decode from verified JWT: jwt.payload.tid
}

Advanced Patterns: Surviving Noisy Neighbors and Silent Cache Leaks

Once you've got basic key separation, the next failure mode is the noisy neighbor that doesn't trip a hard limit but still ruins everyone else's P95. A single tenant running a scraper at 850 RPS sustained won't hit your 1000 req/min limiter, but it'll evict your entire hot set if you're on a shared LRU. For anything over ~500 tenants per edge node, we recommend moving from simple key prefixing to logical cache namespaces or tiered cache with surrogate keys. With Fastly or Cloudflare, use `Cache API` namespaces or `surrogate-key: tenant-123 product-images` so a purge for `tenant-123` nukes 50k objects in under 150ms without touching `tenant-456`.

Troubleshooting cross-tenant leakage is brutal because it’s silent until a customer reports seeing someone else’s data. Don't rely on hit/miss ratios alone. Add a debug header in staging — `X-Cache-Key-Hash` and `X-Tenant-ID` echoed — and log the full composite key for 0.1% of sampled requests. If you see a cache HIT where the logged key's tenant doesn't match the request's tenant, you've got key bleeding. Common culprits? Forgetting to include `Authorization` in the `Vary` header, or normalizing URLs before you prepend the tenant ID. We've also seen teams cache 302 redirects globally — one tenant's login redirect gets served to everyone.

For enterprise tenants bringing their own IdP, isolation gets trickier. You can't just share a single JWT validation key. You'll need per-tenant JWKS endpoints cached at the edge with a 10-minute stale-while-revalidate, and you must fail closed if validation fails. That pattern is the backbone of Enterprise SSO Integration with CDN: SAML, OIDC, and Custom Identity Providers — where each tenant's OIDC discovery document becomes part of your edge config, not an afterthought. And when you do hit rate limits, don't just 429 everyone. Serve stale for GETs for up to 30s while you shed load, and only hard-block mutating methods. Your biggest customers will thank you when a flash sale doesn't take down the whole platform.

Frequently Asked Questions