API Key Rotation Strategies for CDN Gateway Infrastructure

API keys are the most common authentication mechanism for CDN gateway traffic. Every API request that passes through a CDN edge node carries a key that the edge must validate before forwarding the request to the origin. But API keys have a critical operational challenge: they need to be rotated regularly for security, and rotation across a globally distributed CDN network — with hundreds of PoPs and eventual consistency in edge storage — introduces a window where valid requests could be rejected. Getting rotation right is the difference between a secure API gateway and one that trades availability for security.

Why API Keys Need Rotation

API key rotation is not optional. Keys accumulate risk over time. They appear in log files, get cached in developer tools, leak into version control systems, and are shared in chat messages. The longer a key exists, the more likely it has been exposed to unauthorized parties. Regular rotation limits the exposure window — a compromised key that rotates every 90 days has a maximum exposure window of 90 days, while a key that never rotates has an unlimited exposure window.

Compliance frameworks mandate rotation. PCI DSS requires cryptographic key rotation at least annually. SOC 2 expects documented key management procedures including rotation. NIST SP 800-57 provides detailed key lifecycle management guidance that includes periodic replacement. For CDN gateways handling financial, healthcare, or government traffic, rotation is a compliance requirement, not a best practice.

Rotation also serves operational purposes. When an engineer leaves the organization, all keys they had access to should be rotated. When a security incident occurs, affected keys must be rotated immediately. When migrating between CDN providers or upgrading key formats, rotation provides the transition mechanism.

The Dual-Key Acceptance Window

The core challenge of API key rotation at CDN edge is the propagation delay. When a new key is generated and the old key is invalidated, there is a period where some edge PoPs have received the new key data and some have not. During this window, the same valid request might succeed at one PoP and fail at another, depending on whether the PoP has propagated the key change.

The solution is a dual-key acceptance window. Instead of atomically swapping old key for new key, the CDN gateway maintains both keys as valid simultaneously during a transition period. The process follows four phases:

Phase 1: Key Generation

A new API key is generated with the same permissions, rate limits, and scopes as the key being replaced. The new key is stored in edge KV alongside the existing key, marked as "pending" (not yet the primary key). The edge worker is updated to accept both the active key and any pending key associated with the same API consumer.

Phase 2: Dual Acceptance

Both keys are valid. API consumers can use either key, and the CDN edge accepts both. The new key is communicated to the API consumer through a secure channel (admin dashboard, encrypted email, secure API endpoint). The consumer begins migrating their systems to use the new key. During this phase, the CDN gateway monitors the usage ratio between old and new keys to track migration progress.

Phase 3: Primary Swap

Once all or most traffic has migrated to the new key (determined by monitoring), the new key becomes the primary and the old key is marked as "deprecated." The deprecated key still works but the CDN gateway begins returning a deprecation warning header (X-API-Key-Deprecated: true) on responses authenticated with the old key. This signals to any remaining consumers that they need to update.

Phase 4: Revocation

After the deprecation period, the old key is revoked. Requests using the old key receive a 401 Unauthorized response with an error body indicating key expiration. The revoked key is added to a revocation list maintained in edge KV, ensuring that even cached validation results for the old key are rejected.

Key Storage Architecture at CDN Edge

The CDN edge worker needs fast access to API key data for validation. The storage architecture uses three tiers: in-memory cache, edge KV, and origin key management service.

In-Memory Cache

The edge worker maintains an in-memory cache of recently validated keys. The cache uses the key's prefix (first 8 characters) as the lookup key, storing the full key hash, associated metadata (rate limits, scopes, owner ID), and a TTL. The in-memory cache provides sub-millisecond validation for high-frequency keys. The cache is per-worker-instance, not shared across workers or PoPs, so its hit rate depends on request volume and key distribution.

Cache entries have a short TTL (60–300 seconds) to ensure that revoked keys are ejected quickly. For emergency revocations, the edge worker checks a separate revocation list before checking the cache, ensuring revoked keys are rejected even if cached.

Edge KV Storage

The edge KV store holds the authoritative key data for the CDN network. Each API key entry contains the key hash (never the plaintext key), metadata, status (active, pending, deprecated, revoked), creation and expiration timestamps, and rotation history. KV provides consistent global access at 1–5ms latency, with eventual consistency on writes propagating to all PoPs within 60 seconds.

The key structure in KV supports the dual-key pattern: a consumer record contains references to both the active key and any pending/deprecated keys. The edge worker resolves the consumer from any of the associated keys, checking all key statuses to determine whether to accept the request.

Origin Key Management Service

The origin service is the authoritative key store, handling key generation, rotation scheduling, and lifecycle management. The CDN edge only falls back to the origin when a key is not found in KV (a cold key or a very recently generated key). Origin lookups add 50–200ms latency depending on the geographic distance, so they should be rare — well under 1% of total validations.

Automated Rotation Pipelines

Manual key rotation does not scale. A CDN gateway serving hundreds of API consumers needs automated rotation that handles scheduling, notification, monitoring, and revocation without human intervention.

Scheduled Rotation

The rotation pipeline runs on a configurable schedule (30, 60, or 90 days) per consumer. The pipeline generates the new key, stores it in the origin key management service, replicates to edge KV, sends a notification to the consumer (webhook, email, or dashboard alert), and opens the dual-key acceptance window. The pipeline tracks the old key's usage and automatically progresses through the rotation phases as usage shifts to the new key.

Event-Driven Rotation

Certain events trigger immediate rotation outside the regular schedule: a key is detected in a public repository (credential scanning), an anomalous usage pattern suggests compromise (rate limit spikes from unusual IPs), an employee with key access leaves the organization, or a security audit flags a key that has exceeded its maximum age. Event-driven rotations skip the dual-key acceptance window when the event indicates compromise — the old key is revoked immediately and the consumer is notified to adopt the replacement key.

Rotation Monitoring

The pipeline exposes metrics for each rotation: time to complete migration (how long before all traffic uses the new key), failed validations during rotation (requests rejected due to propagation lag), consumers still using deprecated keys (who has not migrated), and rotation frequency by consumer (who is rotating too often or not enough). These metrics feed into operational dashboards and alerting, ensuring rotation health is visible to the operations team.

Key Versioning and Format

A well-designed API key format supports rotation, validation, and operational management. The key should encode enough information for the edge worker to validate efficiently without encoding so much that the key becomes a security liability.

Structured Key Format

A CDN gateway API key typically follows a structured format with four segments: prefix, version, payload, and checksum. The prefix identifies the key type (fly_live_ for production, fly_test_ for sandbox). The version indicates the key generation algorithm and validation rules. The payload is a cryptographically random string (32–48 bytes, base62 encoded). The checksum is a CRC32 or similar fast checksum of the preceding segments, allowing the edge worker to reject malformed keys before any storage lookup.

The structured format provides several operational benefits: the prefix allows quick visual identification in logs without revealing the key, the version supports migration between key formats, and the checksum catches typos and truncation without a round-trip to storage.

Key Hashing

The CDN gateway never stores plaintext API keys in edge KV or the origin database. Keys are hashed using a one-way function (SHA-256 or bcrypt) immediately after generation. The plaintext key is shown to the consumer exactly once (at generation time) and is never stored by the system. Validation compares the hash of the incoming key against the stored hash.

For edge performance, SHA-256 is preferred over bcrypt because it computes in microseconds versus milliseconds. The security tradeoff (bcrypt provides key-stretching that resists brute-force) is acceptable because API keys have high entropy (256+ bits), making brute-force infeasible regardless of the hash function.

Revocation Propagation Across CDN PoPs

When a key is revoked (emergency or scheduled), the revocation must propagate to all edge PoPs as quickly as possible. The propagation mechanism depends on the CDN platform's capabilities.

KV-Based Revocation

The simplest approach writes a revocation entry to edge KV. The entry contains the revoked key hash, revocation timestamp, reason code, and a TTL matching the key's original expiration. Edge workers check the revocation namespace before validating any key. KV propagation is eventual-consistent, typically reaching all PoPs within 60 seconds.

Purge-Based Propagation

For CDN platforms that support cache purge by tag, the API key validation result can be cached with a tag derived from the key hash. When a key is revoked, a purge-by-tag request invalidates all cached validation results for that key across all PoPs. This leverages the CDN's existing cache invalidation infrastructure, which is typically faster than KV propagation (seconds rather than minutes).

Broadcast-Based Propagation

Some CDN platforms support real-time broadcast to all edge workers. A revocation event is published to a broadcast channel, and all running workers receive it immediately. Workers add the revoked key to their in-memory deny list, providing near-instant revocation. The broadcast is supplemented by KV storage to handle workers that were not running during the broadcast (new deployments, PoP restarts).

Rate Limiting During Rotation

API key rotation interacts with rate limiting in non-obvious ways. During the dual-key window, both the old and new key are associated with the same consumer, but rate limit counters are typically keyed by the API key value. If the rate limiter treats old and new keys as separate consumers, the consumer effectively gets double the rate limit during the rotation window.

The solution is to key rate limit counters by consumer ID rather than API key. The edge worker resolves the consumer from the API key, then applies rate limits using the consumer's quota. Both old and new keys resolve to the same consumer, so rate limits are correctly shared. This requires the key-to-consumer mapping to be available in the same edge KV lookup that validates the key — an argument for storing consumer metadata alongside key data rather than in a separate namespace.

Audit Trail and Compliance

Every key lifecycle event — generation, activation, deprecation, revocation — must be logged with a timestamp, actor (who initiated the rotation), reason, and affected consumer. The audit trail serves both operational and compliance purposes.

CDN log pipelines should tag each request with the key version used for authentication. This enables retrospective analysis: if a key is found to be compromised, the audit trail identifies every request authenticated with that key during the exposure window. The tagged logs feed into SIEM systems for security event correlation and into compliance reports for auditor review.

For highly regulated environments, the key management service should maintain a tamper-evident audit log (append-only, cryptographically chained) that provides non-repudiable evidence of key lifecycle events. This goes beyond standard CDN logging and typically requires a dedicated key management service (AWS KMS, HashiCorp Vault, or Azure Key Vault) integrated with the CDN gateway's rotation pipeline.

Multi-Tenant Key Isolation

CDN gateways serving multiple tenants (a SaaS CDN, an API marketplace) must ensure that key rotation for one tenant does not affect others. The isolation model separates key namespaces by tenant: each tenant's keys are stored in a tenant-specific KV prefix, rotation schedules are independent, and revocation of one tenant's key cannot accidentally match another tenant's key hash.

The edge worker enforces tenant isolation by verifying that the resolved consumer belongs to the tenant associated with the requested domain or API path. A valid key that does not match the expected tenant is rejected with a 403 Forbidden response, not a 401 Unauthorized, distinguishing "wrong tenant" from "invalid key" in error handling.

For CDN architectures serving multiple tenants on shared infrastructure, the key namespace isolation must extend to all storage tiers: in-memory cache, edge KV, and origin database. A cache key collision between tenants — where two tenants happen to have keys with the same prefix — must not result in cross-tenant authentication. The cache key should include the tenant identifier alongside the key prefix.