CDN Storage Cost Optimization: Tiering, Compression, and Lifecycle Policies

I didn't notice our origin storage spend had doubled until finance forwarded the invoice with a subject line that just said "???" We'd been busy tuning cache hit ratios and negotiating bandwidth commits, and meanwhile 40TB of old video renditions, uncompressed logs, and forgotten staging assets had quietly accumulated in standard storage. That's the trap with CDN storage — it's cheap per gigabyte until you realize you're paying premium hot-storage rates for content no edge server has requested in eight months.

The Silent Growth Problem Behind Your Origin Bucket

Most teams assume storage costs track traffic. They don't. Traffic is bursty and visible, but storage is cumulative and almost invisible. Every time your CMS creates three image variants, every time your transcode pipeline spits out a 4K, 1080p, and 720p version "just in case," and every time your log delivery dumps another 50GB of access logs into the same bucket, you're adding to a bill that never goes down on its own. I've audited origins where 60% of objects hadn't been accessed in 90+ days yet sat in the most expensive tier because no one had set a default lifecycle.

What's tricky with CDN origins is the interaction between edge cache and origin storage. You might have a 95% cache hit ratio — which looks great on your CDN Egress Cost Modeling: Understanding and Reducing Your Largest Bill Component dashboard — but your origin still stores 100% of the assets for cache fill. And if you're using origin shield or multi-region origins for resilience, you could be storing that same 100% twice or three times. I worked with a media client in 2021 who replicated their entire 80TB library to two shield regions for redundancy without realizing replication doubled their storage cost overnight.

The first step isn't buying more storage; it's getting visibility. Before you touch any tiering rule, you need an inventory with last-access timestamps, not just creation dates. Most object storage provides this via inventory reports or access logs. Here’s a lifecycle skeleton I now use as a starting baseline for almost every S3-compatible origin — it prevents the default of "store everything forever in Standard":

{
  "Rules": [
    {
      "ID": "origin-tiering-baseline",
      "Filter": { "Prefix": "assets/" },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 45,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER_IR"
        }
      ],
      "NoncurrentVersionTransitions": [
        {
          "NoncurrentDays": 30,
          "StorageClass": "STANDARD_IA"
        }
      ],
      "NoncurrentVersionExpiration": {
        "NoncurrentDays": 90
      },
      "AbortIncompleteMultipartUpload": {
        "DaysAfterInitiation": 7
      },
      "Expiration": {
        "Days": 365
      }
    },
    {
      "ID": "logs-cleanup",
      "Filter": { "Prefix": "logs/cdn-access/" },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 180
      }
    }
  ]
}

Designing Tiering That Follows Access Patterns, Not the Calendar

Time-based tiering is the simplest approach — move objects to cheaper storage after 30, 60, or 90 days — but it's also the easiest to get wrong. If you move a 2MB product image to an archive tier after 60 days and that image gets requested once a week from organic search, you'll pay more in retrieval fees than you saved. I've seen that mistake cost a retail site an extra $1,800 in a single month. Real tiering decisions should be based on access frequency, object size, and retrieval penalty, not just age.

For CDN workloads, that means segmenting by content type first. Your top 5% of assets — hero images, homepage JS bundles, viral videos — should stay hot. They’re requested constantly and need millisecond origin fetch when there's a cache miss. The middle 20-30% — long-tail product photos, older blog embeds — are perfect for infrequent-access tiers where storage is ~45% cheaper but retrieval is still fast. The bottom 50% — outdated renditions, raw uploads, logs, and backups — shouldn't be in hot storage at all. When you're evaluating whether to commit to a bandwidth plan versus staying pay-as-you-go, this segmentation becomes crucial, which is why I often point teams to the CDN Bandwidth Commitment Pricing: When Committed Plans Save Money vs Pay-As-You-Go analysis before they lock in storage contracts too.

We’ve had the best results running a monthly access-pattern query before adjusting transitions. Don't guess — pull the last 60 days of edge access logs, join them to your origin inventory, and classify. The script below is a simplified version of what we run on Athena against our CloudFront logs. It flags candidates for downgrading and estimates savings so you can see if the retrieval cost will wipe out the benefit:

-- Find origin objects with low access frequency for tiering review
SELECT
  inventory.key,
  inventory.size_bytes / 1024/1024/1024 as size_gb,
  inventory.storage_class,
  COUNT(logs.request_uri) as edge_requests_60d,
  MAX(logs.date) as last_requested
FROM origin_inventory inventory
LEFT JOIN cdn_access_logs logs
  ON inventory.key = regexp_extract(logs.request_uri, '[^?]+')
  AND logs.date >= date_add('day', -60, current_date)
WHERE inventory.bucket = 'prod-origin-assets'
GROUP BY 1, 2, 3
HAVING COUNT(logs.request_uri) < 5  -- fewer than 5 hits in 60 days
   AND inventory.size_bytes > 10485760 -- larger than 10MB
ORDER BY size_gb DESC
LIMIT 500;

Compression and Format Efficiency: Shrinking What You Store

Tiering moves bytes to cheaper places; compression means you store fewer bytes in the first place. And with HTTP delivery, these two work together. We cut one news publisher's origin footprint by 38% in a week not by deleting anything, but by recompressing their image library to modern formats and enabling aggressive text compression at the origin. Their cache hit ratio stayed identical, but their monthly storage dropped by ~22TB.

For images and video, format modernization is the biggest win. If you’re still storing both a JPEG and a WebP variant for every asset, you’re doubling storage for a problem the CDN can handle with on-the-fly conversion or content negotiation via Accept headers. Store the highest-quality modern source — AVIF or WebP for images, HEVC or AV1 for video — and let the edge transcode or serve fallbacks. For text assets, pre-compressing with Brotli (level 11) at build time and storing the .br variant alongside gzip means you aren’t re-compressing on every origin fetch. The HTTP Working Group Specifications define how Content-Encoding and Vary interact here, and the guides on web.dev Articles give solid benchmarks for size savings by format.

Don’t overlook deduplication and cleanup of non-servable content. Incomplete multipart uploads, orphaned chunks from failed transcodes, and versioned objects where you keep 12 previous versions forever add up fast. That’s where lifecycle policies stop being optional. Pair your tiering rules with expiration for noncurrent versions and abort rules for multipart uploads, and review your origin’s CDN Request Pricing Analysis: Reducing Per-Request Costs at Scale to make sure you aren’t creating extra origin GETs by serving too many tiny un-cacheable objects that should have been concatenated or purged.

Putting It All Together: Lifecycle Automation That Doesn't Tank Performance

Knowing you should tier and compress is the easy part. Making it happen in production without causing a spike in origin fetch latency or cache misses — that's where most teams stumble. Don't try to hand-migrate objects. Let lifecycle policies do the heavy lifting, but test them against your real access logs first, not just assumptions.

We typically start by mirroring 30 days of CDN access logs against our origin inventory using Athena or BigQuery. You're looking for that cliff where objects stop getting requested. For most media-heavy origins, we've found it's surprisingly sharp: 85-90% of objects aren't touched again after 28-32 days. That's your signal to move from Standard to Infrequent Access. But here's the trick: don't go straight to Glacier. Use INTELLIGENT_TIERING or equivalent auto-tiering for the middle ground, and reserve Glacier Instant Retrieval only for versioned objects or compliance archives. On compression, do it once at ingest and cache the variants at the edge. Re-compressing on-the-fly with edge functions is powerful for content negotiation, but you'll want to model that compute cost carefully — check out our breakdown of CDN Edge Compute Cost Optimization: Function Pricing Models and Efficiency Tricks before you deploy Brotli-11 to every request.

# Terraform: tiering that follows access, not the calendar
resource "aws_s3_bucket_lifecycle_configuration" "cdn_origin" {
  bucket = aws_s3_bucket.origin.id

  rule {
    id     = "assets-hot-to-warm"
    status = "Enabled"
    filter { prefix = "assets/" }

    transition {
      days          = 30
      storage_class = "STANDARD_IA" # ~$0.0125/GB vs $0.023/GB Standard
    }
    transition {
      days          = 90
      storage_class = "INTELLIGENT_TIERING" # auto-optimizes for unknown access
    }
    noncurrent_version_transition {
      noncurrent_days = 14
      storage_class   = "GLACIER_INSTANT_RETRIEVAL" # keep old versions cheap but retrievable in ms
    }
    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }

  rule {
    id     = "abort-multipart"
    status = "Enabled"
    filter {}
    abort_incomplete_multipart_upload { days_after_initiation = 7 }
  }
}
# Pro tip: Enable S3 Storage Lens first. If your average object is <128KB,
# STANDARD_IA's 128KB minimum billable size will actually cost you MORE.

Advanced Patterns and Troubleshooting When Savings Disappear

We've seen teams cut storage bills by 40% and then lose it all on retrieval fees the next month. The most common culprit is aggressive archiving of content that looks cold but isn't. If your origin shield or edge cache has a low hit ratio — say, below 92% — you'll be pulling those Glacier objects more often than you expect. Glacier Flexible Retrieval charges $0.03 per 1,000 requests plus $0.01/GB for restores, and Standard restores take 3-5 hours. That's not just expensive, it'll cause origin timeouts and 5xx errors. Before you push anything to an archive tier, model your restore cost: (monthly GETs on tiered objects × retrieval fee) + (egress × cache miss rate). If that number is greater than your Standard-IA savings, you've gone too far. For predictable, always-warm workloads, you're often better off negotiating CDN Reserved Capacity Pricing: Locking In Rates for Predictable Workloads than chasing deep archive tiers.

Another gotcha is compression/format mismatches between origin and edge. You convert 2TB of JPEGs to AVIF and save ~45% on storage — great. But if your CDN isn't caching the Vary: Accept variants correctly, you'll serve AVIF to a browser that doesn't support it or re-transcode at the edge for every request. Make sure your cache key includes the content negotiation header and that your origin returns a proper Content-Encoding and immutable ETag per variant. We've also had great success running batch re-encoding jobs — like bulk WebP/AVIF conversion for a 500k image library — on spare compute to avoid paying for on-demand transcoders. For those batch workloads, CDN Spot Pricing for Edge Compute: Leveraging Unused Capacity for Savings can cut your one-time processing cost by 60-70% compared to running it on always-on workers.

Finally, monitor the right metrics. Don't just watch your AWS bill. Set alerts on OriginFetchTime, CacheHitRatio by path, and LifecycleTransitionCount. If your 95th percentile origin fetch time jumps from 80ms to 400ms after a lifecycle change, you've tiered too aggressively. A good safety net is to keep a 7-day noncurrent version window on Standard before any transition — it gives you an instant rollback without paying for a full Glacier restore if you need to revert a bad policy.

Frequently Asked Questions