DDoS Behavioral Analysis: Machine Learning for Attack Pattern Recognition

At 2:17 AM last November, our NOC dashboards stayed green while a customer’s checkout API quietly fell over. It wasn’t a 100 Gbps flood — we would’ve caught that in seconds. It was 8,000 IPs, each sending three perfectly valid HTTP/2 requests per minute, cycling user-agents and holding connections just long enough to exhaust backend workers. Rate limits didn’t fire, WAF signatures didn’t match, and yet origin latency had tripled. That night is why I stopped trusting thresholds and started building for behavioral analysis.

Why Static Signatures Miss Modern DDoS Patterns

For years, our first line of defense was beautifully simple: if a single IP sends more than X requests per second, block it. If User-Agent is empty, challenge it. If request rate for a PoP exceeds Y, scrub it. That logic still handles the dumb floods, and it’s essential for absorbing the brute-force bandwidth attacks described in DDoS Volumetric Attack Patterns: Identifying and Absorbing Bandwidth Floods. But attackers don’t play that game anymore. They’ve learned that mimicking legitimate browser behavior is cheaper than building a bigger botnet.

I’ve watched the same campaign evolve over three weeks: Day 1 was a plain SYN flood, easy block. Day 7 became a randomized GET flood with proper TLS handshakes and warmed-up cookies. By Day 14, it was a low-rate, distributed request mix — 60% static asset hits, 30% API POSTs with valid JSON, 10% cache-bypass queries — all spread across residential proxies so no single ASN looked suspicious. You can’t write a regex for that. You need to look at the shape of traffic over time, not just its volume. That’s where DDoS Application Layer Attack Techniques: Slowloris, RUDY, and HTTP Floods start to overlap with ML problems: the requests are individually valid per the HTTP Working Group Specifications, but the collective behavior is off.

Behavioral analysis flips the question from “does this request match a bad pattern?” to “does this client’s behavior over the last N minutes look like our normal users?” We model inter-arrival times, request sequence entropy, header order stability, TLS JA3 consistency, and even how a client handles HTTP/2 prioritization. The web.dev Articles on modern loading patterns were actually useful here — real users fetch a document, then a burst of assets, then settle into API polling. Bots rarely replicate that cadence perfectly for hours, even when they’re trying to.

# feature_extraction.py - behavioral features per client window (60s sliding)
import numpy as np
from collections import Counter

def extract_behavioral_features(requests_window):
    # requests_window: list of dicts sorted by timestamp
    timestamps = np.array([r['ts'] for r in requests_window])
    iats = np.diff(timestamps)  # inter-arrival times
    
    # 1. Timing regularity (bots are often too perfect)
    iat_mean = np.mean(iats) if len(iats) > 0 else 0
    iat_std = np.std(iats) if len(iats) > 0 else 0
    iat_cv = iat_std / (iat_mean + 1e-6)  # coefficient of variation
    
    # 2. Request path entropy
    paths = [r['path'] for r in requests_window]
    counts = Counter(paths)
    probs = np.array(list(counts.values())) / len(paths)
    entropy = -np.sum(probs * np.log2(probs))
    
    # 3. HTTP/2 & header behavior
    header_order_variance = len(set(tuple(r['header_order']) for r in requests_window))
    http_version_mix = len(set(r['http_version'] for r in requests_window))
    
    return {
        'iat_cv': iat_cv,
        'path_entropy': entropy,
        'header_order_variance': header_order_variance,
        'http_version_mix': http_version_mix,
        'req_count': len(requests_window)
    }

Feature Engineering for Traffic Analysis at CDN Scale

Your model is only as good as the signals you feed it, and at the edge you’ve got milliseconds and megabytes to decide. I’ve learned the hard way that you can’t just dump raw logs into an autoencoder and hope for miracles. You need features that are cheap to compute at line rate and stable across PoPs. We run most of our extraction in eBPF/XDP before it ever hits userspace — aggregating per 5-tuple and per /24 windows in the kernel, then shipping sketches to our streaming pipeline.

What actually moved the needle for us in ML DDoS detection wasn’t adding 50 more features, but picking the right dozen. Flow-level timing (coefficient of variation of inter-arrival times, burstiness score, connection reuse ratio) consistently outperforms simple counts. Then add behavioral sequencing: does the client respect the site’s link graph? Real browsers crawl breadth-first; bots often hammer a single endpoint or walk APIs in alphabetical order. We also score TLS fingerprint stability — a client that rotates JA3 hashes every 10 requests isn’t a human, no matter how legitimate its headers look. For slow-rate attacks like those covering DDoS DNS Amplification Defense: Stopping Reflected Attacks at CDN Scale at the network layer, application-layer equivalents show up here as periodic, low-entropy POSTs to login or search endpoints.

Pattern recognition also needs context windowing. A fixed 1-minute window misses slowloris-style attacks that stretch over 5-10 minutes, while a 10-minute window smooths out burst attacks. Our current setup uses overlapping tumbling windows — 60s, 300s, and 900s — computed in parallel with incremental Count-Min sketches. Each window emits a feature vector normalized per PoP and per time-of-day. Without that normalization, you’ll flag every flash sale as an attack. I can’t tell you how many times we’ve retrained a model that looked perfect offline but flagged legitimate traffic during Black Friday because we trained it on Tuesday afternoon data.

-- streaming feature aggregation (ClickHouse Materialized View style)
CREATE MATERIALIZED VIEW edge_behavior_mv TO feature_store AS
SELECT
    toStartOfMinute(timestamp) AS window_start,
    client_ip_subnet,           -- /24 aggregated to resist rotation
    pop_id,
    -- traffic analysis features computed incrementally
    avg(inter_arrival_ms) AS iat_mean,
    stddevPop(inter_arrival_ms) / avg(inter_arrival_ms) AS iat_cv,
    uniqExact(path) / count() AS path_uniqueness,
    countIf(http_version = 'h2') / count() AS h2_ratio,
    entropy(header_order_hash) AS header_entropy,
    count() AS reqs_per_window
FROM edge_access_log
GROUP BY window_start, client_ip_subnet, pop_id;

Putting Models Into Production Without Taking Down Legitimate Users

Choosing an algorithm is the easy part. Making it survive production is where most teams get burned. We started with unsupervised models because labeled DDoS data is messy — you’ve got mixed attack and flash crowd traffic, and labeling it after the fact is half guesswork. Isolation Forests worked surprisingly well for initial anomaly triage, but they struggled with concept drift. An LSTM autoencoder trained on sequences gave us better recall on low-and-slow HTTP floods, but inference latency at 250k RPS per PoP was brutal.

What we run now is an ensemble, and honestly it’s less fancy than it sounds. An XGBoost classifier on the engineered window features handles 90% of decisions in under 2ms, with an Isolation Forest as a fallback for unseen bot families. We score every subnet, not every IP — blocking a single IP when the attacker has 40k is pointless, and it keeps cardinality manageable. Crucially, we don’t auto-block on model score alone. A score above 0.85 triggers a soft challenge (interactive proof-of-work or Turnstile), not a hard drop. False positives at CDN scale are far more expensive than a few extra requests hitting origin.

Evaluation has to mirror reality. Offline AUC means nothing if your training set was collected on a quiet day in us-east-1 and you deploy to an Asian PoP during a botnet burst. We test on replayed traffic captures with injected attacks, measure per-PoP false positive rates during peak hours, and monitor drift daily with PSI (Population Stability Index) on key features. If header entropy or h2_ratio drifts beyond 0.2 PSI week-over-week, we trigger retraining. That feedback loop matters more than squeezing another 0.01 of precision.

Deploying Behavioral Models at the Edge: From Notebook to 200 PoPs

Getting a model that hits 99.1% accuracy in Jupyter is the easy part. Getting it to score 3.2 million requests per second across 200 PoPs without adding more than 0.8ms of latency — that's where most teams fall over. We don't run inference in the slow path. If your model lives behind a synchronous API call, you've already lost. Our architecture pushes lightweight, quantized models (we're using ONNX Runtime with INT8 quantization, models under 4.7MB) directly into the data plane via eBPF and VCL-adjacent workers. The heavy lifting — retraining, threshold tuning, and feature store aggregation — stays in the control plane and syncs every 45 seconds.

You'll want to start with a shadow-mode deployment for at least 72 hours. Don't block, just log. We mirror 5% of live traffic through the behavioral scorer and compare its verdict against our existing signature+rate-limit stack. In our last rollout, the new Isolation Forest flagged 18,400 flows per minute that signatures missed, but it also had a 0.34% false positive rate on API gateway traffic during the first 12 hours. That was enough to accidentally throttle legitimate webhook retries from Shopify and Stripe. Shadow mode caught it before customers did. Once your false positive rate holds steady below 0.05% for 48 hours, you can graduate to enforce mode with a soft-action — JS challenge or CAPTCHA — before you go to hard drop.

The trick is keeping your feature computation cheap. You can't calculate 47 features per packet at 800 Gbps. We pre-aggregate at the edge into 1-second tumbling windows and only compute 12 high-signal features inline: packet size entropy, SYN-to-ACK ratio, TTL variance, flow duration, and burstiness coefficient are non-negotiable. Anything requiring a global view — like cross-PoP IP correlation — gets deferred to the central analyzer. Here's the stripped-down feature pipeline we run on every edge worker:

# feature_extractor.py - runs inline on edge worker (pypy, <0.3ms p99)
import numpy as np
from collections import deque

WINDOW_SIZE = 1.0  # 1-second tumbling window
FEATURE_TTL = 60   # seconds

def extract_features(flow_window):
    pkts = flow_window.packets
    if len(pkts) < 10:
        return None  # not enough data, skip scoring
    
    sizes = np.array([p.size for p in pkts])
    iats = np.diff([p.timestamp for p in pkts])  # inter-arrival times
    ttls = np.array([p.ttl for p in pkts])
    
    features = {
        'pkt_size_entropy': -np.sum(np.bincount(sizes) / len(sizes) * 
                             np.log2(np.bincount(sizes) / len(sizes) + 1e-9)),
        'iat_cv': np.std(iats) / (np.mean(iats) + 1e-6),  # coefficient of variation
        'ttl_variance': np.var(ttls),
        'syn_ratio': sum(1 for p in pkts if p.flags == 'SYN') / len(pkts),
        'burst_score': np.max(np.bincount((np.array([p.timestamp for p in pkts]) * 10).astype(int))),
        'flow_duration': flow_window.end - flow_window.start
    }
    # returns dict for ONNX model; total compute < 0.18ms on Xeon 8370C
    return features

# Batch score with ONNX; threshold tuned to 0.87 for <0.05% FPR
# score > 0.87 -> challenge, > 0.94 -> drop + BGP Flowspec push

Advanced Patterns: When Attackers Learn Your Model

Once you've been blocking volumetric attacks reliably, attackers adapt. The two patterns that consistently fool naive behavioral models are low-and-slow application-layer floods and DDoS Carpet Bombing Defense: When Attackers Spread Traffic Across Your Entire Subnet. Carpet bombing is particularly nasty for ML because instead of hammering a single VIP with 40M packets per second, the attacker spreads 800k pps across your entire /22 — 1024 IPs at ~780 pps each. Per-IP, that looks like normal traffic. Your model sees nothing. We've found you need subnet-level aggregation features; we compute entropy across the destination IP distribution within each /24 every 2 seconds. A legitimate flash crowd hits 3-5 IPs heavily. Carpet bombing hits all 256 IPs with a near-uniform distribution (entropy > 7.8). If you're not aggregating at the prefix level, you'll miss it.

The other headache is concept drift. Attack tools like botnet-based HTTP/2 rapid reset or AI-generated browser fingerprints evolve weekly, and what looked anomalous in March is baseline by June. We retrain our autoencoder weekly on the last 7 days of clean traffic (sampled at 1:1000, roughly 4.2TB per day after sampling), but more importantly, we monitor feature drift with Population Stability Index (PSI). If PSI for any top-6 feature exceeds 0.25 over a rolling 6-hour window, we automatically widen thresholds by 12% and trigger a canary retrain. Don't let your model go stale — we saw a client's model degrade from 98.2% recall to 81.4% in just 19 days because they never retrained after a major Chrome update changed TLS fingerprint distributions.

And don't treat every amplification vector the same. Your behavioral features for a DDoS NTP Reflection Mitigation: Blocking Monlist-Based Amplification Attacks (small request, huge 400x response, UDP port 123, packet size > 440 bytes) look completely different from a DDoS Memcached Amplification Defense: Neutralizing 51,000x Amplification Factor where you're seeing 1.2KB UDP packets on port 11211 with payload entropy under 3.1. We run separate, specialized sub-models per vector and ensemble them. A single generic model will average out those distinctions and give you mediocre precision on both. Keep your NTP model focused on UDP amplification signatures and your L7 model focused on request-rate anomalies and header-order randomization.

Frequently Asked Questions