gRPC Proxying Through CDN: Load Balancing and Protocol Translation

gRPC has become the default protocol for service-to-service communication in distributed systems, but extending gRPC beyond the data center—to mobile clients, browser applications, and partner integrations—introduces challenges that CDN infrastructure can solve. The protocol's dependence on HTTP/2, binary protobuf encoding, and bidirectional streaming creates a fundamentally different traffic pattern than the HTTP/1.1 request-response model that CDNs were originally built to accelerate. This guide examines how CDN edge nodes proxy gRPC traffic, the load balancing strategies that account for HTTP/2's multiplexed connections, and the protocol translation patterns that bridge gRPC with REST and gRPC-Web clients.

gRPC Over HTTP/2: What CDN Edge Nodes Must Handle

gRPC encodes remote procedure calls as HTTP/2 frames. A unary RPC (single request, single response) maps to a standard HTTP/2 request-response exchange. The method name appears in the :path pseudo-header as /package.Service/Method, and the request and response bodies carry length-prefixed protobuf messages. The content-type is application/grpc rather than the application/json that CDN routing rules typically expect.

Streaming RPCs are where the complexity escalates. Server-streaming RPCs send multiple response messages on a single HTTP/2 stream. Client-streaming RPCs send multiple request messages before receiving a response. Bidirectional streaming combines both: the client and server exchange messages concurrently over the same stream, with neither side closing until the application logic decides the conversation is complete.

A CDN edge node proxying gRPC must maintain HTTP/2 connections to both the client and the origin, translate frames between the two connections, handle flow control for streams with asymmetric throughput, and propagate gRPC-specific trailers (the grpc-status and grpc-message headers that appear after the response body rather than before it). This trailer requirement is particularly important: unlike HTTP where all headers precede the body, gRPC uses HTTP/2 trailing headers to communicate the RPC's success or failure status after all response data has been sent.

Protocol Translation: gRPC-Web at the Edge

Browsers cannot make native gRPC calls. The Fetch API and XMLHttpRequest lack the HTTP/2 frame-level control that gRPC requires: they cannot send HTTP/2 trailers, they cannot initiate multiple streams on a single connection, and they cannot read response data incrementally in the way that server-streaming RPCs demand. The gRPC-Web protocol bridges this gap by encoding gRPC messages in a format compatible with standard browser HTTP APIs.

A CDN edge node running a gRPC-Web proxy accepts these browser-compatible requests and translates them into native gRPC calls to the origin. The translation involves converting the gRPC-Web binary or text encoding to standard protobuf-over-HTTP/2, extracting trailers from the response body (where gRPC-Web embeds them) and converting them to HTTP/2 trailing headers, and managing the response framing for server-streaming RPCs.

// gRPC-Web request from browser (HTTP/1.1 or HTTP/2)
POST /grpc.health.v1.Health/Check HTTP/1.1
Content-Type: application/grpc-web+proto
X-Grpc-Web: 1
Accept: application/grpc-web+proto

// Binary protobuf body with gRPC-Web frame prefix:
// [1 byte: compressed flag] [4 bytes: message length] [N bytes: protobuf]

// CDN Edge translates to native gRPC (HTTP/2):
// POST /grpc.health.v1.Health/Check HTTP/2
// Content-Type: application/grpc
// TE: trailers
// [standard gRPC length-prefixed protobuf]

// Origin responds with gRPC (HTTP/2 + trailers):
// 200 OK
// Content-Type: application/grpc
// [protobuf response body]
// TRAILERS: grpc-status: 0, grpc-message: OK

// CDN Edge translates back to gRPC-Web:
// 200 OK
// Content-Type: application/grpc-web+proto
// [protobuf body + embedded trailers in body frame]

Running this translation at the CDN edge rather than at the origin eliminates the need for a separate Envoy or nginx gRPC-Web proxy in front of every gRPC service. The edge proxy handles TLS termination, connection coalescing, and protocol translation in one hop, reducing the infrastructure complexity and latency that a multi-proxy architecture introduces.

Stream-Level Load Balancing

HTTP/2 multiplexing creates a load balancing problem that does not exist with HTTP/1.1. Under HTTP/1.1, each request opens a new connection (or reuses one from a small pool), and the load balancer distributes connections across backends. Since each connection carries one request at a time, connection-level balancing provides reasonable request-level distribution.

gRPC clients maintain persistent HTTP/2 connections to the CDN edge, and each connection carries potentially thousands of concurrent streams. If the CDN edge forwards all streams from a client connection to a single backend connection, one backend absorbs all the traffic from that client while others remain idle. This connection affinity problem worsens as clients become "stickier"—a mobile app that maintains a single gRPC connection for its entire session concentrates all its traffic on one backend.

// Connection-level balancing (WRONG for gRPC):
// Client A ──── Connection 1 ──── Backend 1 (overloaded: 1000 streams)
// Client B ──── Connection 2 ──── Backend 2 (overloaded: 800 streams)
//                                  Backend 3 (idle: 0 streams)
//                                  Backend 4 (idle: 0 streams)

// Stream-level balancing (CORRECT for gRPC):
// Client A ──── Connection 1 ──┬── Backend 1 (250 streams)
//                              ├── Backend 2 (250 streams)
//                              ├── Backend 3 (250 streams)
//                              └── Backend 4 (250 streams)
// Each RPC routed independently, regardless of source connection

CDN providers that support gRPC implement stream-level (or request-level) load balancing. For each individual RPC within an HTTP/2 connection, the edge node makes an independent routing decision. Cloudflare's gRPC support routes each unary RPC to the least-loaded backend. AWS Application Load Balancer (ALB) distributes gRPC requests using a round-robin algorithm across target group instances. Google Cloud's gRPC load balancer uses latency-based routing combined with backend utilization signals.

Health Checking and Connection Management

gRPC defines a standard health checking protocol (grpc.health.v1.Health/Check) that CDN edge nodes can use to monitor backend availability. The health check RPC returns one of four statuses: SERVING, NOT_SERVING, SERVICE_UNKNOWN, or UNKNOWN. The CDN should remove backends from the rotation when they return anything other than SERVING and restore them when they return to healthy status.

Connection management between the CDN edge and origin requires balancing connection reuse (to avoid the overhead of TCP+TLS handshakes) with connection freshness (to pick up DNS changes and infrastructure updates). A common pattern sets a maximum connection lifetime of 10-15 minutes and a maximum number of streams per connection of 100-1000, after which the edge opens a new connection and gracefully drains the old one using HTTP/2 GOAWAY frames.

Timeout and Deadline Propagation

gRPC uses a deadline model rather than a timeout model. The client sets an absolute deadline (a point in time), and every hop in the call chain subtracts the time it has already consumed before forwarding the remaining deadline to the next hop. This prevents the "timeout amplification" problem where each proxy adds its own timeout, potentially allowing a request to consume far more total time than the client intended.

The CDN edge must correctly propagate the grpc-timeout header, which encodes the remaining time in a compact format (e.g., 5S for 5 seconds, 100m for 100 milliseconds). When the edge receives a request with grpc-timeout: 5S and spends 50 milliseconds on TLS termination and routing, it should forward the request to the origin with grpc-timeout: 4950m. This header must not be confused with or overridden by the CDN's own HTTP timeout configuration.

// Deadline propagation through CDN edge
// Client sets 5-second deadline

Client → CDN Edge (50ms TLS + routing)
  grpc-timeout: 5S    →    grpc-timeout: 4950m

CDN Edge → Origin (20ms network)
  grpc-timeout: 4950m  →  grpc-timeout: 4930m

// If origin takes 4940ms to respond:
// Only 10ms remains → DEADLINE_EXCEEDED at origin
// CDN edge receives grpc-status: 4 (DEADLINE_EXCEEDED)
// Propagates deadline error to client

For streaming RPCs, the timeout semantics change. The grpc-timeout applies to the entire stream, not to individual messages. A server-streaming RPC with a 30-second timeout must deliver all messages and close the stream within 30 seconds. The CDN edge needs separate configurations for its idle timeout (time between messages) and the overall stream timeout. Setting the idle timeout too low will kill active streams during natural pauses between messages; setting it too high wastes edge resources on abandoned connections.

Metadata and Header Management

gRPC uses HTTP/2 headers for metadata: authentication tokens, trace context, custom application headers. The CDN edge must pass through all grpc-* prefixed headers and any custom metadata the application defines. At the same time, the edge adds its own headers for observability: X-CDN-Edge, X-Request-Id, CF-Ray (Cloudflare). These additions must not conflict with gRPC's reserved header space or exceed HTTP/2 header table limits.

Distributed tracing integration is particularly important for gRPC through CDN. The edge should propagate W3C Trace Context headers (traceparent, tracestate) or OpenTelemetry baggage headers without modification. If the edge generates its own trace span for the proxy hop, it should create a child span of the incoming trace rather than starting a new trace context. This preserves end-to-end visibility from the client through the CDN edge to the origin and across any downstream microservices.

Performance Optimization: Connection Pooling

The CDN edge maintains a pool of HTTP/2 connections to each gRPC origin. The optimal pool size depends on the traffic volume, the number of concurrent streams each connection supports (negotiated via HTTP/2 SETTINGS frames), and the origin's capacity. Too few connections force excessive stream multiplexing, increasing head-of-line blocking at the HTTP/2 layer. Too many connections waste origin resources and reduce the efficiency of TCP congestion control.

// Connection pool sizing for gRPC origin
//
// Given:
//   Peak concurrent RPCs: 10,000
//   Max streams per connection (server-set): 250
//   Target utilization: 60% (headroom for bursts)
//
// Pool size = peak_rpcs / (max_streams * utilization)
//           = 10,000 / (250 * 0.6)
//           = 67 connections per edge node to this origin
//
// Connection lifecycle:
//   max_age: 900s (15 min)
//   max_idle: 300s (5 min)
//   max_streams_lifetime: 10,000 (per connection)
//   health_check_interval: 10s

Connection warmup is another consideration. Cold HTTP/2 connections require TCP handshake (1 RTT), TLS handshake (1-2 RTT), and HTTP/2 settings exchange (1 RTT) before the first RPC can be sent. For latency-sensitive services, the CDN should pre-establish connections to known origins before traffic arrives. This "pre-warming" strategy keeps a minimum pool size of warm connections even during low-traffic periods, trading a small amount of resource consumption for consistently low first-RPC latency.

Security: mTLS and gRPC at the Edge

Many gRPC deployments use mutual TLS (mTLS) for service-to-service authentication. When a CDN edge terminates the client's TLS connection, the origin loses visibility into the client's certificate. The CDN must forward the client certificate information through headers (e.g., X-Client-Cert containing the PEM-encoded certificate) or use a separate mTLS connection between the edge and origin with the CDN's own certificate.

For public-facing gRPC APIs, the CDN edge can enforce authentication before proxying to the origin. This might involve validating JWT tokens in gRPC metadata headers, checking API keys, or enforcing rate limits per client identity. Performing these checks at the edge prevents unauthenticated traffic from consuming origin resources.

Operational Recommendations

Start by enabling gRPC support on your CDN for unary RPCs only. These behave identically to standard HTTP/2 request-response patterns and require no special timeout or streaming configuration. Once unary gRPC is stable, enable server-streaming RPCs with extended idle timeouts, then client-streaming, and finally bidirectional streaming—each type requiring progressively more complex CDN configuration and monitoring.

Monitor gRPC-specific metrics at the CDN edge: RPC success rate by method, p99 latency per service method, stream duration distribution, and grpc-status code distribution. These metrics reveal problems that HTTP-level monitoring misses: a service returning HTTP 200 but grpc-status: 13 (Internal) is functionally broken despite appearing healthy to standard HTTP monitors. Configure alerts on non-zero rates of grpc-status codes 4 (DeadlineExceeded), 8 (ResourceExhausted), and 14 (Unavailable) as indicators of capacity or timeout misconfiguration.