TopolearnUnbalanced gRPC traffic is one of the most common production surprises.
A gRPC client opens one HTTP/2 connection and multiplexes every call over it. That is efficient, but it breaks connection-level (L4) load balancing: a TCP load balancer picks a backend once per connection, so all calls from a client stick to one server, and newly scaled-up pods receive nothing.
Two ways to balance per call:
- Proxy (L7) load balancing: Envoy, Linkerd, Istio, NGINX or a cloud L7 load balancer terminates HTTP/2 and spreads individual calls across backends. Simple for clients, costs a hop.
- Client-side load balancing: the client resolves several addresses (e.g. a Kubernetes headless service via
dns:///my-svc:50051) and the channel's policy spreads calls. Setround_robinthrough the service config; the defaultpick_firstuses just one address. xDS-based clients get endpoints and policies from a control plane, proxyless.
Connections also die silently (NAT timeouts, idle proxies). Keepalive sends HTTP/2 PINGs to detect that: grpc.keepalive_time_ms and grpc.keepalive_timeout_ms channel options. Servers enforce a minimum ping interval and will close connections from clients that ping too often (too_many_pings), so coordinate the settings.
Servers can also set grpc.max_connection_age_ms so clients periodically reconnect and rediscover new backends.
Example
import * as grpc from "@grpc/grpc-js";
import * as protoLoader from "@grpc/proto-loader";
const { hello } = grpc.loadPackageDefinition(protoLoader.loadSync("hello.proto"));
// Client-side round robin over every address DNS returns
const client = new hello.v1.Greeter(
"dns:///greeter.default.svc.cluster.local:50051", // headless service
grpc.credentials.createInsecure(),
{
"grpc.service_config": JSON.stringify({ loadBalancingConfig: [{ round_robin: {} }] }),
"grpc.keepalive_time_ms": 30_000, // ping after 30 s idle
"grpc.keepalive_timeout_ms": 10_000, // drop if no ack in 10 s
},
);
// Server: make clients reconnect every 5 minutes to rebalance
const server = new grpc.Server({
"grpc.max_connection_age_ms": 5 * 60_000,
"grpc.keepalive_permit_without_calls": 1,
});
console.log(typeof client.SayHello, typeof server.addService); // function functionRemember
- One long-lived HTTP/2 connection defeats L4 load balancers
- Balance per call with an L7 proxy or client-side round_robin
- Keepalive pings detect dead connections; match server limits
- max_connection_age forces periodic rebalancing
Watch out for
- Putting gRPC behind a plain TCP load balancer and wondering why one pod gets all the traffic.