TopolearnThroughput problems in gRPC usually come from windows, limits and connection usage, not CPU.
HTTP/2 flow control stops a fast sender from overwhelming a slow receiver: each stream and each connection has a window of bytes the peer may send before receiving a WINDOW_UPDATE. The C++, Go and Java implementations use BDP probing (bandwidth-delay product estimation with PINGs) to grow windows automatically on high-latency, high-bandwidth links. When a streaming consumer stops reading, the window fills and the sender's write() reports backpressure: this is how gRPC streaming is flow-controlled end to end.
Other levers, in rough order of usefulness:
- Reuse channels. Create channels once; each new one costs DNS, TCP and TLS handshakes.
- Concurrent stream limits. Servers advertise
MAX_CONCURRENT_STREAMS(often 100). Under heavy load a single connection queues calls; use several channels (a channel pool) or client-side balancing across many backends. - Message size. The default 4 MB receive limit (
grpc.max_receive_message_length) exists for a reason; prefer streaming chunks to raising it. - Compression (
gzip) trades CPU for bandwidth; worthwhile on large, repetitive payloads over slow links, wasteful on small messages in a data centre. - Serialization cost: in Node, dynamic proto-loader objects are convenient but slower than generated code; large
bytesfields avoid copying when kept as Buffers.
Measure with ghz or the gRPC benchmark tools, watch p99 latency rather than averages, and change one setting at a time.
Example
import * as grpc from "@grpc/grpc-js";
// Channel options for a high-throughput internal client
const options = {
"grpc.max_receive_message_length": 8 * 1024 * 1024, // 8 MB instead of 4 MB
"grpc.max_send_message_length": 8 * 1024 * 1024,
"grpc.default_compression_algorithm": 2, // 2 = gzip
"grpc.keepalive_time_ms": 60_000,
};
// Server side: cap concurrent streams per connection
const server = new grpc.Server({ "grpc.max_concurrent_streams": 200 });
console.log(Object.keys(options).length, typeof server.bindAsync); // 4 function
// Load test from a shell (ghz):
// ghz --insecure --proto hello.proto --call hello.v1.Greeter.SayHello \
// -d '{"name":"Ann"}' -c 50 -n 20000 localhost:50051Remember
- HTTP/2 windows give end-to-end backpressure; BDP probing grows them
- MAX_CONCURRENT_STREAMS can queue calls on one connection
- Stream large payloads instead of raising message limits
- Compress only when bandwidth, not CPU, is the bottleneck
Watch out for
- Raising max message size to hundreds of MB instead of streaming, causing memory spikes.