TopolearnCalls without deadlines are how one slow service takes down a whole system.
A deadline is an absolute point in time by which the client wants an answer. gRPC sends it to the server as the grpc-timeout header, and if it passes, the call fails with DEADLINE_EXCEEDED on the client and is cancelled on the server.
By default, many gRPC implementations have no deadline: a call to a hung server waits forever while holding resources. Always set one, sized to the operation (e.g. 300 ms for a lookup, 30 s for a report).
In grpc-js you pass { deadline: Date.now() + 500 } as call options (a Date or millisecond timestamp). On the server:
call.getDeadline()tells you the remaining budget.call.cancelledand thecancelledevent tell you the client gave up, timed out, or disconnected. Stop work and free resources.
Deadline propagation: when your handler calls other services, give those calls the incoming deadline (minus a little margin), not a fresh fixed timeout. Otherwise downstream services keep working long after the original caller has gone. Several gRPC libraries (Go, Java) propagate it through the context automatically; in Node you pass it yourself, e.g. { deadline: call.getDeadline() }, or with propagate_flags and parent.
Clients can also cancel explicitly with call.cancel(), which yields CANCELLED.
Example
import * as grpc from "@grpc/grpc-js";
import * as protoLoader from "@grpc/proto-loader";
const { hello } = grpc.loadPackageDefinition(protoLoader.loadSync("hello.proto"));
const server = new grpc.Server();
server.addService(hello.v1.Greeter.service, {
SayHello: (call, callback) => {
const timer = setTimeout(() => callback(null, { message: "late" }), 2000);
call.on("cancelled", () => { clearTimeout(timer); console.log("server: cancelled"); });
},
});
server.bindAsync("127.0.0.1:50054", grpc.ServerCredentials.createInsecure(), () => {
const client = new hello.v1.Greeter("127.0.0.1:50054", grpc.credentials.createInsecure());
client.SayHello({ name: "Ann" }, { deadline: Date.now() + 200 }, (err) => {
console.log("client:", grpc.status[err.code]); // client: DEADLINE_EXCEEDED
setTimeout(() => server.forceShutdown(), 100);
});
});
// also printed: server: cancelledRemember
- Set a deadline on every call; the default is often none
- Deadline passing = DEADLINE_EXCEEDED for client, cancellation for server
- Propagate the incoming deadline to downstream calls
- Stop work on the
cancelledevent
Watch out for
- Using a fixed 30 s timeout on each hop, so a 5-hop chain can run far past the user's patience.