TopolearnRunning more than one application instance needs a balancer that also routes around failures.
An upstream block names a group of servers; proxy_pass http://name; sends requests to it.
Balancing methods:
- round robin (default), optionally with
weight=. least_conn;: the server with the fewest active connections; good when request durations vary.ip_hash;orhash $key consistent;: sticky mapping of clients or keys to servers.random two least_conn;: pick two at random, take the less busy.
Passive health checks: with max_fails=3 fail_timeout=30s a server that fails 3 times within 30 seconds is marked unavailable for 30 seconds. What counts as a failure, and whether the request is retried on the next server, is set by proxy_next_upstream (by default error timeout). Retries are only done for non-idempotent methods like POST if you explicitly allow it. A server marked backup only receives traffic when the others are down. Active health checks (probing a URL) are an NGINX Plus feature.
Keepalive to upstreams avoids a new TCP connection per request: add keepalive 32; to the upstream, plus proxy_http_version 1.1; and proxy_set_header Connection ""; in the location. Without those two, NGINX uses HTTP/1.0 with Connection: close to the backends.
Example
upstream app {
least_conn;
server 10.0.0.11:8000 max_fails=3 fail_timeout=30s;
server 10.0.0.12:8000 max_fails=3 fail_timeout=30s;
server 10.0.0.13:8000 backup; # only when both are down
keepalive 32; # idle connections kept per worker
}
server {
listen 80;
location / {
proxy_pass http://app;
proxy_http_version 1.1; # needed for upstream keepalive
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_next_upstream error timeout http_502;
proxy_next_upstream_tries 2;
}
}Remember
- upstream groups servers; round robin by default, least_conn and hash available
- max_fails/fail_timeout are passive health checks; backup is for failover
- Upstream keepalive needs HTTP/1.1 and an empty Connection header
Watch out for
- Adding keepalive in upstream but not proxy_http_version 1.1, so connections are still closed.