Master, workers and the event loop

NGINX · Expert · 12 min

Capacity planning, file descriptor limits and zero-downtime operations all follow from the process model.

NGINX runs one master process (as root, to bind ports and read certificates) and several worker processes (as an unprivileged user) that handle all traffic. Optional cache manager and cache loader processes maintain the disk cache.

  • worker_processes auto; starts one worker per CPU core. Workers are single-threaded event loops using epoll (Linux) or kqueue; they never block on network I/O.
  • worker_connections caps simultaneous connections per worker, counting both client and upstream connections. So a proxy with 4 workers and 1024 connections each handles about 2,000 proxied clients, not 4,096.
  • Each connection is a file descriptor, so raise worker_rlimit_nofile and system limits accordingly.
  • Blocking disk reads can stall a worker; aio threads offloads them to a thread pool.

By default the kernel wakes workers to accept connections; listen ... reuseport gives each worker its own socket so the kernel spreads connections evenly.

On reload (SIGHUP) the master validates the new configuration, opens new sockets if needed, starts new workers and asks old ones to shut down gracefully; old workers stop accepting and finish their in-flight requests, which for long WebSockets can take a long time (worker_shutdown_timeout bounds it). A binary upgrade (SIGUSR2) starts a new master with the new executable alongside the old one, so even upgrades lose no connections.

Example

worker_processes auto;            # one per core
worker_rlimit_nofile 65535;       # fd limit per worker
worker_shutdown_timeout 30s;      # cap graceful shutdown on reload

events {
    worker_connections 16384;     # clients + upstream connections, per worker
}

http {
    server {
        listen 80 reuseport;      # per-worker listening sockets
        location / { return 200 "ok\n"; }
    }
}
# ps -o pid,user,cmd -C nginx
#   PID USER   CMD
#   812 root   nginx: master process /usr/sbin/nginx
#   813 nginx  nginx: worker process
#   814 nginx  nginx: worker process

Remember

  • Master (root) manages; workers (unprivileged) run event loops
  • worker_connections counts client and upstream connections per worker
  • Reload starts new workers and drains old ones; SIGUSR2 upgrades the binary live

Watch out for

  • Sizing capacity as workers x worker_connections clients for a proxy; each proxied request uses two connections.

Check yourself

  1. 4 workers, worker_connections 1024, every request proxied upstream. Roughly how many concurrent clients?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: nginx.org/en/docs/ngx_core_module.html#worker_connections