File descriptors and secure spawning

subprocess · Expert · 12 min

Leaked descriptors and over-privileged children are subtle security and reliability bugs.

When a child starts, it may inherit open file descriptors from the parent: sockets, files, pipe ends. Leaks cause real bugs: a child holding a listening socket keeps the port busy after your server restarts, and a child holding a pipe's write end means the reader never sees EOF.

Python defaults are safe: descriptors created by Python are non-inheritable (PEP 446), and close_fds=True (the default) closes everything except 0, 1 and 2 in the child. To deliberately hand over specific descriptors, list them in pass_fds=(fd,); they are made inheritable in the child only.

Other POSIX controls on Popen and run:

  • user=, group=, extra_groups= (3.9+): drop privileges when running as root.
  • umask= (3.9+): file creation mask for the child.
  • preexec_fn=: runs Python code in the child before exec. Avoid it: it is not safe in the presence of threads. Use the dedicated parameters (start_new_session, process_group, user) instead.
  • executable=: run a different binary than args[0], which becomes the child's argv[0].

Security checklist: list args (no shell), absolute paths for sensitive tools, a minimal env, no secrets in arguments (they are visible in ps), and stdin=DEVNULL.

Example

import os, subprocess, sys

r, w = os.pipe()
print(os.get_inheritable(w))                  # False (PEP 446)

child = "import os, sys; fd = int(sys.argv[1]); os.write(fd, b'via fd'); os.close(fd)"
subprocess.run([sys.executable, "-c", child, str(w)], pass_fds=(w,), check=True)
os.close(w)
print(os.read(r, 100))                        # b'via fd'

res = subprocess.run([sys.executable, "-c", child, str(r)],
                     capture_output=True, text=True)
print(res.stderr.strip().splitlines()[-1])    # OSError: [Errno 9] Bad file descriptor
os.close(r)

Remember

  • Python-created descriptors are non-inheritable and close_fds=True by default
  • pass_fds hands specific descriptors to the child
  • Prefer user=, umask=, start_new_session over preexec_fn

Watch out for

  • Passing secrets on the command line, where any user can read them with ps.

Check yourself

  1. Why is preexec_fn discouraged?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/subprocess.html#popen-constructor