TopolearnSpawn mechanics explain performance at scale, memory errors in big processes, and Windows quoting bugs.
On POSIX, the classic way to start a program is fork (clone the current process) then exec (replace the clone with the new program). subprocess does this in C (_posixsubprocess), running the minimal setup between the two: redirect descriptors, change directory, set the session, drop privileges.
Forking a large process used to be expensive and could fail with ENOMEM under strict memory overcommit, even though the memory is shared copy-on-write. So CPython picks faster primitives when it can:
- posix_spawn on Linux and macOS, when the arguments allow it (for example
close_fds=Falseand nocwd,preexec_fnor session options), - vfork on Linux (3.10+), which shares memory with the parent until exec, avoiding page-table copies; used whenever it is safe.
Exec failures (a missing binary, a bad cwd) are reported back to the parent through a dedicated pipe and re-raised as FileNotFoundError and friends, which is why you get a proper exception rather than exit code 127.
On Windows, CreateProcess takes a single command-line string. subprocess converts your list with list2cmdline, following the MS C runtime quoting rules; programs that parse their command line differently (notably cmd.exe) may see different arguments.
Example
import subprocess, sys, time
print(subprocess.list2cmdline(["prog", "a b", 'say "hi"']))
# prog "a b" "say \"hi\"" (Windows quoting rules)
N = 50
t = time.perf_counter()
for _ in range(N):
subprocess.run(["true"])
print(f"{(time.perf_counter() - t) / N * 1000:.2f} ms per spawn") # ~1 ms
try:
subprocess.run(["true"], cwd="/no/such/dir")
except FileNotFoundError as e:
print("raised in parent:", e.strerror) # raised in parent: No such file or directoryRemember
- POSIX spawning is fork/vfork + exec, or posix_spawn when options allow
- Exec errors come back through a pipe and are raised in the parent
- Windows gets one command-line string built by list2cmdline
Watch out for
- Assuming Windows programs receive your list elements exactly; quoting depends on how the target parses its command line.