TopolearnLong builds, servers and log tailers need live output and control while the process runs.
run waits for completion. Popen starts the process and returns immediately, giving you a handle for live interaction. It takes the same arguments as run (except input, timeout, check and capture_output).
with subprocess.Popen(cmd, stdout=subprocess.PIPE, text=True) as p:
for line in p.stdout:
print("got", line.rstrip())
print(p.returncode)Key methods and attributes:
p.stdin,p.stdout,p.stderr: file objects when you usedPIPE.p.poll(): returns the exit code if finished, elseNone(non-blocking).p.wait(timeout=None): block until exit.p.communicate(input, timeout): send input, read all output, wait.p.pid,p.returncode,p.terminate(),p.kill().
Use it as a context manager: on exit it closes the pipes and waits for the process, so you don't leave zombie processes behind.
Output buffering matters: many programs buffer stdout fully when it's a pipe rather than a terminal, so lines arrive in bursts. For Python children, pass -u or set PYTHONUNBUFFERED=1; other tools often have a --line-buffered flag.
In asyncio code, use asyncio.create_subprocess_exec instead, which gives you awaitable streams.
Example
import subprocess, sys, time
child = "import time\nfor i in range(3):\n print('step', i, flush=True); time.sleep(0.2)"
t0 = time.perf_counter()
with subprocess.Popen([sys.executable, "-c", child],
stdout=subprocess.PIPE, text=True) as p:
print("started pid", p.pid > 0, "poll:", p.poll()) # started pid True poll: None
for line in p.stdout: # arrives live
print(f"{time.perf_counter() - t0:.1f}s {line.rstrip()}")
print("exit code", p.returncode)
# 0.0s step 0
# 0.2s step 1
# 0.4s step 2
# exit code 0Remember
- Popen starts a process without waiting and gives you live pipes
- Iterate p.stdout for line-by-line streaming
- Use Popen as a context manager so it is always waited for
Watch out for
- Children that buffer output when writing to a pipe, so 'live' lines only show up at the end.