Testing CLIs with CliRunner

Automated tests keep a CLI's behaviour and error messages stable as it grows.

click.testing.CliRunner runs a command in-process with isolated standard streams:

runner = CliRunner()
result = runner.invoke(cli, ["init", "demo"])

The Result object holds:

result.exit_code: 0 on success, 2 for usage errors, 1 for unhandled exceptions or click.Abort.
result.output: what a user would see in the terminal, stdout and stderr combined. result.stdout and result.stderr are available separately (since Click 8.2 both are always captured).
result.exception and result.exc_info: an unhandled exception raised by your code. By default invoke catches it, so a test can silently pass with a crashed command unless you check exit_code. Pass catch_exceptions=False to let it propagate and get a real traceback in pytest.

Other options:

input="yes\n" feeds stdin, for prompts and confirmations.
env={"APP_TOKEN": "x"} sets environment variables for the run.
obj= passes a context object, letting you inject fakes.

For files, use pytest's tmp_path fixture and pass absolute paths. Click 8.5 deprecated CliRunner.isolated_filesystem() because it changes the process-wide working directory.

CliRunner swaps the process-global sys.stdin/sys.stdout, so it is not safe to use from several threads at once; run tests in parallel with processes (pytest-xdist).

Example

import click
from click.testing import CliRunner

@click.command()
@click.option("--count", type=int, default=1)
@click.option("--name", prompt="Your name")
def hello(count, name):
    if name == "boom":
        raise RuntimeError("unexpected")
    for _ in range(count):
        click.echo(f"Hello {name}")

runner = CliRunner()
r = runner.invoke(hello, ["--count", "2"], input="Ada\n")
print(r.exit_code, repr(r.output))  # 0 'Your name: Ada\nHello Ada\nHello Ada\n'

r = runner.invoke(hello, ["--count", "x", "--name", "Ada"])
print(r.exit_code, "Invalid value" in r.output)   # 2 True

r = runner.invoke(hello, ["--name", "boom"])
print(r.exit_code, type(r.exception).__name__)    # 1 RuntimeError

Remember

runner.invoke(cmd, args) returns exit_code, output, stdout, stderr, exception
Always assert exit_code; use catch_exceptions=False for real tracebacks
Use input=, env= and obj= to drive prompts, configuration and fakes

Watch out for

Checking only the output and missing that the command crashed (exit_code 1 with result.exception set).

Check yourself

A command raises ValueError inside runner.invoke(cmd, []) with default settings. What happens?

Official docs: click.palletsprojects.com/en/stable/testing/

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks