TopolearnKnowing the run protocol explains import errors at collection, fixture teardown timing and how plugins and custom items fit in.
A pytest run has two phases. Collection walks the given paths starting at the Session node: Dir/Package collectors yield Module collectors for matching files, which import the module and yield Class collectors and Function items. Each node has a nodeid (tests/test_a.py::TestX::test_y) and keywords (markers, names) used by -k and -m. Parametrization expands one function into several items at this stage. An exception while importing a module is a collection error, reported as ERROR before any test runs.
You can add your own collectors via pytest_collect_file, for example turning YAML files into test items with a custom Item subclass implementing runtest().
Then runtestloop executes each item through the runtest protocol, three phases each producing a TestReport:
- setup: create the item's fixtures, highest scope first, reusing cached higher-scope ones.
- call: run the test function.
- teardown: finalize fixtures no longer needed. Function-scoped fixtures end now; module or session fixtures are torn down when the next item no longer needs them (the SetupState stack), which is why teardown of a module fixture appears after that module's last test.
An error during setup or teardown is reported as E (error), distinct from a failure in the call phase. pytest uses item order to minimize fixture setup, so higher-scoped parametrized fixtures may reorder tests to group by parameter.
Example
# conftest.py: collect *.yaml files as tests (custom collector + item)
import pytest, yaml
def pytest_collect_file(parent, file_path):
if file_path.suffix == ".yaml" and file_path.name.startswith("test"):
return YamlFile.from_parent(parent, path=file_path)
class YamlFile(pytest.File):
def collect(self):
for name, spec in yaml.safe_load(self.path.read_text()).items():
yield YamlItem.from_parent(self, name=name, spec=spec)
class YamlItem(pytest.Item):
def __init__(self, *, spec, **kwargs):
super().__init__(**kwargs)
self.spec = spec
def runtest(self):
if eval(self.spec["expr"]) != self.spec["equals"]:
raise AssertionError(f"{self.spec['expr']} != {self.spec['equals']}")
def reportinfo(self):
return self.path, 0, f"yaml: {self.name}"
# test_math.yaml: add: {expr: "1 + 1", equals: 2}
# $ pytest -v -> test_math.yaml::add PASSEDRemember
- Collection builds a node tree: Session, Dir/Package, Module, Class, Function items
- Each item runs setup, call and teardown, each producing a report
- Errors in setup/teardown show as E, failures in call as F
- Custom collectors and Items extend pytest to new test formats
Watch out for
- Doing expensive work at module import time, which runs during collection even for deselected tests.