Loaders, suites, runners and results

Custom runners, reporting and test selection all build on these four pieces.

python -m unittest is assembled from small parts you can use directly:

TestLoader finds tests: loadTestsFromTestCase(cls), loadTestsFromModule(mod), loadTestsFromName("pkg.mod.Class.test"), and discover(start_dir, pattern). testMethodPrefix and testNamePatterns control which methods count.
TestSuite is a composite: a list of tests and nested suites, run in order. Each TestCase instance represents exactly one test method.
TextTestRunner runs a suite and prints progress. Options: verbosity, failfast, buffer, stream, resultclass, and durations (3.12).
TestResult collects outcomes: failures, errors, skipped, expectedFailures, unexpectedSuccesses, and wasSuccessful(). Subclass it to produce JSON or JUnit XML, or to hook startTest / stopTest.

The `load_tests` protocol: if a test module or a package's __init__.py defines load_tests(loader, tests, pattern), the loader calls it instead of collecting the default way, so you can add doctests (doctest.DocTestSuite) or filter tests.

Test methods run in alphabetical order within a class (by sortTestMethodsUsing), and suites keep insertion order. Tests should never depend on order anyway.

Example

import io, unittest

class TestA(unittest.TestCase):
    def test_ok(self): pass
    def test_fail(self): self.assertEqual(1, 2)

class TestB(unittest.TestCase):
    @unittest.skip("not today")
    def test_skip(self): pass

class CountingResult(unittest.TextTestResult):
    def startTest(self, test):
        super().startTest(test)
        print("starting", test.id().split(".", 1)[1])

loader = unittest.TestLoader()
suite = unittest.TestSuite([loader.loadTestsFromTestCase(TestA),
                            loader.loadTestsFromTestCase(TestB)])
print(suite.countTestCases())                         # 3
runner = unittest.TextTestRunner(stream=io.StringIO(), resultclass=CountingResult)
result = runner.run(suite)
# starting TestA.test_fail / starting TestA.test_ok / starting TestB.test_skip
print(len(result.failures), len(result.skipped), result.wasSuccessful())  # 1 1 False

Remember

TestLoader collects, TestSuite groups, TextTestRunner runs, TestResult records
Subclass TestResult for custom reporting
load_tests(loader, tests, pattern) customises a module's or package's tests

Watch out for

Writing tests that rely on alphabetical method order to share state.

Check yourself

A module defines load_tests(loader, tests, pattern). When is it used?

Official docs: docs.python.org/3/library/unittest.html#load-tests-protocol

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks