Parametrised tests with subTest

unittest · Practical · 8 min

Table-driven tests cover edge cases cheaply, and subTest keeps one failure from hiding the others.

Testing a function against a table of inputs is common. A plain loop has a drawback: the first failing assertion ends the test, so you only learn about one bad case, and the message doesn't say which input it was.

with self.subTest(**params): fixes both:

for raw, expected in cases:
    with self.subTest(raw=raw):
        self.assertEqual(parse(raw), expected)

A failure inside the block is recorded with the parameters ((raw='x')) and the loop continues, so every failing case is reported. The test as a whole fails if any subtest failed.

Notes:

  • Parameters are only labels for the report; pass whatever identifies the case.
  • -v output counts the method once; failures are listed per subtest.
  • Subtests can be nested; parameters combine.
  • setUp and tearDown run once for the whole method, not per subtest, so subtests must not depend on fresh fixtures.

pytest's @pytest.mark.parametrize generates separate tests instead. Recent pytest versions (9+) report unittest subtests natively; older ones need the pytest-subtests plugin.

Example

import unittest

def is_leap(year: int) -> bool:
    return year % 4 == 0 and year % 100 != 0      # bug: ignores the 400 rule

class TestLeap(unittest.TestCase):
    def test_years(self):
        cases = [(2024, True), (2023, False), (1900, False), (2000, True)]
        for year, expected in cases:
            with self.subTest(year=year):
                self.assertEqual(is_leap(year), expected)

unittest.main(exit=False)
# FAIL: test_years (__main__.TestLeap.test_years) (year=2000)
# AssertionError: False != True
# Ran 1 test ... FAILED (failures=1)

Remember

  • with self.subTest(param=value): labels a case and keeps the loop going
  • Every failing case is reported with its parameters
  • setUp/tearDown run once per method, not per subtest

Watch out for

  • Asserting in a loop without subTest, so one failure hides all later cases.

Check yourself

  1. Inside a loop over 10 cases with subTest, cases 3 and 7 fail. What is reported?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests