Testing warnings and log output

pytest · Advanced · 10 min

Deprecations and log messages are part of a library's contract, and silent warnings hide upcoming breakage.

Warnings: with pytest.warns(DeprecationWarning, match="use new_api"): asserts that the block emits a matching warning, like pytest.raises for exceptions. The context object records all warnings for further checks. pytest also collects warnings raised anywhere during tests and shows a summary at the end.

Control them with the filterwarnings configuration or the @pytest.mark.filterwarnings("ignore:...") marker. A strong policy is filterwarnings = ["error"] in configuration: every unexpected warning fails the test, so you upgrade dependencies with deprecations fixed rather than discovering them when the feature is removed. Add targeted ignore entries for warnings you can't fix.

Logs: the caplog fixture captures logging records:

  • caplog.records is a list of LogRecords (.levelname, .getMessage(), .name).
  • caplog.messages gives just the formatted messages; caplog.text the full formatted output.
  • caplog.set_level(logging.DEBUG, logger="app") or with caplog.at_level(...) changes the level, which is necessary when the logger would otherwise filter out lower-level records.

Test logs only where they're a real requirement (an audit message, an error report). Asserting on every debug line makes tests brittle.

Example

# test_warn_log.py
import logging, warnings
import pytest

log = logging.getLogger("billing")

def old_total(xs):
    warnings.warn("old_total is deprecated; use total", DeprecationWarning)
    return sum(xs)

def refund(amount):
    if amount > 1000:
        log.warning("large refund: %s", amount)
    return -amount

def test_deprecation():
    with pytest.warns(DeprecationWarning, match="use total"):
        assert old_total([1, 2]) == 3

def test_large_refund_logged(caplog):
    caplog.set_level(logging.INFO, logger="billing")
    refund(5000)
    assert caplog.messages == ["large refund: 5000"]
    assert caplog.records[0].levelname == "WARNING"

Remember

  • pytest.warns(Category, match=...) asserts a warning is emitted
  • filterwarnings = ['error'] turns unexpected warnings into failures
  • caplog.records / caplog.messages capture logging output
  • Set the capture level when the logger would filter records out

Watch out for

  • Asserting on debug log text in many tests, so every wording change breaks the suite.

Check yourself

  1. Why configure filterwarnings = ['error']?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.pytest.org/en/stable/how-to/capture-warnings.html