Logging in libraries

logging · Advanced · 9 min

A library that configures logging overrides its users' choices and floods their output.

Applications configure logging; libraries only emit. Rules for library code:

  • Use logging.getLogger(__name__) in each module, so users can tune yourlib or yourlib.http independently.
  • Never call basicConfig, add handlers to the root, or set levels in import-time code.
  • Optionally add logging.NullHandler() to your top-level package logger. It does nothing, but it means "this library has a handler", so when the app hasn't configured logging, your warnings don't fall through to lastResort on stderr. Python's own docs recommend it for libraries.
  • Use DEBUG generously for internal details and WARNING sparingly, only for things the app owner should act on.
  • Don't log and raise the same error; raise it and let the app decide.

From the application side, quiet a chatty dependency with logging.getLogger("lib").setLevel(logging.WARNING).

logging.captureWarnings(True) routes the warnings module (for example DeprecationWarnings) into the logger named py.warnings, so they reach your handlers and log aggregator instead of bare stderr.

Example

import logging, sys, warnings

# --- in yourlib/__init__.py ---
lib_log = logging.getLogger("yourlib")
lib_log.addHandler(logging.NullHandler())

def connect():
    lib_log.debug("opening socket")
    lib_log.warning("using insecure default port")
    warnings.warn("connect() is deprecated", DeprecationWarning)

# --- in the application ---
logging.basicConfig(stream=sys.stdout, format="%(name)s: %(message)s")
logging.captureWarnings(True)             # warnings -> 'py.warnings' logger
connect()
# yourlib: using insecure default port
# py.warnings: .../app.py:10: DeprecationWarning: connect() is deprecated
#   warnings.warn("connect() is deprecated", DeprecationWarning)
logging.getLogger("yourlib").setLevel(logging.ERROR)  # quiet the library

Remember

  • Libraries call getLogger(__name__) and never configure handlers or levels
  • A NullHandler on the package logger prevents lastResort output
  • captureWarnings(True) sends warnings into logging

Watch out for

  • Calling logging.basicConfig at import time in a library, hijacking the application's configuration.

Check yourself

  1. What does adding logging.NullHandler() to a library's top-level logger do?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/howto/logging.html#configuring-logging-for-a-library