Layered lookups with ChainMap

collections · Advanced · 9 min

Settings from defaults, files, environment and command line are naturally layered.

A ChainMap groups several mappings into one view. Lookups search the maps in order and return the first hit:

config = ChainMap(cli_args, env_vars, file_settings, defaults)
config["port"]   # first map that has "port"

Nothing is copied: the ChainMap holds references, so changes to the underlying dicts show through immediately. That makes it cheaper and more live than merging dicts with {**defaults, **file, **env}.

Writes, updates and deletes affect only the first mapping (maps[0]). That's useful for scopes: child = config.new_child() adds a new empty dict at the front, so assignments in the child don't touch the parents, and child.parents gives the chain without the first map. This mirrors how variable scopes in interpreters work.

cm.maps is the list of mappings, which you can inspect or modify.

Gotchas:

  • Deleting a key that exists only in a later map raises KeyError, since deletes only touch the first map.
  • len() and iteration compute the union of keys across all maps, which costs time for big maps.
  • For a one-off merge, d1 | d2 (3.9+) is simpler and produces an ordinary dict.

Example

import os
from collections import ChainMap

defaults = {"host": "localhost", "port": 8000, "debug": False}
file_cfg = {"port": 9000}
env_cfg = {k[4:].lower(): v for k, v in os.environ.items() if k.startswith("APP_")}
cli = {"debug": True}

config = ChainMap(cli, env_cfg, file_cfg, defaults)
print(config["port"], config["debug"], config["host"])   # 9000 True localhost

file_cfg["host"] = "db.internal"                         # changes show through
print(config["host"])                                    # db.internal

scope = config.new_child()
scope["port"] = 1234                                     # written to the new layer
print(scope["port"], config["port"])                     # 1234 9000
print(dict(scope.parents) == dict(config))               # True
print(sorted(config))      # ['debug', 'host', 'port']

Remember

  • ChainMap searches several mappings in order without copying them
  • Writes and deletes affect only the first mapping
  • new_child() adds a scope layer; parents drops it

Watch out for

  • Expecting del chain[key] to remove a key that lives in a later mapping.

Check yourself

  1. cm = ChainMap({}, {'a': 1}); cm['a'] = 2. What does the second dict contain afterwards?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/collections.html#collections.ChainMap

Builds on

Unlocks

Nothing builds on this yet.