Grouping with defaultdict

collections · Essentials · 8 min

Grouping records by a key is everywhere: orders by customer, files by extension, edges by node.

A defaultdict is a dict that calls a factory function to create a value for a missing key on first access:

from collections import defaultdict
by_ext = defaultdict(list)
by_ext[".py"].append("app.py")   # list created automatically

The argument is the factory itself, not a value: pass list, set, int or a lambda, without calling it. Common choices:

  • defaultdict(list) groups items into lists.
  • defaultdict(set) groups unique items.
  • defaultdict(int) counts (though Counter is usually nicer).
  • defaultdict(lambda: "n/a") for any other default.

Any access with d[key] inserts the default, even if you're only reading. That can silently grow the dict; use key in d or d.get(key) for lookups that shouldn't create entries. get never calls the factory.

Compared with dict.setdefault(key, []).append(x), defaultdict is clearer and doesn't build an unused empty list on every call.

A defaultdict prints as defaultdict(<class 'list'>, {...}). Convert with dict(d) before returning it from an API or serialising, so callers don't get the auto-insert behaviour unexpectedly.

Example

from collections import defaultdict

files = ["app.py", "README.md", "test_app.py", "setup.cfg", "notes.md"]
by_ext = defaultdict(list)
for name in files:
    by_ext[name.rsplit(".", 1)[-1]].append(name)
print(dict(by_ext))
# {'py': ['app.py', 'test_app.py'], 'md': ['README.md', 'notes.md'], 'cfg': ['setup.cfg']}

graph = defaultdict(set)
for a, b in [("A", "B"), ("A", "C"), ("B", "C")]:
    graph[a].add(b)
    graph[b].add(a)
print(sorted(graph["C"]))           # ['A', 'B']

print(graph.get("Z"))               # None: get doesn't insert
print(graph["Z"], "Z" in graph)     # set() True: [] does insert

Remember

  • defaultdict(factory) creates missing values by calling the factory
  • Pass the factory uncalled: list, not list()
  • d[key] inserts on read; use in or get to check without inserting

Watch out for

  • Writing defaultdict([]); the argument must be a callable such as list.

Check yourself

  1. d = defaultdict(int); d['a'] += 1; x = d['b']. What is len(d)?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/collections.html#collections.defaultdict