Subclassing dict vs UserDict

collections · Expert · 11 min

Custom dictionaries that silently skip your validation are a common, hard-to-spot bug.

Subclassing dict directly looks natural, but the built-in methods are implemented in C and don't call your overrides. If you override __setitem__ to normalise keys, then dict.update(), the constructor dict(...), setdefault() and |= all bypass it. Likewise, overriding __getitem__ doesn't affect get().

Options:

  • collections.UserDict wraps a real dict in self.data, and its methods are written in Python on top of your __getitem__, __setitem__ and __delitem__. Override those three and everything (update, constructor, setdefault) stays consistent. The cost is some speed.
  • collections.abc.MutableMapping gives the same consistency with your own storage, as in the abc lesson.
  • dict subclass with `__missing__`: dict's __getitem__ calls self.__missing__(key) when the key is absent. That hook is honoured, and it's how defaultdict and Counter work. Note get() doesn't call __missing__.

UserList and UserString play the same role for lists and strings.

Subclassing dict is still fine when you only add new methods or __missing__, and when speed matters. When you change how existing operations behave, use UserDict or MutableMapping.

Example

from collections import UserDict

class LowerDict(dict):
    def __setitem__(self, key, value):
        super().__setitem__(key.lower(), value)

class LowerUserDict(UserDict):
    def __setitem__(self, key, value):
        super().__setitem__(key.lower(), value)

a = LowerDict(Name="x")               # constructor bypasses __setitem__
a.update({"Age": 1})                  # so does update
a["City"] = "Oslo"
print(sorted(a))                      # ['Age', 'Name', 'city']

b = LowerUserDict(Name="x")
b.update({"Age": 1})
b["City"] = "Oslo"
print(sorted(b))                      # ['age', 'city', 'name']

class Fallback(dict):
    def __missing__(self, key):
        return f"<{key}>"
f = Fallback(a=1)
print(f["a"], f["zzz"], f.get("zzz"))  # 1 <zzz> None

Remember

  • Built-in dict methods don't call your overridden __setitem__/__getitem__
  • UserDict and MutableMapping route every operation through your methods
  • __missing__ is honoured by d[key] but not by get

Watch out for

  • Overriding __setitem__ on a dict subclass for validation and assuming update() uses it.

Check yourself

  1. class D(dict): def __setitem__(self, k, v): super().__setitem__(k, v * 2). What is D(a=1)['a']?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/collections.html#collections.UserDict