Greedy vs lazy quantifiers

re · Practical · 9 min

Greedy matching is behind the classic 'my pattern swallowed everything' bug.

Quantifiers are greedy: .* first grabs everything it can, then the engine backtracks one character at a time until the rest of the pattern can match. So on <b>bold</b> and <b>more</b>, the pattern r"<b>.*</b>" matches from the first <b> to the last </b>.

Adding ? makes a quantifier lazy (non-greedy): *?, +?, ??, {m,n}?. A lazy quantifier takes as little as possible and extends only when needed, so r"<b>.*?</b>" matches each bold element separately.

Often a negated character class is better than a lazy dot: r"<b>[^<]*</b>" states exactly what may appear inside, can't run past the closing tag, and backtracks less. Similarly, r'"[^"]*"' matches a quoted string.

Lazy doesn't mean "shortest possible match overall": the engine still returns the leftmost match, then makes it as short as it can from that starting point.

Python 3.11 added possessive quantifiers (*+, ++, ?+) and atomic groups, which never give back what they matched. They're covered with backtracking performance.

For real HTML or XML, use a parser such as html.parser or BeautifulSoup; regex is fine for quick extraction from predictable snippets.

Example

import re

s = "<b>bold</b> and <b>more</b>"
print(re.findall(r"<b>.*</b>", s))     # ['<b>bold</b> and <b>more</b>']
print(re.findall(r"<b>.*?</b>", s))    # ['<b>bold</b>', '<b>more</b>']
print(re.findall(r"<b>([^<]*)</b>", s))  # ['bold', 'more']

q = 'say "hi" then "bye"'
print(re.findall(r'"(.*)"', q))        # ['hi" then "bye']
print(re.findall(r'"([^"]*)"', q))     # ['hi', 'bye']

print(re.search(r"a+?", "aaa").group())    # a
print(re.search(r"x.*?y", "xaxby").group())  # xaxby  (leftmost wins)

Remember

  • Greedy quantifiers take as much as possible, then backtrack
  • Add ? for lazy matching: .*?, +?
  • Negated classes like [^<]* are clearer and faster than lazy dots

Watch out for

  • Using .* between delimiters on text that contains several delimited parts.

Check yourself

  1. What does re.search(r'\((.*)\)', 'f(a) + g(b)').group(1) return?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: docs.python.org/3/library/re.html#regular-expression-syntax