Atomic groups and possessive quantifiers

Regular expressions · Advanced · 12 min

They are the precise tool for making some patterns fail fast instead of exploring thousands of useless paths.

When a backtracking engine fails later in the pattern, it returns to earlier quantifiers and tries giving back characters. Usually that is what makes greedy matching work. Sometimes you know giving back can never help, and you want to forbid it.

  • An atomic group (?>...) matches like a normal group, but once it has matched, the engine throws away its backtracking positions. If the rest of the pattern fails, the group won't try a shorter alternative.
  • A possessive quantifier *+, ++, ?+, {m,n}+ is shorthand for an atomic group around a greedy quantifier: \d++ equals (?>\d+).

Example: \d+0 matches 1230 because \d+ gives back the final 0. \d++0 never matches, since \d++ keeps all digits. That shows the semantic difference: atomic constructs can change whether a pattern matches, not just how fast.

They shine at preventing catastrophic backtracking: in "(?:[^"\\]++|\\.)*+" no input can make the engine retry the same characters in different ways.

Support: PCRE, Perl, Java, .NET, Ruby, and Python 3.11+. JavaScript has neither; a lookahead with a backreference (?=(\d+))\1 emulates an atomic group there.

Example

import re   # Python 3.11+

print(re.fullmatch(r"\d+0", "1230"))          # <re.Match ...match='1230'>
print(re.fullmatch(r"\d++0", "1230"))         # None: \d++ won't give back the 0
print(re.fullmatch(r"(?>\d+)0", "1230"))      # None: same thing as an atomic group

# a quoted string with escapes, no possible double-matching of characters
qstr = re.compile(r'"(?:[^"\\]++|\\.)*+"')
print(qstr.findall(r'say "hi \"there\"" and "bye"'))
# ['"hi \\"there\\""', '"bye"']

# emulating an atomic group in JavaScript-compatible syntax
print(re.fullmatch(r"(?=(\d+))\1[0]", "1230"))  # None, like (?>\d+)0

Remember

  • (?>...) and possessive quantifiers never give back what they matched
  • They can change whether a pattern matches, not only its speed
  • Available in PCRE, Java, .NET and Python 3.11+, not JavaScript

Watch out for

  • Making a quantifier possessive when the rest of the pattern needs some of those characters, so it can never match.

Check yourself

  1. Does a++b match the string aaab?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: www.pcre.org/current/doc/html/pcre2pattern.html