TopolearnRules give precise, explainable extraction for things models miss, and are often all you need.
The `Matcher` finds sequences of tokens described by patterns: a list of dicts, one per token, each mapping attributes to required values.
[{"LOWER": "machine"}, {"LOWER": "learning"}]
[{"POS": "ADJ", "OP": "*"}, {"POS": "NOUN"}]
[{"LIKE_NUM": True}, {"LOWER": {"IN": ["kg", "g"]}}]Attributes include TEXT, LOWER, LEMMA, POS, TAG, DEP, ENT_TYPE, IS_PUNCT, LIKE_NUM, SHAPE, LENGTH. Values can be exact or use IN, NOT_IN, REGEX, and comparisons such as {">=": 3}. "OP" adds quantifiers: ? optional, * zero or more, + one or more, ! negation, {n,m} ranges.
matcher = Matcher(nlp.vocab), matcher.add("LABEL", [pattern1, pattern2]), then matcher(doc) returns (match_id, start, end) tuples; nlp.vocab.strings[match_id] gives the label. With operators, overlapping matches appear; greedy="LONGEST" when adding, or spacy.util.filter_spans, keeps the longest.
The `PhraseMatcher` is built for long term lists such as product catalogues or gazetteers. Patterns are Docs: matcher.add("DRUG", [nlp.make_doc(t) for t in terms]). It scales to hundreds of thousands of terms. Pass attr="LOWER" for case-insensitive matching.
Patterns relying on POS, LEMMA or DEP need the corresponding trained components; LOWER and TEXT work even with spacy.blank. as_spans=True returns Span objects directly.
Example
import spacy
from spacy.matcher import Matcher, PhraseMatcher
nlp = spacy.load("en_core_web_sm")
doc = nlp("Order 3 kg of Green Tea and 250 g of green tea leaves. Ship 2 boxes.")
matcher = Matcher(nlp.vocab)
matcher.add("WEIGHT", [[{"LIKE_NUM": True}, {"LOWER": {"IN": ["kg", "g"]}}]])
matcher.add("ADJ_NOUN", [[{"POS": "ADJ", "OP": "+"}, {"POS": "NOUN"}]], greedy="LONGEST")
for match_id, start, end in matcher(doc):
print(nlp.vocab.strings[match_id], doc[start:end].text)
# WEIGHT 3 kg
# WEIGHT 250 g
# ADJ_NOUN green tea (relies on the tagger's POS predictions)
terms = ["green tea", "black tea", "oolong"]
pm = PhraseMatcher(nlp.vocab, attr="LOWER") # case-insensitive
pm.add("TEA", [nlp.make_doc(t) for t in terms])
print([s.text for s in pm(doc, as_spans=True)])
# ['Green Tea', 'green tea']Remember
- Matcher patterns are lists of per-token attribute dicts
- OP quantifiers (?, *, +, !) and IN/REGEX/comparisons add flexibility
- PhraseMatcher matches large term lists efficiently; attr='LOWER' ignores case
- Use greedy='LONGEST' or filter_spans to drop overlapping matches
Watch out for
- Writing POS- or LEMMA-based patterns and running them with spacy.blank, where those attributes are empty.