TopolearnSentences and noun phrases are the natural units for summaries, search snippets and keyword extraction.
doc.sents yields one Span per sentence. In the trained English pipelines, sentence boundaries come from the dependency parser, which is accurate but means you need the parser enabled. Each token also has token.is_sent_start, and span.sent gives the sentence containing a span.
If you only need sentences, there are faster options:
- the rule-based `sentencizer` component splits on punctuation:
nlp = spacy.blank("en"); nlp.add_pipe("sentencizer") - the trained
sentercomponent (included but disabled in core pipelines) is faster than the parser
doc.sents is a generator, so wrap it in list() to index or count. Accessing it on a Doc with no sentence boundaries raises an error (E030), telling you to add a parser, senter or sentencizer.
doc.noun_chunks yields base noun phrases: a noun plus the words describing it, such as "the striped bats" or "a new dataset". They come from the dependency parse, need the parser and are available for languages with noun chunk rules (English, German, French and others). chunk.root is the head noun, and chunk.root.head the word the phrase attaches to, e.g. the verb it is the object of.
Noun chunks are a cheap way to get candidate keywords and topics, especially after filtering out pronouns and counting lemma forms of their roots.
Example
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("Dr. Lee released a new dataset. The striped bats love ripe mangoes. Really!")
sents = list(doc.sents)
print(len(sents), [s.text for s in sents][:2])
# 3 ['Dr. Lee released a new dataset.', 'The striped bats love ripe mangoes.']
for chunk in doc.noun_chunks:
print(f"{chunk.text:20} root={chunk.root.text:8} head={chunk.root.head.text}")
# Dr. Lee root=Lee head=released
# a new dataset root=dataset head=released
# The striped bats root=bats head=love
# ripe mangoes root=mangoes head=love
fast = spacy.blank("en")
fast.add_pipe("sentencizer") # rule-based, no model
print(len(list(fast("One. Two? Three!").sents))) # 3Remember
- doc.sents yields sentence Spans; in core pipelines the parser sets them
- Use the sentencizer or senter when you need only sentences, faster
- doc.noun_chunks gives base noun phrases from the parse
- chunk.root and chunk.root.head link a phrase to its verb
Watch out for
- Indexing doc.sents directly (doc.sents[0]); it's a generator, so use list(doc.sents).