TopolearnWordNet adds lexical knowledge (meanings, synonyms, is-a hierarchies) to rule-based NLP, query expansion and data augmentation.
WordNet is a lexical database of English that groups words into synsets: sets of synonyms expressing one meaning. Access it with from nltk.corpus import wordnet as wn (data package wordnet).
wn.synsets("bank")lists every sense, as names likebank.n.01(lemma, POS, sense number). Filter withpos=wn.VERB.syn.definition()andsyn.examples()explain a sense.syn.lemmas()gives the words in the synset;lemma.name()the text, andlemma.antonyms()opposites.
Synsets are linked by relations:
- hypernyms (more general: dog to canine) and hyponyms (more specific)
- meronyms and holonyms (part and whole: tree has trunk)
syn.hypernym_paths()gives the chains up to the rootentity.n.01, andlowest_common_hypernyms(other)the closest shared ancestor
Similarity measures compare senses through the hierarchy: path_similarity (0 to 1, based on the shortest path), wup_similarity (Wu-Palmer, based on depth of the common ancestor), and information-content measures such as res_similarity that need corpus counts.
Word sense is the hard part: a word has several synsets, and picking the right one needs context. nltk.wsd.lesk(context_tokens, word) is a simple baseline for that. With the Open Multilingual WordNet data (omw-1.4, or omw-2.0 in the newest releases), lemma_names(lang="fra") gives words in other languages.
Example
from nltk.corpus import wordnet as wn
senses = wn.synsets("bank")
print(len(senses) > 10, senses[0].name()) # True bank.n.01
print(wn.synset("bank.n.02").definition())
# a financial institution that accepts deposits and channels the money ...
dog = wn.synset("dog.n.01")
print(dog.hypernyms()) # [Synset('domestic_animal.n.01'), Synset('canine.n.02')]
print([l.name() for l in wn.synset("good.a.01").lemmas()[0].antonyms()]) # ['bad']
cat, car = wn.synset("cat.n.01"), wn.synset("car.n.01")
print(round(dog.path_similarity(cat), 2), round(dog.path_similarity(car), 2))
# 0.2 0.08 (dog is closer to cat than to car)
print(round(dog.wup_similarity(cat), 2)) # 0.86
print(dog.lowest_common_hypernyms(cat)) # [Synset('carnivore.n.01')]
print(dog.lemma_names("fra")) # ['canis_familiaris', 'chien'] (needs omw data)Remember
- Synsets are meanings; a word usually has several
- definition(), examples(), lemmas() describe a synset
- Hypernyms/hyponyms form an is-a hierarchy you can walk
- path_similarity and wup_similarity compare senses, not words
Watch out for
- Taking
wn.synsets(word)[0]as the meaning without considering context.