Topolearn

Learn NLTK

The Natural Language Toolkit: classic NLP algorithms, corpora and teaching tools

16 lessons · 3 hours · 4 levels · covers NLTK 3.9+

NLTK is a Python library of classic natural language processing building blocks: tokenizers, stemmers, taggers, parsers, WordNet, sentiment lexicons and many text corpora. It is widely used for teaching, research prototypes and lightweight text preprocessing. Learning it gives you a clear, hands-on understanding of how NLP pipelines work under the hood.

Sign in free to track progressOfficial docs ↗

Before you start

NLTK builds on these; the tree links to the exact lessons you need.

PythonEssentials level

How deep do you want to go?

1. EssentialsRead and write simple code, do the everyday basics5 lessons · 46 min in total
2. PracticalUse it productively at work, with the usual idioms9 lessons · 1 hour in total
3. AdvancedThe less common features, performance and trade-offs13 lessons · 2 hours in total
4. ExpertInternals and the edge cases experts know16 lessons · 3 hours in total

The NLTK tree

NLTKInstalling NLTK and its dataInstalling NLTK andits dataSentence and word tokenizationSentence and wordtokenizationFrequency counts and stopwordsFrequency countsand stopwordsBuilt-in corpora and corpus readersBuilt-in corpora andcorpus readersStemming with Porter and SnowballStemming with Porterand SnowballPart-of-speech taggingPart-of-speechtaggingLemmatization with WordNetLemmatization withWordNetN-grams and collocationsN-grams andcollocationsSentiment analysis with VADERSentiment analysiswith VADERModules and importsModules and importsVirtual environments and pipVirtual environmentsand pipStrings and f-stringsStrings and f-stringsComprehensionsComprehensionscollections and other stdlib gemscollections and otherstdlib gemsReading and writing filesReading and writingfilesfor and while loopsfor and while loopsDictionariesDictionaries
Essentials and Practical shown (9 of 16 lessons); the full list is below. Click a lesson to read it.

All 16 lessons, in order

Setup and corpora

  1. Installing NLTK and its data · Essentials · 8 min

    Install the library, download the data packages (tokenizer models, corpora, lexicons) and control where NLTK looks for them.

  2. Built-in corpora and corpus readers · Essentials · 10 min

    Explore Gutenberg, Brown, movie_reviews and others through corpus readers, and load your own files the same way.

Tokens and counts

  1. Sentence and word tokenization · Essentials · 10 min

    Split text into sentences with sent_tokenize and into words and punctuation with word_tokenize.

  2. Frequency counts and stopwords · Essentials · 10 min

    Count tokens with FreqDist, remove stopwords, and find the most characteristic words of a text.

  3. N-grams and collocations · Practical · 10 min

    Build bigrams and n-grams, and find statistically significant phrases with collocation finders and association measures.

  4. Training Punkt and building custom tokenizers · Expert · 12 min

    Adapt sentence splitting to your domain by training Punkt or adding abbreviations, and compose regex and multi-word tokenizers.

Normalising words

  1. Stemming with Porter and Snowball · Essentials · 8 min

    Reduce words to crude stems with PorterStemmer, SnowballStemmer and LancasterStemmer, and know their limits.

  2. Lemmatization with WordNet · Practical · 10 min

    Map words to dictionary forms with WordNetLemmatizer, and pass the part of speech to get correct results.

Tagging and structure

  1. Part-of-speech tagging · Practical · 10 min

    Tag tokens with nltk.pos_tag, read Penn Treebank tags, and use the universal tagset for simpler categories.

  2. Chunking and named entities · Advanced · 12 min

    Group tagged words into phrases with RegexpParser grammars and find named entities with ne_chunk.

  3. Context-free grammars and parsing · Advanced · 15 min

    Write a context-free grammar, parse sentences into trees with chart parsers, and see why ambiguity makes parsing hard.

Meaning and sentiment

  1. Sentiment analysis with VADER · Practical · 10 min

    Score the sentiment of short texts with VADER's lexicon and rules, and interpret its compound score.

  2. WordNet: synsets and word relations · Advanced · 12 min

    Navigate WordNet synsets, definitions, synonyms, hypernyms and similarity scores.

Models and evaluation

  1. Text classification with Naive Bayes · Advanced · 15 min

    Turn documents into feature dicts, train NLTK's NaiveBayesClassifier, evaluate accuracy and inspect informative features.

  2. N-gram language models with nltk.lm · Expert · 15 min

    Train MLE, Laplace and Kneser-Ney n-gram models, handle padding and unknown words, generate text and compute perplexity.

  3. Evaluation: metrics, agreement and BLEU · Expert · 12 min

    Measure NLP systems with precision, recall, F1, confusion matrices, edit distance, inter-annotator agreement and BLEU.