Transformer and LLM pipelines

Transformers give state-of-the-art accuracy inside the same spaCy API, at a real cost in speed and hardware.

The trf pipelines replace the small CNN tok2vec with a transformer (RoBERTa-base for English) via the spacy-transformers package. The API stays the same: nlp = spacy.load("en_core_web_trf"), then doc.ents, token.dep_ and so on, with noticeably better accuracy on parsing and entities.

Costs to plan for:

Much slower on CPU; a GPU helps a lot. Call spacy.prefer_gpu() (or require_gpu()) before loading, with a CUDA-enabled install such as pip install spacy[cuda12x].
More memory. Use nlp.pipe with a moderate batch_size and process texts in streams.
Transformers see a limited window; spaCy splits long docs into overlapping spans and recombines the outputs.
No static word vectors: similarity isn't meaningful with these pipelines.

To train your own transformer-backed pipeline, generate a config with spacy init config ... --optimize accuracy --gpu. The [components.transformer] section names any Hugging Face model, for example bert-base-multilingual-cased or a domain model. Downstream components connect to it through a listener layer so they share one transformer pass.

`spacy-llm` is a separate package that adds components powered by large language models via prompts, for NER, text classification and more, without training data. It's useful for prototyping and bootstrapping annotations; costs, latency and output consistency usually push production towards a trained model.

Example

import spacy

using_gpu = spacy.prefer_gpu()          # False on CPU-only machines
print("GPU:", using_gpu)

try:
    nlp = spacy.load("en_core_web_trf")  # pip: spacy-transformers + the package
    print(nlp.pipe_names)
    # ['transformer', 'tagger', 'parser', 'attribute_ruler', 'lemmatizer', 'ner']
    texts = ["Ada Lovelace worked with Charles Babbage in London."] * 8
    for doc in nlp.pipe(texts, batch_size=4):
        pass
    print([(e.text, e.label_) for e in doc.ents])
    # [('Ada Lovelace', 'PERSON'), ('Charles Babbage', 'PERSON'), ('London', 'GPE')]
except OSError:
    print("install with: python -m spacy download en_core_web_trf")

# Training config for a transformer pipeline:
#   python -m spacy init config cfg.cfg --lang en --pipeline ner --optimize accuracy --gpu
# [components.transformer.model]
# name = "roberta-base"            <- any Hugging Face model name

Remember

trf pipelines use a transformer backbone with the same spaCy API
Expect slower CPU speed and more memory; prefer_gpu() helps
Downstream components share one transformer via listeners
spacy-llm adds prompt-based components for prototyping

Watch out for

Calling .similarity() with a trf pipeline, which has no static vectors.

Check yourself

You call spacy.prefer_gpu() after spacy.load('en_core_web_trf'). What happens?

Official docs: spacy.io/usage/embeddings-transformers

Example

Remember

Watch out for

Check yourself

Builds on

Unlocks