TopolearnProduction NLP processes millions of documents; storing results and parallelising correctly matter as much as the model.
Pipelines: nlp.to_disk(path) writes a directory with config.cfg, meta.json, the vocab, tokenizer settings and each component's weights; spacy.load(path) restores it. spacy package turns that directory into an installable Python package for deployment. Custom components and factories must be importable when loading, or spacy.load fails with an unknown factory error.
Docs: avoid pickling individual Docs. Each pickle includes vocab strings, so pickling many Docs is large and slow. DocBin(attrs=[...], store_user_data=True) stores many Docs compactly, sharing strings: db.add(doc), then db.to_bytes() or to_disk(), and later list(db.get_docs(nlp.vocab)). With store_user_data, custom ._ values stored as attributes travel too.
Often you don't need the Doc at all: extract what you need (entity tuples, lemmas, JSON via doc.to_json()) inside the worker and return plain data. That is the cheapest thing to move between processes.
Scaling:
nlp.pipe(texts, n_process=4, batch_size=...)uses multiprocessing; each process loads its own model copy, so memory multiplies.- Disable components you don't need before forking.
- Set thread counts for underlying math libraries to 1 per process to avoid oversubscription.
- On a GPU, use a single process with a larger batch.
- For clusters, run the same function in Spark, Ray or Dask workers, loading
nlponce per worker rather than per task.
Example
import spacy
from spacy.tokens import DocBin, Doc
Doc.set_extension("source", default=None, force=True)
nlp = spacy.load("en_core_web_sm", exclude=["parser"])
texts = ["Ada met Alan in London.", "Grace worked at Harvard.", "Paris in May."]
db = DocBin(store_user_data=True)
for i, doc in enumerate(nlp.pipe(texts)):
doc._.source = f"doc-{i}"
db.add(doc)
data = db.to_bytes() # compact, shared strings
print(len(data) < 10_000) # True
restored = list(DocBin().from_bytes(data).get_docs(nlp.vocab))
print(restored[1]._.source, [e.text for e in restored[1].ents]) # doc-1 ['Grace', 'Harvard']
def extract(texts): # return plain data, not Docs
return [[(e.text, e.label_) for e in d.ents]
for d in nlp.pipe(texts, n_process=2, batch_size=64)]
if __name__ == "__main__":
print(extract(texts * 2)[0]) # [('Ada', 'PERSON'), ('Alan', 'PERSON'), ('London', 'GPE')]
nlp.to_disk("my_pipeline")
print(spacy.load("my_pipeline").pipe_names[-1]) # nerRemember
- nlp.to_disk / spacy.load(path) save and restore whole pipelines
- DocBin stores many Docs compactly with shared strings; avoid pickling Docs
- Return plain extracted data from workers when you can
- n_process multiplies model memory; use one process with big batches on GPU
Watch out for
- Loading the spaCy model inside the per-item function of a parallel job, reloading it thousands of times.