TopolearnFine-tuning NER or text classification on your own data is how spaCy goes from generic to domain-accurate.
spaCy v3 trains from a config file that fully describes the pipeline, model architectures and hyperparameters, so experiments are reproducible.
1. Data. Training examples are Doc objects carrying the gold annotations (entities, categories, tags), stored in the binary `DocBin` format: build Docs, add them to DocBin(), and to_disk("train.spacy"). python -m spacy convert turns CoNLL, IOB and JSON formats into .spacy files. Create a separate dev.spacy for evaluation.
2. Config. python -m spacy init config config.cfg --lang en --pipeline ner --optimize efficiency writes a complete config. Its sections cover [paths], [nlp] (language and pipeline), [components] (each component's model architecture), [training] (optimizer, batching, dropout, patience) and [initialize]. Values can be overridden on the command line: --paths.train train.spacy. Use --optimize accuracy or --gpu for transformer-based configs.
3. Train. python -m spacy train config.cfg --output ./out --paths.train train.spacy --paths.dev dev.spacy. It prints losses and scores each evaluation step and saves model-best and model-last.
4. Evaluate and use. python -m spacy evaluate out/model-best dev.spacy reports precision, recall and F-score per label; spacy.load("out/model-best") loads it.
python -m spacy debug data config.cfg catches data problems early: misaligned entity spans, unseen labels, class imbalance. To update an existing pipeline instead of training from scratch, source components from it in the config. A few hundred good examples per label is a reasonable start.
Example
import spacy
from spacy.tokens import DocBin
nlp = spacy.blank("en")
TRAIN = [
("Order a WidgetPro for Ana", [(8, 17, "PRODUCT"), (22, 25, "PERSON")]),
("The GizmoMax broke again", [(4, 12, "PRODUCT")]),
]
db = DocBin()
for text, spans in TRAIN:
doc = nlp.make_doc(text)
ents = [doc.char_span(s, e, label=l) for s, e, l in spans]
assert None not in ents, "span not aligned to tokens"
doc.ents = ents
db.add(doc)
db.to_disk("train.spacy")
print(len(DocBin().from_disk("train.spacy"))) # 2
# Then, in a shell:
# python -m spacy init config config.cfg --lang en --pipeline ner --optimize efficiency
# python -m spacy debug data config.cfg --paths.train train.spacy --paths.dev dev.spacy
# python -m spacy train config.cfg --output ./out \
# --paths.train train.spacy --paths.dev dev.spacy
# python -m spacy evaluate out/model-best dev.spacy
# nlp = spacy.load("out/model-best")Remember
- Gold annotations live on Docs saved in DocBin (.spacy) files
- spacy init config generates a complete, reproducible config
- spacy train saves model-best and model-last; spacy evaluate scores them
- spacy debug data catches misaligned spans and label problems
Watch out for
- Creating entity spans whose character offsets don't align with token boundaries: char_span returns None and the example is broken.