TopolearnChunking is one of the biggest levers on RAG quality, and the defaults are rarely optimal for your data.
A node parser (or text splitter) cuts documents into nodes. The default is `SentenceSplitter`: it fills chunks up to chunk_size tokens (1024 by default) while trying not to break sentences, and repeats chunk_overlap tokens (200 by default) between neighbouring chunks so an idea cut at a boundary is not lost.
splitter = SentenceSplitter(chunk_size=512, chunk_overlap=50)
nodes = splitter.get_nodes_from_documents(documents)To use it during indexing, pass it as a transformation: VectorStoreIndex.from_documents(docs, transformations=[splitter]).
Choosing a size:
- Small chunks (around 128 to 256 tokens) match precise questions well but may lack context.
- Large chunks (around 1024) carry more context but mix topics, which blurs their embeddings and costs more prompt tokens.
- Start near 512, then measure retrieval on real questions and adjust.
Structure-aware parsers often beat plain splitting:
MarkdownNodeParsersplits on headings and records the heading path in metadata.CodeSplittersplits source code by syntax.SentenceWindowNodeParserandHierarchicalNodeParsersupport advanced retrieval tricks (an Expert lesson).SemanticSplitterNodeParsersplits where the meaning changes, using embeddings.
Parsers can be chained: Markdown sections first, then SentenceSplitter for sections that are still too long. Whatever you choose, keep useful metadata (title, heading) on every node, because a chunk without context is hard to retrieve.
Example
from llama_index.core import Document
from llama_index.core.node_parser import MarkdownNodeParser, SentenceSplitter
md = Document(text="# Billing\n## Refunds\nRefunds take 5 days.\n## Invoices\nInvoices are PDFs.")
sections = MarkdownNodeParser().get_nodes_from_documents([md])
for n in sections:
print(n.metadata.get("header_path"), "|", n.text.splitlines()[0])
# / | # Billing
# /Billing/ | ## Refunds
# /Billing/ | ## Invoices
splitter = SentenceSplitter(chunk_size=512, chunk_overlap=50)
long_doc = Document(text="A sentence about refunds. " * 400)
nodes = splitter.get_nodes_from_documents([long_doc])
print(len(nodes), len(nodes[0].text.split())) # 6 340 (6 chunks, ~340 words each)
# During indexing:
# index = VectorStoreIndex.from_documents(docs, transformations=[splitter])Remember
SentenceSplitterdefaults: chunk_size 1024, overlap 200 tokens- Pass parsers as
transformationswhen building an index - Structure-aware parsers (Markdown, code) keep sections intact
Watch out for
- Picking a chunk size once by intuition and never measuring retrieval quality on real questions.