TopolearnWhen chunking and reranking are not enough, these patterns fix specific, recurring failure modes.
Each pattern targets one failure you can observe in evaluations.
Small chunks match well but lack context. Use small-to-big retrieval: search on small units, then give the LLM bigger surroundings.
- Sentence window:
SentenceWindowNodeParsermakes one node per sentence and stores the surrounding sentences in metadata (window). At query time,MetadataReplacementPostProcessor(target_metadata_key="window")swaps each matched sentence for its window. - Auto-merging:
HierarchicalNodeParsercreates parent chunks (for example 2048 tokens) split into children (512, then 128). Only the leaves (get_leaf_nodes) go into the vector index; all nodes go into the docstore.AutoMergingRetrieverreplaces many retrieved children of one parent with the parent itself.
Different questions need different sources or strategies. A RouterQueryEngine with an LLM selector (LLMSingleSelector) picks one of several query engines by their descriptions, for example a summary engine for "summarise" questions and a vector engine for specific facts.
Complex questions combine several lookups ("Compare the 2024 and 2025 leave policies"). SubQuestionQueryEngine asks the LLM to split the question into sub-questions, each sent to a matching tool, then combines the answers. An agent with query-engine tools solves the same problem more flexibly.
Vague questions. Query rewriting (HyDE, which embeds a hypothetical answer instead of the question, or QueryFusionRetriever with several generated queries) improves recall.
All of these add LLM calls or storage. Adopt one at a time, and keep it only if your retrieval and answer metrics improve.
Example
from llama_index.core import Document, VectorStoreIndex
from llama_index.core.node_parser import SentenceWindowNodeParser
from llama_index.core.postprocessor import MetadataReplacementPostProcessor
text = ("The 2025 leave policy changed. Staff now get 25 days. "
"Unused days expire in March. Managers must approve carry-over. "
"Part-time staff get a pro-rated amount.")
parser = SentenceWindowNodeParser.from_defaults(
window_size=2, window_metadata_key="window", original_text_metadata_key="original_text")
nodes = parser.get_nodes_from_documents([Document(text=text)])
print(len(nodes)) # 5 (one per sentence)
print(nodes[2].text) # Unused days expire in March.
print(nodes[2].metadata["window"][:60]) # The 2025 leave policy changed. Staff now get 25 days. ...
index = VectorStoreIndex(nodes)
qe = index.as_query_engine(
similarity_top_k=2,
node_postprocessors=[MetadataReplacementPostProcessor(target_metadata_key="window")],
)
print(qe.query("When do unused leave days expire?")) # In March; carry-over needs approval.Remember
- Sentence window and auto-merging: retrieve small, answer with bigger context
- RouterQueryEngine picks an engine; SubQuestionQueryEngine decomposes questions
- Add one technique at a time and keep it only if metrics improve
Watch out for
- Stacking every advanced technique at once, making the pipeline slow and impossible to debug.