TopolearnThe query engine is the main interface for question answering over your data.
A query engine takes a question and returns an answer. index.as_query_engine() builds one from two parts:
- a retriever that finds relevant nodes, and
- a response synthesizer that sends them with the question to the LLM.
Common options:
similarity_top_k=4: how many nodes to retrieve (default 2). More gives the LLM more context but costs tokens and can add noise.response_mode="compact": how nodes are turned into an answer (the default; a later lesson compares modes).streaming=True: return tokens as they are generated.filters=...: only search nodes with certain metadata (a later lesson).llm=...: use a different LLM for this engine.
The result of query_engine.query("...") is a Response:
str(response)orresponse.response: the answer text.response.source_nodes: a list ofNodeWithScore, the retrieved nodes with similarity scores. Use them to show citations ("Source: refunds.md").response.metadata: metadata of the source nodes by id.
With streaming=True you get a streaming response: iterate response.response_gen for text pieces, or call response.print_response_stream().
The default prompt tells the LLM to answer from the context and not from prior knowledge. Even so, check answers against sources for important uses, and consider telling users when no relevant source was found.
Example
from llama_index.core import Document, VectorStoreIndex
index = VectorStoreIndex.from_documents([
Document(text="Refunds take 5 business days.", metadata={"source": "faq.md"}),
Document(text="Our office is in Oslo.", metadata={"source": "about.md"}),
])
qe = index.as_query_engine(similarity_top_k=2)
response = qe.query("How long do refunds take?")
print(response.response) # Refunds take 5 business days.
for n in response.source_nodes:
print(f"{n.score:.2f}", n.node.metadata["source"])
# 0.86 faq.md
# 0.72 about.md
streaming_qe = index.as_query_engine(streaming=True)
stream = streaming_qe.query("Where is the office?")
for piece in stream.response_gen: # text arrives piece by piece
print(piece, end="", flush=True) # The office is in Oslo.Remember
as_query_engine(similarity_top_k=..., streaming=...)configures retrieval and output- Response:
.responsetext plus.source_nodeswith scores - Use source metadata for citations
Watch out for
- Raising similarity_top_k to 20 to "be safe", which adds noise and cost and can make answers worse.