TopolearnUsers ask follow-up questions like "and for Germany?"; a chat engine remembers what they mean.
A query engine treats every question on its own. A chat engine keeps the conversation history, so follow-ups work:
chat_engine = index.as_chat_engine(chat_mode="condense_plus_context")
chat_engine.chat("What is the refund period?")
chat_engine.chat("Does that apply to sale items too?")The second question only makes sense with the first. The `condense_plus_context` mode handles this in two steps:
1. It asks the LLM to rewrite ("condense") the latest message plus history into a standalone question, such as "Does the refund period apply to sale items?". 2. It retrieves context for that question and answers with the history and the context.
Other modes:
context: retrieves using the raw latest message, then answers with history. Simpler, weaker on follow-ups.condense_question: rewrites the question and runs a query engine, without history in the final answer.simple: plain chat with the LLM, no retrieval.
Useful methods:
chat(msg)returns a response with.responseand.source_nodes.stream_chat(msg)streams tokens through.response_gen.reset()clears the history.chat_historyshows the stored messages.
History is kept in a memory object with a token limit, so very long conversations drop their oldest turns. A chat engine object holds one conversation, so in a web app create or restore one per user session; never share one engine between users.
For conversations that also need tools or several data sources, use an agent instead (a later lesson).
Example
from llama_index.core import Document, VectorStoreIndex
index = VectorStoreIndex.from_documents([
Document(text="Refunds are possible within 30 days of delivery."),
Document(text="Sale items can be refunded within 14 days only."),
])
chat = index.as_chat_engine(chat_mode="condense_plus_context",
system_prompt="You answer questions about our shop policies.")
print(chat.chat("What is the refund period?").response)
# You can request a refund within 30 days of delivery.
print(chat.chat("And for items bought on sale?").response)
# Sale items can only be refunded within 14 days.
print(len(chat.chat_history)) # 4 messages (2 user, 2 assistant)
chat.reset() # start a fresh conversationRemember
- Chat engines keep history so follow-up questions work
condense_plus_contextrewrites the question, then retrieves and answers- One chat engine per conversation;
reset()clears history
Watch out for
- Sharing one chat engine instance across all users of a web app, so they see each other's conversation context.