TopolearnRAG grounds answers in current, private data and reduces hallucinations without fine-tuning.
RAG has two phases. Indexing (offline): load, split, embed and store documents. Answering (per query): retrieve relevant chunks and put them into the prompt so the model answers from them.
Two common designs:
1. A RAG chain (always retrieve once). With LCEL:
chain = ({"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt | model | StrOutputParser())Simple, predictable, one model call; good for FAQ-style questions.
2. Agentic RAG: wrap the retriever as a tool and give it to create_agent. The model decides whether to search, can rephrase queries, and can search several times for multi-part questions. More flexible, more calls.
Prompting matters: instruct the model to use only the provided context, say when the answer is not there, and cite sources from metadata. Format documents with their source so citations are possible.
Improving quality usually comes from retrieval, not the prompt:
- Tune chunking,
k, and use MMR or hybrid search. - Add query rewriting or multi-query retrieval for vague questions.
- Rerank retrieved chunks with a cross-encoder or reranking API.
- Filter by metadata (date, product, permissions).
Evaluate with a set of questions and expected answers or sources, and trace runs to see which chunks were retrieved when an answer is wrong.
Example
from langchain_core.documents import Document
from langchain_core.embeddings import DeterministicFakeEmbedding
from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.vectorstores import InMemoryVectorStore
store = InMemoryVectorStore(DeterministicFakeEmbedding(size=64)) # use real embeddings
store.add_documents([Document("Refunds take 5 business days.", metadata={"source": "faq.md"})])
retriever = store.as_retriever(search_kwargs={"k": 1})
def format_docs(docs):
return "\n".join(f"[{d.metadata['source']}] {d.page_content}" for d in docs)
prompt = ChatPromptTemplate.from_messages([
("system", "Answer only from the context and cite the source.\n\n{context}"),
("human", "{question}"),
])
model = GenericFakeChatModel(messages=iter(["Refunds take 5 business days [faq.md]."]))
rag = ({"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt | model | StrOutputParser())
print(rag.invoke("How long do refunds take?")) # Refunds take 5 business days [faq.md].
# Agentic RAG: wrap retriever in a @tool and pass it to create_agent(...)Remember
- Index once (load, split, embed, store); retrieve and generate per query
- A RAG chain retrieves once; agentic RAG lets the model decide when and what to search
- Most quality gains come from retrieval: chunking, k, hybrid search, reranking
Watch out for
- Debugging bad RAG answers by tweaking the prompt without checking which chunks were actually retrieved.