TopolearnSeeing the whole pipeline work end to end first makes every later component easier to place.
The smallest useful LlamaIndex program answers questions about a folder of files:
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("What is our refund policy?")
print(response)What happens behind these lines:
1. SimpleDirectoryReader reads every supported file in data/ (text, Markdown, PDF, Word and more) into Document objects. 2. VectorStoreIndex.from_documents splits the documents into chunks, sends each chunk to an embedding model and keeps the vectors in memory. 3. as_query_engine() builds an object that, for each question, embeds the question, finds the most similar chunks (2 by default), puts them into a prompt, and asks the LLM to answer from them. 4. query returns a Response. Printing it shows the answer; response.source_nodes lists the chunks that were used, with similarity scores.
By default LlamaIndex uses OpenAI for both the LLM and the embeddings, so set the OPENAI_API_KEY environment variable first. The next lesson shows how to choose other models, including local ones.
Always look at source_nodes when an answer seems wrong. If the right chunk isn't there, the problem is retrieval, not the LLM, and no prompt change will fix it.
Example
# pip install llama-index ; export OPENAI_API_KEY=sk-...
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
documents = SimpleDirectoryReader("data").load_data() # every file in ./data
index = VectorStoreIndex.from_documents(documents) # chunk + embed + store
query_engine = index.as_query_engine()
response = query_engine.query("How long do refunds take?")
print(response) # Refunds are processed within 5 business days.
for src in response.source_nodes: # the chunks the answer was based on
print(round(src.score, 2), src.node.metadata["file_name"])
# 0.83 refunds.md
# 0.71 faq.mdRemember
- Reader, then
VectorStoreIndex.from_documents, thenas_query_engine().query() - The default LLM and embeddings are OpenAI's (needs OPENAI_API_KEY)
response.source_nodesshows which chunks the answer used
Watch out for
- Judging a RAG app only by its answers without checking source_nodes, so retrieval problems look like model problems.