TopolearnRe-embedding every document on each start is slow and costs money; persistence avoids it.
An in-memory index disappears when your program exits. Rebuilding it means embedding every chunk again, which is slow and costs money with paid embedding APIs.
Save it to a folder:
index.storage_context.persist(persist_dir="storage")This writes JSON files for the vectors, the docstore (node texts and metadata) and the index store.
Load it later:
storage_context = StorageContext.from_defaults(persist_dir="storage")
index = load_index_from_storage(storage_context)A common pattern is "build once, load afterwards": check whether the folder exists, and only build the index if it doesn't.
Things to watch:
- The embedding model is not saved. Set the same
Settings.embed_modelbefore loading, or queries will be compared against vectors from a different model. - The default stores are JSON files loaded fully into memory. That is fine for thousands of chunks but not for millions.
- If the source files change, you need to update the index (insert, delete, refresh) and persist again.
For larger or shared data, use a vector database integration instead (Chroma, Qdrant, pgvector, Pinecone and many others). Those store vectors themselves, so you reconnect to them rather than loading files; a later lesson covers it.
Example
import os
from llama_index.core import (SimpleDirectoryReader, StorageContext, VectorStoreIndex,
load_index_from_storage)
PERSIST_DIR = "storage"
if not os.path.exists(PERSIST_DIR):
documents = SimpleDirectoryReader("data").load_data()
index = VectorStoreIndex.from_documents(documents) # embeds everything once
index.storage_context.persist(persist_dir=PERSIST_DIR)
print("built and saved")
else:
storage_context = StorageContext.from_defaults(persist_dir=PERSIST_DIR)
index = load_index_from_storage(storage_context) # no re-embedding
print("loaded from disk")
print(index.as_query_engine().query("How long do refunds take?"))Remember
index.storage_context.persist(persist_dir=...)saves to diskload_index_from_storage(StorageContext.from_defaults(persist_dir=...))loads it- Set the same embedding model before loading
Watch out for
- Loading a saved index with a different Settings.embed_model than the one used to build it.