TopolearnProduction RAG needs persistent, scalable, shareable storage instead of JSON files in memory.
A vector store is the database that holds embeddings and runs similarity search. LlamaIndex's default SimpleVectorStore keeps everything in memory. For real applications you use an integration, each in its own package: Chroma, Qdrant, Weaviate, Milvus, Pinecone, Elasticsearch, OpenSearch, Postgres with pgvector, MongoDB Atlas, Redis and many more.
The pattern is the same for all of them:
1. Create the database client and the LlamaIndex vector store object. 2. Wrap it in a StorageContext and build the index with it:
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(docs, storage_context=storage_context)3. Later, reconnect without re-embedding:
index = VectorStoreIndex.from_vector_store(vector_store)Most of these databases also store the node text and metadata, so you don't need to persist a separate docstore.
How to choose:
- Chroma or LanceDB: embedded, file-based, easy for local work and small deployments.
- pgvector: if you already run Postgres, keeps vectors next to your other data.
- Qdrant, Weaviate, Milvus, Pinecone: dedicated servers or managed services for large collections, with strong filtering and hybrid search.
Check what each integration supports: metadata filter operators, hybrid (keyword plus vector) search, deletion by document id and async calls. The vector dimension is fixed by your embedding model, so changing models means creating a new collection.
Example
# pip install llama-index-vector-stores-chroma chromadb
import chromadb
from llama_index.core import Document, StorageContext, VectorStoreIndex
from llama_index.vector_stores.chroma import ChromaVectorStore
client = chromadb.PersistentClient(path="./chroma_db") # data saved on disk
collection = client.get_or_create_collection("handbook")
vector_store = ChromaVectorStore(chroma_collection=collection)
# First run: embed and store
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
[Document(text="Refunds take 5 days.", metadata={"dept": "support"})],
storage_context=storage_context,
)
# Any later run (or another process): reconnect, no re-embedding
index = VectorStoreIndex.from_vector_store(vector_store)
print(index.as_retriever().retrieve("refund time")[0].node.text) # Refunds take 5 days.
print(collection.count()) # 1Remember
- Each vector database is a separate integration package
- Build with
StorageContext.from_defaults(vector_store=...) - Reconnect with
VectorStoreIndex.from_vector_store(vector_store)
Watch out for
- Calling from_documents on every application start against a persistent store, which inserts duplicate copies of every chunk.