Vector stores and RAG

Ask a model about your refund policy and it will guess, because your policy isn't in its training data. RAG (retrieval-augmented generation) fixes this: before answering, the system looks up the relevant passages in your documents and gives them to the model.

The lookup uses embeddings: lists of numbers that capture what a piece of text means, so texts with similar meaning get similar numbers. A vector store is a database that stores these numbers and finds the closest matches to a question, even when the wording differs.

Step 1: insert your documents (a separate workflow you run when documents change):

• Fetch the files (Google Drive, a website, a database).
• A vector store node (for example Simple Vector Store for testing, or Pinecone, Qdrant, Supabase or Postgres PGVector for real use) in Insert Documents mode.
• Plug in an Embeddings sub-node (the model that turns text into numbers).
• Plug in a Default Data Loader with a text splitter that cuts documents into chunks (small passages). The Recursive Character Text Splitter is a good default. Add metadata such as source or date to each chunk.

Step 2: query. Attach the same vector store to your AI Agent as a tool, with a description such as "Search the company handbook". Set how many chunks to return. You must use the same embeddings model as for insertion, otherwise searches return nonsense.

Keep chunks focused, return a handful rather than dozens (they all cost tokens), and tell the agent in its system message to say "I don't know" when the search finds nothing relevant.

Simple Vector Store lives in memory and is lost on restart: use a real database in production.