TopolearnModel choice sets quality, cost, privacy and speed, and mixing up embedding models silently breaks retrieval.
LlamaIndex uses two kinds of models:
- an LLM, which writes answers, and
- an embedding model, which turns text into vectors for search.
`Settings` is a global object holding the defaults every component uses:
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")Set these once at program start. You can also pass llm= or embed_model= to a single component (for example index.as_query_engine(llm=other_llm)) to override the default there.
Each provider is its own package: llama-index-llms-anthropic gives Anthropic, llama-index-llms-ollama gives Ollama, llama-index-embeddings-huggingface gives HuggingFaceEmbedding, and so on. Model names change often, so check the provider's current list.
Local models. With Ollama (a free app that runs open models on your own machine), use Ollama(model="llama3.1", request_timeout=120.0) for the LLM and OllamaEmbedding or HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5") for embeddings. Nothing leaves your computer, but local LLMs are slower and less capable.
Settings also holds defaults such as chunk_size (1024 tokens) and chunk_overlap.
The most important rule: index and query with the same embedding model. Vectors from different models are not comparable. If you change the embedding model, rebuild the index.
Example
# pip install llama-index-llms-ollama llama-index-embeddings-huggingface
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0) # OPENAI_API_KEY
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
Settings.chunk_size = 512 # default 1024
# Fully local alternative (Ollama running on this machine):
# from llama_index.llms.ollama import Ollama
# from llama_index.embeddings.huggingface import HuggingFaceEmbedding
# Settings.llm = Ollama(model="llama3.1", request_timeout=120.0)
# Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
vec = Settings.embed_model.get_text_embedding("hello world")
print(len(vec)) # 1536 for text-embedding-3-small
print(Settings.llm.complete("Say hi in 3 words.").text)Remember
Settings.llmandSettings.embed_modelset global defaults- Override per component with
llm=orembed_model= - Index and query with the same embedding model; re-index after changing it
Watch out for
- Switching Settings.embed_model while loading an index built with another model, which returns nonsense matches without any error.