TopolearnMost real apps use Ollama through a framework or automation tool rather than raw API calls.
Popular frameworks have Ollama integrations, so swapping a hosted model for a local one is often a few lines.
LangChain (package langchain-ollama):
ChatOllama(model="qwen3:8b", temperature=0)is a chat model with the usual LangChain interface (invoke,stream, tools, structured output).OllamaEmbeddings(model="embeddinggemma")for embeddings.- Useful arguments:
num_ctxfor context length,reasoning=Truefor thinking models,base_urlfor a remote server.
LlamaIndex (packages llama-index-llms-ollama and llama-index-embeddings-ollama):
Ollama(model="qwen3:8b", request_timeout=120.0, context_window=16000). Raiserequest_timeout(30 seconds by default), because local models on modest hardware can be slow, especially the first call while the model loads.OllamaEmbedding(model_name="embeddinggemma").
n8n (a no-code automation tool): create an Ollama credential with the base URL http://localhost:11434. If n8n runs in Docker, localhost means the container itself, so use http://host.docker.internal:11434 instead (on Linux servers, add a host mapping for that name). Then pick an Ollama model in the chat model or embeddings nodes.
Many other tools (coding assistants, chat interfaces, CrewAI and more) connect the same way: a base URL plus a model name, or the OpenAI-compatible /v1 endpoint.
Common problems: the model isn't downloaded (pull it first), the context is too small for agents and RAG prompts, timeouts on the first slow call, and wrong addresses between containers.
Example
# pip install langchain-ollama llama-index-llms-ollama llama-index-embeddings-ollama
from langchain_ollama import ChatOllama, OllamaEmbeddings
llm = ChatOllama(model="qwen3:8b", temperature=0, num_ctx=16384)
print(llm.invoke("Give one tip for naming variables.").content)
vec = OllamaEmbeddings(model="embeddinggemma").embed_query("hello")
print(len(vec)) # 768
from llama_index.core import Settings
from llama_index.embeddings.ollama import OllamaEmbedding
from llama_index.llms.ollama import Ollama
Settings.llm = Ollama(model="qwen3:8b", request_timeout=120.0, context_window=16000)
Settings.embed_model = OllamaEmbedding(model_name="embeddinggemma")
print(Settings.llm.complete("Say hi in 3 words.").text)
# n8n in Docker -> Ollama credential base URL: http://host.docker.internal:11434Remember
- LangChain:
ChatOllamaandOllamaEmbeddings(langchain-ollama) - LlamaIndex:
Ollama(raise request_timeout) andOllamaEmbedding - Apps in Docker reach the host's Ollama via host.docker.internal
Watch out for
- Using http://localhost:11434 from a container, which points at the container itself rather than the host running Ollama.