TopolearnToo small a context silently cuts off long documents and agent conversations; too large wastes memory.
The context window is how much text (in tokens) a model can consider at once: your prompt, the conversation so far and its answer. A model supports some maximum (shown by ollama show), but Ollama decides how much to allocate when loading it, because every extra token of context needs memory.
Default. Ollama picks a default based on your GPU memory: 4,096 tokens with less than 24 GB, 32,768 with 24 to 48 GB, and 262,144 (256k) with 48 GB or more. Cloud models use their maximum.
That 4k default on ordinary machines is small: roughly 3,000 words in total. If your prompt is longer, the oldest part is dropped and the model never sees it, usually without an error. Agents, coding tools and document questions need much more; the docs suggest at least 64,000 tokens for agents and coding tools.
Changing it:
- Desktop app: the context length slider in Settings.
- Server-wide: start the server with the environment variable
OLLAMA_CONTEXT_LENGTH=32768. - In a chat:
/set parameter num_ctx 32768. - Per API request:
"options": {"num_ctx": 32768}. - In a custom model:
PARAMETER num_ctx 32768in a Modelfile.
Checking it: ollama ps shows the allocated context in the CONTEXT column, next to the CPU/GPU split.
Memory grows with context. If raising it pushes part of the model onto the CPU, speed drops sharply. Find the largest context that still keeps the model 100% on the GPU, or pick a smaller model.
Example
# Server-wide default (Linux/macOS shell; restart the server after changing it)
OLLAMA_CONTEXT_LENGTH=32768 ollama serve
# Per chat session
ollama run qwen3:8b
>>> /set parameter num_ctx 32768
# Per API request
curl http://localhost:11434/api/chat -d '{
"model": "qwen3:8b",
"messages": [{"role": "user", "content": "Summarize this long report: ..."}],
"options": {"num_ctx": 32768},
"stream": false
}'
ollama ps # CONTEXT column shows 32768; check PROCESSOR is still 100% GPURemember
- Default context depends on VRAM: 4k under 24 GB, then 32k, then 256k
- Raise it with the app slider, OLLAMA_CONTEXT_LENGTH, num_ctx or a Modelfile
- More context needs more memory; keep the model fully on the GPU
Watch out for
- Sending a 20-page document with the default 4k context and trusting the summary, although most of the document was cut off.