TopolearnThe right model is the difference between a fast, useful assistant and a slow or useless one.
The Ollama library (ollama.com/library and ollama.com/search) lists models with their sizes and capabilities. Choosing comes down to three questions.
1. What must it do? Look for capability tags:
- tools: can request function calls, needed for agents.
- vision: understands images.
- thinking: can reason step by step before answering.
- embedding: produces vectors for search, not chat.
2. How big can it be? Size is measured in parameters, the learned numbers inside the model, written like 8b (8 billion). As a rough guide:
- 1 to 4 billion: fast on almost any machine; fine for simple rewriting, classification and extraction.
- 7 to 14 billion: good general assistants on a decent laptop or a single gaming GPU.
- 20 to 35 billion: noticeably smarter; needs a lot of memory.
- 70 billion and up: workstation or server territory.
Some models use a mixture of experts design: they contain many parameters but use only part of them for each word, so they run faster than their total size suggests.
3. How much text must it handle? The context length you need (long documents, agents) costs memory too (a later lesson).
Practical approach: start with a well-known, recent family in a size your hardware handles, test it on 10 to 20 real examples from your task, then try one size up and one size down. Specialised variants (for code, or instruction-tuned versions) often beat larger general ones on their speciality.
Leaderboards are a starting point, not an answer; your own examples decide.
Example
# See what a model can do before relying on it
ollama show qwen3:8b # (example output; details vary by version)
# Model
# architecture qwen3
# parameters 8.2B
# context length 40960
# quantization Q4_K_M
# Capabilities
# completion
# tools
# thinking
# Compare two sizes on the same real task
ollama run qwen3:4b "Classify as billing/bug/other: 'I was charged twice.'"
ollama run qwen3:8b "Classify as billing/bug/other: 'I was charged twice.'"Remember
- Match capabilities (tools, vision, thinking, embedding) to the task
- Parameters (e.g. 8b) drive quality, speed and memory
- Test candidates on your own examples, one size up and down
Watch out for
- Picking the biggest model that barely fits, which then runs partly on the CPU and is too slow to use.