TopolearnRunning models yourself changes the trade-offs on privacy, cost and quality, so it helps to know what you are getting.
When you use ChatGPT or Claude in a browser, the model runs on the company's computers. Your text travels over the internet, and you pay per use or by subscription.
Some language models are open-weight: their creators publish the trained model files (the "weights", billions of numbers learned during training), so anyone can download and run them. Examples include Google's Gemma, Meta's Llama, Alibaba's Qwen and OpenAI's gpt-oss.
Ollama is a free program that makes running those models on your own computer easy. It:
- downloads models from its online library with one command,
- runs them using your graphics card (GPU) when possible, or your processor (CPU),
- gives you a chat in the terminal and a desktop app,
- runs a small local server (a program that waits for requests) so your own code and other apps can use the models through an API on your machine.
Why people use it:
- Privacy: prompts and documents stay on your machine.
- No per-request cost: once downloaded, using a model is free (apart from electricity and hardware).
- Offline use: works without internet.
- Experimenting: try many open models quickly.
The trade-off: models small enough for a laptop are usually less capable than the biggest hosted models, and speed depends on your hardware. Ollama also offers optional cloud models, which run bigger models on Ollama's servers through the same commands; a later lesson covers when that makes sense.
Remember
- Open-weight models can be downloaded and run by anyone
- Ollama downloads, runs and serves them locally via CLI, app and API
- Local = private, offline and free per use, but limited by your hardware
Watch out for
- Expecting a small laptop model to match the quality of the largest hosted models on hard reasoning tasks.