When local beats hosted, and when not

Local and hosted models each win in different situations.

Local (Ollama on your own hardware) is a good fit when:

• Data is sensitive: health, legal or customer data that may not leave your network.
• You need offline use, for example on a plane, in a lab or on a factory floor.
• Volume is high and tasks are simple (classifying or extracting from thousands of documents), where per-request fees add up.
• You want predictable behaviour: a model file you control doesn't change unless you update it.
• You are learning and experimenting.

Hosted APIs (OpenAI, Anthropic, Google and others) are a better fit when:

• You need the strongest reasoning, writing or coding quality available; the largest models don't fit on ordinary hardware.
• You have spiky or low volume, so paying per request is cheaper than buying GPUs.
• Many users need fast answers at once; providers have far more hardware.
• You don't want to operate servers, GPUs and updates.

Hidden costs of local: hardware, electricity, your time for setup and updates, and slower speed on modest machines. Hidden costs of hosted: data agreements, rate limits, price changes and model retirements.

Common hybrid pattern: develop and test locally, run private or bulk tasks on local models, and send the hard minority of tasks to a hosted model. Ollama's OpenAI-compatible API (a later lesson) makes switching mostly a configuration change. Ollama's own cloud models are a middle ground: bigger open models through the same tool, but your data then leaves your machine.

Decide with a small test: run your real examples through both and compare quality, speed and cost.