The context window

The context window is the model's working space: the maximum number of tokens it can look at in one request. Everything counts towards it: the system instructions, every earlier message in the conversation, any documents you pasted, and the answer the model is writing.

Sizes have grown fast. Current flagship models from OpenAI, Anthropic and Google can handle very large inputs, often hundreds of thousands of tokens and in some cases about a million, which is several long books. Smaller and older models have much smaller windows. Check your model's documentation, because the number varies by model and changes often.

What happens when you hit the limit:

• If the input alone is too big, the request is refused with an error.
• If there is not enough room left for the answer, it may stop early.
• Chat apps and agents quietly drop or summarise older messages to stay inside the window, which is why a long chat can "forget" something from the start.

A bigger window is not a reason to paste everything. Models get less accurate at finding and using details as the context fills up; Anthropic's docs call this context rot. Long inputs also cost more (more tokens) and make the answer slower to start.

Good habits:

• Include what the task needs, not everything you have.
• Put long documents first and your question after them, and label each document clearly.
• For a new topic, start a new conversation instead of continuing a very long one.