Counting tokens before sending

Claude API · Practical · 7 min

Knowing the size up front lets you stay inside the context window and estimate cost.

Before sending a big request you often want to know how large it is: will it fit in the context window, and roughly what will it cost? Guessing from character counts is unreliable, because how text splits into tokens depends on the model's tokenizer (the rule set that cuts text into tokens).

client.messages.count_tokens(...) takes the same model, system, messages and tools you would send to messages.create and returns input_tokens, without generating anything.

Useful facts:

  • It is free, but it has its own requests-per-minute limit.
  • The number is an estimate and can differ slightly from the real bill.
  • Count against the model you will actually use. Claude 4.7 and later models use a newer tokenizer that produces noticeably more tokens (around 30 percent) for the same text than older models.
  • It cannot count files referenced by URL or file_id, or server tools; send images and PDFs as Base64 when counting.

Typical uses: refuse or trim a user upload that is too large, decide when to summarise an old conversation, and estimate the cost of a batch job before running it. After the real call, usage tells you the exact numbers.

Example

import anthropic

client = anthropic.Anthropic()
LIMIT = 150_000                       # your own budget for one request

request = dict(
    model="claude-opus-5-5",
    system="You summarise legal contracts.",
    messages=[{"role": "user", "content": open("contract.txt").read()}],
)
count = client.messages.count_tokens(**request)
print(count.input_tokens)

if count.input_tokens > LIMIT:
    print("Too long: split the contract first")
else:
    reply = client.messages.create(max_tokens=1000, **request)

Remember

  • count_tokens takes the same inputs as create and returns input_tokens
  • Free, rate-limited, and an estimate
  • Count against the model you will use; tokenizers differ

Check yourself

  1. Why might the same prompt count more tokens on a newer Claude model?

Sign in to answer, see the explanations, and get this lesson back as a quick review later.

Official docs: platform.claude.com/docs/en/build-with-claude/token-counting