Token counter & AI cost calculator

What a token actually is

A token is the unit a language model reads, thinks in, and charges for. It is not a word, and it is not a character — it sits deliberately between the two.

Models do not see letters. Before anything else happens, your text is cut into pieces drawn from a fixed vocabulary the model was trained with, and each piece becomes one token. Common words survive whole: the, because and important are each a single token. Rarer words are assembled from fragments, so antidisestablishmentarianism costs five or six.

The technique is called byte-pair encoding. Starting from individual characters, the most frequently adjacent pair is merged into a new unit, and that merge repeats tens of thousands of times over a training corpus. What survives is a vocabulary in which frequency buys brevity: the more often a sequence appeared in the training data, the more likely it is to be one token today.

Two consequences follow from that, and both cost money. Text that resembles the training data is cheap — ordinary English prose, most of all. Text that does not is expensive, whether that is a language with a different script, a run of digits, or a JSON blob full of braces and quotation marks.

Why the number matters

Three separate limits are all denominated in tokens, and it is easy to run into one while watching another.

1. It is what you are billed for

Every API charges per token, input and output priced separately. At a small scale this is invisible — a single question costs a fraction of a cent. At the scale of a product it is the whole line item. A feature summarising a 4,000-token document for 10,000 users a day is not a rounding error on any frontier model, and the gap between the cheapest and most expensive model that could do the job is routinely forty-fold.

2. It is a hard ceiling on what fits

Every model has a context window measured in tokens, and it holds everything at once: your instructions, whatever you attached, the conversation so far, and the reply being written. Exceed it and you get an error, or worse, silent truncation — the model quietly drops the oldest part of the conversation and answers as though it were never there. Silent truncation is the more dangerous failure because nothing looks broken.

3. It sets how long you wait

Output tokens are produced one at a time. A 2,000-token reply takes roughly four times as long as a 500-token reply on the same model, regardless of how fast the hardware is. If a feature feels slow, the reply length is usually a more effective thing to change than the model.

How many tokens is that?

Useful anchors for English prose. Every figure below is what this tool reports for representative text of that size, so you can reproduce them above.

The working ratio for English is about 1.3 tokens per word, or four to four and a half characters per token. Every other kind of content is worse than that, never better — which is why estimating from a word count tends to under-budget rather than over-budget.

A real bill, worked end to end

Abstract per-million rates are hard to reason about, so here is a concrete feature: a support assistant that reads a customer email and drafts a reply. Each call sends a 400-token system prompt, a 600-token email, and generates a 500-token draft. The product handles 2,000 emails a day.

That is 1,000 input tokens and 500 output tokens per call — 2 million input and 1 million output tokens a day, or 60 million and 30 million a month.

Same feature, same traffic, and the bill moves by a factor of forty. Two things are worth noticing. Output dominates even though it is half the token volume, because it is priced several times higher. And the practical question is rarely "which model is best" but "which is the cheapest model that is good enough for this specific job" — a task like reformatting a support reply does not need a frontier model, while a task like diagnosing a subtle bug does.

The same text is a different number on every model

There is no universal token. Each family trains its own tokeniser on its own corpus, so a sentence that is 100 tokens for GPT might be 110 for Claude and 105 for Gemini. For English the spread is usually five to fifteen per cent — annoying but rarely decisive.

For everything else it can be decisive. A tokeniser trained mostly on English handles other scripts badly, and the penalty is severe: text in a language written in a non-Latin script can cost two to three times as many tokens as its English translation, meaning speakers of those languages pay more for the same information and hit context limits sooner. This is a well-documented inequity in how these systems are priced, and it is invisible unless you measure it.

The family selector above shows all four counts at once for exactly this reason. If you are choosing a model for non-English content, compare the token counts and not only the headline rates — a model that is 20% cheaper per token but needs 40% more tokens is more expensive.

What makes text expensive

Some content costs far more per character than prose does. Knowing which kind you are sending is most of budgeting.

Context windows, and what "fits" really means

A context window is the total token budget for a single request. It is shared, not divided: the system prompt, retrieved documents, the conversation history and the reply all draw on the same pool. A 128K model given a 120K document has 8K left for both your question and its answer.

Two practical points get missed.

Output competes with input. If you ask for a 4,000-token summary, those 4,000 tokens must fit inside the window alongside everything you sent. The cost column above marks any model whose window your text already exceeds, so a choice that cannot work is visible before you make it.

A big window is not the same as good recall. Models are measurably better at using material near the beginning and the end of a long context than material buried in the middle. Filling a million-token window is possible; relying on the model to notice one sentence in the middle of it is optimistic. Where it matters, put the important material at the start or the end, and say explicitly what you want done with it.

Input and output are not priced alike

Output typically costs four to eight times input, and the reason is architectural rather than commercial. Input tokens are processed together in a single forward pass — the model reads the whole prompt at once. Output tokens are produced one at a time, each requiring its own pass over everything generated so far. Generating is simply much more work than reading.

The practical consequence is that the reply length is the most valuable number in your prompt. "Summarise this" with no limit invites a 1,500-token answer; "summarise this in under 150 words" produces roughly 200 tokens and usually a better summary, because the constraint forces prioritisation. Asking for a table rather than prose cuts output further still.

It also means that a long, careful prompt is cheap. Adding 200 tokens of context and constraints to avoid one wasted 800-token answer pays for itself immediately. People optimise the wrong half: they trim their instructions and leave the output unbounded.

Eight ways to cut a token bill

1. Cap the output. Set a word or item limit in the prompt and a max-tokens value in the request. The single highest-leverage change available.
2. Send less history. Keep the last few turns plus a short running summary rather than the whole transcript. Most conversations do not need turn three verbatim at turn thirty.
3. Retrieve, do not dump. Sending a whole 200-page manual so the model can answer one question costs the whole manual every time. Send the three relevant paragraphs.
4. Match the model to the task. Classification, extraction, formatting and routing rarely need a frontier model. Reserve the expensive one for work that visibly fails without it.
5. Strip formatting from pasted content. HTML tags, markdown tables and JSON scaffolding are tokens. If the model only needs the text, send the text.
6. Ask for structure. A table or a list is more useful than prose and consistently shorter — often by half for the same information.
7. Batch related questions. Five questions about the same document in one call pay for that document once instead of five times.
8. Write a better prompt. The most under-counted token cost is regeneration. A vague prompt answered three times costs three times a good one answered once — and the good one is the answer you keep.

Caching and batching, which most people leave on the table

Two discounts are available on most major APIs and are routinely unused.

Prompt caching stores the unchanging prefix of your prompt — a system prompt, a style guide, a document you ask many questions about — so repeat requests are billed at a fraction of the input rate, commonly around a tenth. Anything with a stable prefix and repeated traffic benefits, and the change is usually a matter of ordering your prompt so the fixed part comes first.

Batch processing takes roughly half off every token, input and output, in exchange for asynchronous delivery within a window rather than an immediate reply. Any workload that is not user-facing — nightly summarisation, bulk classification, backfilling a dataset — should be batched. The calculator above shows undiscounted list rates, so treat its numbers as a ceiling.

How this estimate works, and how far off it can be

This counter is deterministic: the same text always returns the same number, which is what makes it usable for comparing two drafts. No model is called and nothing is sent anywhere for analysis.

It is an estimate, and it says so rather than implying a precision it does not have. An exact count requires the target model's own vocabulary file — roughly 200,000 entries, different for every family, and several megabytes to load. Instead, the text is segmented into words, numbers, whitespace, punctuation and non-Latin characters, and each segment is priced the way byte-pair encoding demonstrably behaves: a leading space is free, digits merge in groups of at most three, punctuation runs partially merge, and characters outside the Latin range cost far more than characters inside it.

Measured against known counts, it lands within about 5% on short text and 10% on English prose, and it is materially better than characters-over-four on code, numbers and other scripts — which is exactly where budgeting goes wrong. For contract-grade precision, use the provider's own tokeniser library; for deciding whether a document fits and what it will cost, this is the right tool.

Nothing you paste is stored. The text is analysed in memory and discarded — not logged, not saved, not used for training.

Common questions

How many tokens is 1,000 words?

About 1,300 for ordinary English. Technical writing runs a little higher, source code much higher, and Chinese or Japanese higher again.

How many words is 1,000 tokens?

About 750 English words — roughly a page and a half. This is the ratio the rules of thumb are built on.

Are tokens the same as characters?

No. A token averages about four characters in English, but the relationship is not fixed: a is one token and so is international. Character counts are a poor proxy for anything that is not plain prose.

Does the system prompt count?

Yes, every time. A 500-token system prompt sent with every request in a chat application is 500 tokens per message, not per conversation. This is the strongest argument for prompt caching.

Do I pay for tokens if the answer is bad?

Yes. Billing is on tokens processed, not on whether you liked the result — which is why prompt quality is a cost question and not only a quality one. A prompt that needs three attempts costs three times as much as one that works first time.

Why do two tools give me different token counts?

Either they are modelling different tokenisers, or one of them is using characters-over-four. Compare against the family you are actually sending to, and treat any tool that reports one number for all models with suspicion.

Do images and files count as tokens?

Yes. Images are converted to a token count based on their dimensions — often several hundred to a couple of thousand each — and attached documents are converted to text and counted normally. This calculator models text only.

Glossary

• Token — the unit a model reads and bills in, usually part of a word. About four characters of English.
• Tokeniser — the component that cuts text into tokens. Each model family has its own.
• Byte-pair encoding (BPE) — the algorithm that builds the vocabulary by repeatedly merging the most frequent adjacent pair.
• Context window — the maximum tokens a model can hold at once, covering prompt, history and reply together.
• Input tokens — everything you send. Cheaper, processed in parallel.
• Output tokens — everything generated. More expensive, produced one at a time.
• Max tokens — the request-level ceiling on reply length. Your hard stop against a runaway bill.
• Prompt caching — billing a repeated prompt prefix at a reduced rate, commonly about a tenth of the normal input price.
• Batch API — asynchronous processing at roughly half price, for work that does not need an immediate reply.
• Truncation — silently dropping content that no longer fits the context window. The failure that does not look like one.