What LLM Tokens Really Cost (and How to Estimate Your Bill)

What a token actually is

Large language models don't read characters or words — they read tokens. A token is a chunk of text, roughly ¾ of a word on average in English. A useful rule of thumb:

• ~4 characters ≈ 1 token
• ~750 words ≈ 1,000 tokens
• A dense page of text ≈ 500–800 tokens

Punctuation, code, and non-English text tokenize differently (often more tokens per character), so treat these as estimates, not guarantees.

Why your bill has two prices

Almost every API charges input tokens and output tokens at different rates — and output is usually several times more expensive. That matters because the two scale differently:

• Input grows with your prompt and everything you stuff into context (instructions, retrieved documents, chat history).
• Output grows with how much the model writes back.

A chatbot that quotes long documents but answers briefly is input-heavy. A code generator or writing assistant is output-heavy. Knowing which you are tells you where to optimize.

The context window trap

The biggest surprise in most bills is conversation history. In a multi-turn chat, many implementations resend the entire prior conversation as input on every new message. So turn 20 might send 20 turns' worth of tokens — even though the user only typed one line.

Costs then grow roughly with the square of conversation length if you're not careful. Mitigations:

• Truncate or summarize old turns instead of resending them verbatim.
• Cache stable parts of the prompt where the provider supports it.
• Retrieve only the relevant snippets rather than dumping whole documents.

Estimating a monthly bill

The basic formula per request is:

cost = (input_tokens  × input_price_per_token)
     + (output_tokens × output_price_per_token)

Then multiply by requests per day and days per month. The parts people underestimate:

• System prompt + retrieved context — often larger than the user's actual question.
• Retries and tool calls — an agent that calls tools loops through the model multiple times per user turn.
• Growth — traffic rarely stays flat.

Rather than do this by hand, plug your numbers into the LLM token budget calculator to project a monthly cost.

API vs self-hosting

At low-to-moderate volume, a hosted API almost always wins on total cost of ownership — you pay only for what you use and skip GPUs, ops, and idle capacity. Self-hosting starts to make sense at high, steady volume, where fixed infrastructure amortizes across enough requests.

The crossover depends on your throughput, model size, and how well you'd keep the hardware busy. The self-hosted vs API calculator lets you compare the two for your specific load instead of guessing.

The short version

• Tokens ≈ ¾ of a word; budget both input and output, and output costs more.
• Conversation history and retrieved context are usually the hidden cost drivers.
• Estimate before you launch, then watch real usage — the first bill is where assumptions go to die.