LLM Token Counter

Mastering AI Efficiency, A Developer's Guide to Tokens, Costs, and Context

LLM tokens are the unit of currency, the unit of latency, and the unit of attention all at once. Every prompt you write has a price tag (input × $/M), a generation cost (output × $/M, typically 3–5× more), and a hard ceiling beyond which the API simply refuses. Building production LLM features without thinking in tokens is like writing database queries without thinking about indexes, it works until it doesn't, and then it breaks expensively. This counter is built to make the token economics legible before you ship a feature, not after the bill arrives.

What Tokens Actually Are

A token is the output of a tokenizer: a learned function that maps a string to a sequence of integer IDs from a fixed vocabulary (32K to 256K IDs depending on the model). The tokenizer is part of the model, change the tokenizer and you change what the model 'sees.' Tokens are not characters and not always words; they're subword units chosen during model training to maximize the compressed representation of the training corpus.

For modern LLMs, two algorithm families dominate:

• Byte-Pair Encoding (BPE), used by OpenAI (tiktoken) and Anthropic Claude. Starts with byte-level tokens and iteratively merges the most common adjacent pairs until the vocabulary fills. Result: common words become single tokens, rare words split into a handful of pieces, completely novel strings fall back to individual bytes (so any input is representable).
• SentencePiece (Unigram or BPE variant), used by Google Gemini, Meta Llama, Mistral. Treats text as a raw byte stream including whitespace (it explicitly tokenizes the leading space as part of the word, which is why " hello" and "hello" are often different tokens).

This is why "use 4 characters per token" is just a rule of thumb, the actual count depends on the model, the language, and what kind of content you're tokenizing.

A Heuristic Cheat Sheet

CJK is the surprise gotcha, a single Chinese character often tokenizes to 1.5 tokens because the tokenizer's vocabulary is English-biased. A 1,000-character Chinese document might be 1,800 tokens on Claude vs. ~600 for the equivalent English translation. If your product is multilingual, budget separately by language.

The Cost Equation in Practice

Cost is the most concrete reason to count tokens. The formula is uniform across providers:

Reference pricing snapshot (subject to change, always verify):

(Gemini 1.5 has a price step at 128K, under that, the lower number; above, the higher. Verify all prices against current provider docs.)

The 50× spread between flagships and 'mini' models is the single most consequential decision you'll make. A workload that runs comfortably on Haiku, mini, or Flash for $50/month will cost $2,500/month on the corresponding flagship, same accuracy on routine classification, summarization, and extraction tasks, dramatically different bill.

Cost Worked Examples

Example 1, Per-request chatbot. 1,500-token system prompt + 500-token average user message + 800-token response. On GPT-4o: (2000 × $2.50 + 800 × $10) / 1M = $0.013/request. At 10,000 requests/day: $130/day = ~$4,000/month. Same workload on GPT-4o-mini: $0.0008/request = $8/day = ~$250/month. 15× cost reduction with a 'mini' model.

Example 2, Document Q&A. 50,000-token document + 200-token question + 800-token answer. On Claude Sonnet 4: (50200 × $3 + 800 × $15) / 1M = $0.163/query. 100 queries/day = $16/day = $500/month. Add prompt caching on the document (assume 80% cache hit rate after warm-up): (50000 × $3 × 0.1 + 50000 × $3 × 0.9 × 0.1 × 0.8 + ...) ≈ 75% input cost reduction. New cost: ~$125/month.

Example 3, Bulk classification. 200 tokens of context per item, 10 tokens of label output, 1 million items. On Claude Haiku 4: (200M × $0.80 + 10M × $4) / 1M = $200. On Sonnet 4: $750. On Opus 4: $3,750. Pick the cheapest model that passes your eval. Run a 1,000-item sample on Haiku first; only upgrade if accuracy is insufficient.

Context Windows, A Total, Not a Suggestion

The context window is the sum of every token in the request:

• System prompt
• Tool/function definitions (often 1,000–5,000 tokens of JSON schema you forgot to count)
• Conversation history (every prior user message + every prior assistant message)
• Current user message
• Tool results streamed back during the turn
• Reserved space for the model's reply (controlled by max_tokens)

A 128K model with a 4K system prompt + 3K tool schemas + 50K conversation history + 5K current user input has 66K free, of which max_tokens will reserve more before generation. Hitting the cap fails the request before generation. Streamed tool calls inside a turn can also push you over mid-generation, particularly with retrieval tools that return long documents.

Practical guidance:

• Target ≤60% utilization of the model's window. The remaining 40% is your safety margin for tool results, unexpected long replies, and conversation growth.
• Architect for rolling summarization: after N turns, replace the oldest M turns with a single 'summary so far' message. Frameworks like LangChain's ConversationSummaryBufferMemory automate this.
• For RAG (retrieval-augmented generation): your retrieved passages compete with everything else. Keep k (chunks retrieved) and chunk size proportionate to the model, a 128K-context model can take k=20 chunks of 1,000 tokens; a 4K-context model needs k=3 chunks of 500 tokens.

Prompt Caching. The Largest Free Optimization Available

Anthropic and OpenAI both support automatic or explicit prompt caching: a prefix of your prompt that is identical across many requests gets stored server-side and replayed at a 90% input-token discount (Anthropic, explicit cache_control markers, 5-minute default TTL with optional 1-hour) or 50% discount (OpenAI, automatic for prefixes of ≥1024 tokens).

When it applies:

• Long static system prompts (the canonical case, the same 2,000-token persona/guidelines on every call).
• Few-shot examples that don't change between requests.
• Document Q&A where the same document is queried multiple times within the TTL.
• Agent workflows where each step calls the LLM with the same tool definitions and reasoning preamble.

When it doesn't help:

• Prompts where every call has a different system prompt (rare in production but common in experimentation).
• Prefixes shorter than the minimum cacheable length (1024 tokens on most providers).
• Single-call workflows with no repetition within the TTL.

The order of fields in the prompt matters: caching matches prefix equality. Anything you want cached must come before anything that varies. The canonical layout is: [tool definitions] → [system prompt] → [cache marker] → [conversation history] → [current user message].

Output Length Discipline

Output tokens cost 3–5× more than input across every provider. They also dominate latency: a 4,000-token response takes ~4× longer than a 1,000-token response on the same model. Strategies that work:

• max_tokens ceiling. Set it. Don't rely on the model to stop on its own.
• Explicit length instructions. 'Reply in one sentence.' 'Limit to 100 words.' 'JSON only, no prose.' Surprisingly effective.
• Structured output / JSON schema enforcement. Forces concise responses bounded by the schema; eliminates verbose prose around the answer.
• Stop sequences. Tell the model to stop generating when it emits a specific marker.
• Streaming + early termination. If your UI can render partial output, stream and let the user cancel, saves output tokens on long responses that the user would have skimmed anyway.

Tokens-per-Second and Latency Budgeting

Each model has a roughly constant generation speed (with provider-specific variance):

If your UX requires <2 second responses, your output token budget is 2 × tokens_per_second. On Sonnet, that's ~120 tokens, barely enough for two paragraphs. On Groq, 600+ tokens fits comfortably. Hardware accelerators (Groq, Cerebras, SambaNova) trade some model selection for dramatic latency wins, they matter most for interactive consumer apps.

Multimodal Token Math

Mixing images, audio, and video into a prompt adds tokens at provider-specific rates:

OpenAI GPT-4o (and 4o-mini):

• Low-res images ("detail": "low"): 85 tokens flat.
• High-res: 85 + 170 × (number of 512×512 tiles). A 1024×1024 image = 85 + 170×4 = 765 tokens.

Anthropic Claude:

• Approximately (width × height) ÷ 750 tokens per image. A 1024×1024 image ≈ 1,400 tokens; capped at 1,600.

Google Gemini 1.5:

• Images: 258 tokens flat, regardless of size.
• Video: 263 tokens per second of video.
• Audio: 32 tokens per second of audio.

A 30-second video on Gemini = 7,890 tokens before you've added any prompt. A 5-minute meeting transcript audio file = 9,600 tokens. Always count multimodal inputs separately and prominently in your cost budget, they swamp text.

Heuristic vs. Exact Counting

This tool uses a character-based heuristic for speed. When does that matter, and when is exactness required?

Heuristic is fine for:

• Pre-flight cost estimation ('how much will this batch cost?').
• Prompt-engineering iteration ('is this system prompt getting too long?').
• Context-window sanity checks ('will this fit in 128K?').
• Side-by-side cost comparison across providers.

Exact tokenization is required for:

• Billing reconciliation against an invoice.
• Triggering on a hard token threshold (cache markers, max-token gates).
• Detecting tokenizer-specific edge cases (long URLs, unusual unicode, binary data).
• Research / academic measurement of compression ratios.

For exact counts, use:

• OpenAI: tiktoken Python or JS package, encoding_for_model(...).
• Anthropic: SDK's client.messages.count_tokens(model=..., messages=[...]) endpoint.
• Gemini: model.count_tokens(...) in the GenAI SDK.
• Llama: HuggingFace AutoTokenizer.from_pretrained('meta-llama/...').

Prompt Engineering for Token Efficiency

The 80/20 of token reduction:

1. Pick a smaller model first. Run your eval suite on the cheapest model that exists; only upgrade if it fails. Most engineers default to flagships and never test the cheaper tier, leaving 10–50× on the table.
2. Cache the static prefix. If your system prompt is reused across requests, mark it cacheable.
3. Strip ceremony from system prompts. 'You are a helpful AI assistant. Please be polite and try your best to...' is a 50-token preamble that adds nothing. Direct instructions are shorter and more effective: 'Classify the input as spam or not-spam. Reply with one word.'
4. Use compact data formats. YAML over JSON. CSV over JSON arrays. Custom shorthand over verbose XML.
5. Summarize, don't accumulate. Rolling-summary memory beats raw-history memory once you're past a handful of turns.
6. Retrieval over context-stuffing. If you're sending the same 50,000-token reference document on every call, you should be embedding it once and retrieving the relevant 2,000-token chunk per query.
7. Constrain output length. Use max_tokens, structured output, and explicit instructions.
8. Batch where you can. Many providers offer batch APIs (OpenAI Batch, Anthropic Batches) at 50% discount with 24-hour SLA, perfect for offline processing.

Common Mistakes

• Forgetting tool definitions count. A 3,000-token JSON schema for your tools is on every request whether the model uses the tools or not. Trim unused tools per workflow.
• Counting tokens with the wrong tokenizer. Estimating with cl100k_base (GPT-4) when deploying to Claude or Gemini. Counts can differ by 20%+.
• Not budgeting for retries. Failed requests with rate limits, JSON parse errors, or content filter triggers still cost the input tokens. Build a 10–20% buffer into cost estimates.
• Ignoring the output side at the design stage. Most token-cost discussion focuses on input, but output is 3–5× more expensive per token and dominates the bill on chat/agent workloads.
• Caching the wrong part. Putting variable content (the user message) before stable content (the system prompt) defeats prefix caching entirely.
• Treating context window as marketing. A 1M-token window doesn't mean you should use 1M tokens, attention performance degrades on very long contexts ('lost in the middle'), and cost scales linearly. Use the smallest window that holds the relevant content.

Local-First Privacy

Loading a full tokenizer library (tiktoken is ~2MB, HuggingFace tokenizers are 5–20MB per model) is overkill for a quick token estimate, and uploading prompts to a third-party 'token counter' service is exactly the kind of telemetry your security team is asking you to avoid. This counter uses a fast in-browser heuristic, your prompts never leave the page. Useful when:

• The prompt contains user PII (medical notes, support transcripts, internal customer data).
• The prompt encodes a proprietary system prompt you don't want logged or reused.
• You're under an NDA on the upstream application and even the prompt structure is confidential.
• You're working offline (on a plane, behind a strict corporate firewall).

Verify with DevTools' Network tab: zero outbound requests during counting. The estimate is your character count multiplied by per-language calibration constants, entirely client-side arithmetic.