Skip to content
The Chat Archive
gpt-4.1-mini-2025-04-14· OpenAI LMArena Human Preference 140K

100k characters = ~25k input token??? is this a joke?

  1. User

    100k characters = ~25k input token??? is this a joke?

  2. Assistant

    The relationship between characters and tokens isn't a fixed ratio because tokens can be as short as one character or as long as a whole word or more, depending on the tokenizer and the language.

    However, as a rough estimate for English text:

    • 1 token ≈ 4 characters on average.

    So, 100,000 characters ÷ 4 ≈ 25,000 tokens.

    This is not a joke but a typical approximation used in many NLP contexts, including OpenAI's token counting. The exact number can vary depending on the text's complexity, punctuation, and language.

    If you want a more precise token count, you can use tokenizers like the one from the tiktoken library (used by OpenAI) to tokenize your specific text.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.