Skip to content
The Chat Archive
claude-3-5-sonnet-20241022· Anthropic LMArena Human Preference 140K

100k characters = ~25k input token??? is this a joke?

  1. User

    100k characters = ~25k input token??? is this a joke?

  2. Assistant

    The exact ratio between characters and tokens varies depending on the text content, but roughly:

    1 token ≈ 4 characters (English text)

    So yes, 100,000 characters would be approximately 25,000 tokens. This is because tokens can represent common words or parts of words, not just individual characters. The tokenization process tries to break text into meaningful units while being efficient.

    This isn't a joke - it's just how modern language models process text. Different models may use slightly different tokenization schemes, but this rough 4:1 ratio is fairly common for English text.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.