Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-opus-4-20250514· AnthropicHugging Face LMArena Human Preference 140K

ELI5 shortly: "BT Rating Calculation for Zero Strength: Problem: In…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen3-235b-a22b’s answer.
  1. User

    ELI5 shortly: "BT Rating Calculation for Zero Strength:

    Problem: In calculateBradleyTerryRatings, stats[m].rating = strengths[m] > 0 ? Math.round(DEFAULT_RATING + 400 * Math.log10(strengths[m])) : DEFAULT_RATING; If strengths[m] is 0, it defaults to DEFAULT_RATING. This is a reasonable fallback, but Math.log10(0) is -Infinity. While the strengths[m] > 0 check handles this, it's worth noting the behavior. If a model has 0 effective wins and participated in games where the denominator calculation also leads to 0 (e.g., only played against other 0-strength models), its strength might remain 0."
    
  2. Claude Assistant

    Imagine you're trying to rank players in a game. The Bradley-Terry system gives each player a score based on their wins.

    The problem: What happens when a player has zero wins? The math formula uses something called log10, which breaks when you try to calculate log10(0) - it gives you "negative infinity" which computers can't handle.

    The code already has a safety check: if a player has 0 strength (no wins), it just gives them a default score instead of doing the broken math.

    But here's the catch: If a player only plays against other players who also have 0 wins, they stay stuck at 0 forever. It's like a group of beginners who only play each other - none of them can earn their first win because they're all equally bad, so they all stay at the bottom ranking forever.

  3. User

    i don't understand

  4. Claude Assistant

    Let me make it even simpler!

    Think of it like a video game where players get points for winning:

    1. The scoring system uses a special math formula to turn wins into a score
    2. The problem: If someone has 0 wins, the math formula breaks (like dividing by zero on a calculator - it just shows ERROR)
    3. The fix: The code says "if someone has 0 wins, just give them the default starting score of 1500"

    The weird situation: Imagine 3 new players who only play against each other. Since they're all equally new:

    • Player A beats Player B
    • Player B beats Player C
    • Player C beats Player A

    They each win once and lose once, so the math might still think they all have "0 strength" and keep them all at the starting score forever. They're stuck because they only play against other beginners.

    It's like being stuck at level 1 in a game because you only fight other level 1 players!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.