Skip to content
TheChatArchiveThe Chat Archive
chatgpt-4o-latest-20250326· OpenAIHugging Face LMArena Human Preference 140K

There are two rating systems in which the same players play. Why are they different?…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Metallama-4-maverick-17b-128e-instruct’s answer.
  1. User

    There are two rating systems in which the same players play. Why are they different? Namely.

    Sytstem A      System B
    500	           1500	
    600	           1600	
    700	           1700	
    800	           1800	
    900	           1900	
    1000	       2000
    1100	       2100
    1150	       2150
    1200	       2200
    1250	       2250
    1300	       2300
    1350	       2350
    1400	       2400
    1450	       2450
    1500	       2500
    1550	       2550
    1600	       2600
    1650	       2650
    1700	       2700
    1750	       2750
    1800	       2800
    1850	       2850
    1900	       2900
    1950	       2950
    2000	       3000
    2100	       3100
    2200	       3200
    2300	       3300
    2400	       3400
    2500	       3500
    2600	       3600
    2700	       3700
    2800	       3800
    2900	       3900
    3000	       4000
    

    Why wouldn't the two systems converge with the same rating? Note that in system A the starting rating is 1500, in system B it is 2500. Some say because the two systems are identical, only they are scaled of a factor of 1000 points between each other.

    Yet two other different rating systems where more or less the same players play in both converge! chess.com blitz has a starting rating around 1000 while lichess blitz has a starting rating around 1500, and they have those rating ranges.

    Chess.com  Lichess Blitz
    Blitz    (+/- 75)
               N=2620
    500        945
    600        1030
    700        1115
    800        1205
    900        1305
    1000    1395
    1100    1485
    1150    1520
    1200    1560
    1250    1600
    1300    1630
    1350    1685
    1400    1720
    1450    1760
    1500    1795
    1550    1825
    1600    1855
    1650    1895
    1700    1925
    1750    1960
    1800    2000
    1850    2025
    1900    2060
    1950    2100
    2000    2120
    2100    2205
    2200    2270
    2300    2330
    2400    2405
    2500    2495
    2600    2585
    2700    2630
    2800    2715
    2900    2830
    3000    2925
    

    So why one systems does not converge and the other does? It feels like wrong.

  2. Assistant

    This is a great question and touches on the deeper mechanics of rating systems, particularly how they are initialized, scaled, and maintained over time. Let's break it down.


    🔍 Summary of the Question

    You are comparing two pairs of rating systems:

    Pair 1: System A vs. System B

    • System A: Starts at 500, goes to 3000
    • System B: Starts at 1500, goes to 4000
    • The difference between the two is exactly 1000 points across the board.

    You ask: Why don’t these systems "converge" to the same rating scale, even though the same players are in both?

    Pair 2: Chess.com Blitz vs. Lichess Blitz

    • These systems have different starting ratings (Chess.com ~1000, Lichess ~1500)
    • But over time, the ratings seem to "converge" — a 2000 on Chess.com is roughly a 2120 on Lichess, not a 1000-point difference.

    So the core question is:

    Why do some rating systems (like Chess.com vs. Lichess) converge to similar scales, while others (like System A vs. System B) remain offset by a fixed amount?


    🧠 Key Concepts

    1. Rating Systems Are Arbitrary Scales

    • Ratings are not absolute measures of skill; they are relative.
    • The numbers themselves are arbitrary — what matters is how they compare to other players in the same system.

    2. Linear Transformations

    • If System B is always System A + 1000, then System B is just a linear transformation of System A.
    • This means the two systems are functionally identical — they just use different scales.

    3. Convergence Depends on Design

    • Whether two systems "converge" depends on:
      • How ratings are initialized
      • How ratings are updated (e.g., K-factor, volatility)
      • Whether the systems are trying to match each other
      • Whether they are anchored to a common reference point (e.g., FIDE ratings)

    🧩 Why System A and System B Don’t Converge

    Because they are intentionally offset by 1000 points. If System B is defined as:

    System B rating = System A rating + 1000

    Then they will never "converge" — they are just different scales of the same system. There is no mechanism in place to bring them together, nor is there a need to.

    This is like measuring temperature in Celsius vs. Kelvin. They are different numbers, but they describe the same reality.


    ✅ Why Chess.com and Lichess Ratings Do Converge

    This is more subtle.

    • Both systems are independently maintained, but they are anchored to the same player pool.
    • Over time, as players play on both platforms, the systems self-correct to reflect actual skill levels.
    • Even though the starting ratings are different (e.g., 1000 vs. 1500), the rating algorithms adjust based on performance.
    • This leads to a statistical convergence — not perfect, but close.

    In other words, the systems are not linearly related, but they are empirically correlated due to shared player data and similar rating goals.


    🧪 Analogy

    Imagine two thermometers:

    • One starts at 0°C for freezing water.
    • Another starts at 32°F for freezing water.

    They are different scales, but they measure the same thing. If you know the conversion formula, you can translate between them.

    But if one thermometer is just a shifted version of the other (e.g., always +1000), then they will never "converge" — they are just different labels.


    🧮 Mathematical View

    If:

    • System A rating = R
    • System B rating = R + 1000

    Then:

    • The systems are isomorphic — they preserve order and difference.
    • There is no convergence because they are already perfectly aligned, just offset.

    But if:

    • Chess.com rating = f(skill)
    • Lichess rating = g(skill)

    And f and g are different functions, then over time, as more data is collected, f and g may start to approximate each other — leading to convergence.


    🧠 Final Thoughts

    • System A and B are identical in structure, just offset — no convergence is needed or expected.
    • Chess.com and Lichess are independent systems that happen to converge due to shared player behavior and similar rating goals.
    • Convergence is not a property of the rating system itself, but of how it is used and maintained.

    ✅ TL;DR

    • System A and B are just linearly offset — they don’t need to converge because they are already functionally identical.
    • Chess.com and Lichess ratings converge over time because they are based on shared player data and similar rating goals, even though they start differently.
    • Convergence happens when systems are designed to reflect the same reality, not when they are just shifted versions of each other.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.