Skip to content
TheChatArchiveThe Chat Archive
Geminigemini-2.5-pro-preview-03-25· GoogleHugging Face LMArena Human Preference 140K

There are two rating systems in which the same players play. Why are they different?…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.0-flash-001’s answer.
  1. User

    There are two rating systems in which the same players play. Why are they different? Namely.

    Sytstem A      System B
    500	           1500	
    600	           1600	
    700	           1700	
    800	           1800	
    900	           1900	
    1000	       2000
    1100	       2100
    1150	       2150
    1200	       2200
    1250	       2250
    1300	       2300
    1350	       2350
    1400	       2400
    1450	       2450
    1500	       2500
    1550	       2550
    1600	       2600
    1650	       2650
    1700	       2700
    1750	       2750
    1800	       2800
    1850	       2850
    1900	       2900
    1950	       2950
    2000	       3000
    2100	       3100
    2200	       3200
    2300	       3300
    2400	       3400
    2500	       3500
    2600	       3600
    2700	       3700
    2800	       3800
    2900	       3900
    3000	       4000
    

    Why wouldn't the two systems converge with the same rating? Note that in system A the starting rating is 1500, in system B it is 2500. Some say because the two systems are identical, only they are scaled of a factor of 1000 points between each other.

    Yet two other different rating systems where more or less the same players play in both converge! chess.com blitz has a starting rating around 1000 while lichess blitz has a starting rating around 1500, and they have those rating ranges.

    Chess.com  Lichess Blitz
    Blitz    (+/- 75)
               N=2620
    500        945
    600        1030
    700        1115
    800        1205
    900        1305
    1000    1395
    1100    1485
    1150    1520
    1200    1560
    1250    1600
    1300    1630
    1350    1685
    1400    1720
    1450    1760
    1500    1795
    1550    1825
    1600    1855
    1650    1895
    1700    1925
    1750    1960
    1800    2000
    1850    2025
    1900    2060
    1950    2100
    2000    2120
    2100    2205
    2200    2270
    2300    2330
    2400    2405
    2500    2495
    2600    2585
    2700    2630
    2800    2715
    2900    2830
    3000    2925
    

    So why one systems does not converge and the other does? It feels like something is not working. As a additional point, imagine lichess starting with a rating of 3000. I could not believe it would converge to the chess.com ratings after a while because the points in the system (for each player) will be just massive.

  2. Gemini Assistant

    Okay, let's break down why these rating systems behave differently. The core reason lies in whether the systems are truly identical except for the starting point, or if there are other underlying differences.

    Scenario 1: System A vs. System B (Perfect Offset)

    1. Identical Mechanics: The data Rating_B = Rating_A + 1000 strongly suggests that System A and System B use the exact same underlying rating algorithm and parameters. This means:

      • They use the same formula (e.g., Elo, Glicko, Glicko-2).
      • They use the same parameters within that formula (e.g., the same K-factor in Elo, or the same system constants and initial deviation/volatility settings in Glicko).
      • Crucially, the calculation for expected score depends only on the difference in ratings between two players.
      • The rating update formula applies the same change based on the difference between the actual score and the expected score.
    2. Why the Offset is Maintained: If Player X has rating R_A in System A and Player Y has rating S_A, their difference is R_A - S_A. In System B, their ratings are R_B = R_A + 1000 and S_B = S_A + 1000. Their difference in System B is R_B - S_B = (R_A + 1000) - (S_A + 1000) = R_A - S_A.

      • Since the rating difference is identical in both systems, the expected outcome of a match between them is calculated identically in both systems.
      • If they play a game and the actual outcome is determined, the rating change (delta) calculated by the identical formula will be the same in both systems.
      • Therefore, if New_R_A = R_A + Delta and New_S_A = S_A + Delta', then New_R_B = R_B + Delta = (R_A + 1000) + Delta = New_R_A + 1000, and similarly for Player Y.
      • The 1000-point difference is perfectly preserved for every player after every game assuming they play the exact same games with the exact same results in both hypothetical systems.
    3. Starting Point vs. Offset: You mentioned starting ratings of 1500 (A) and 2500 (B). This means new players are seeded with a 1000-point difference. Since the mechanics preserve this difference, the entire rating distribution just shifts up by 1000 points in System B compared to System A. They don't converge because there's no mechanism within the identical systems to erode that initial offset.

    Scenario 2: Chess.com Blitz vs. Lichess Blitz (Non-Linear Relationship)

    This is different because, despite having largely overlapping player pools, the systems are not identical implementations. Here's why they don't maintain a simple offset and exhibit the complex relationship shown in your table:

    1. Different Rating Algorithms/Parameters:

      • Lichess uses the Glicko-2 rating system.
      • Chess.com uses a Glicko-1 based system (details may be proprietary but it's known to be Glicko-based, likely Glicko-1).
      • Even if both used Glicko-2, they likely use different parameters within the system (like the τ (tau) constant controlling volatility change, different initial Rating Deviations (RD), or different initial volatilities (σ)). These parameters affect how quickly ratings change and stabilize. Lichess starts new accounts at 1500 rating, 350 RD, 0.06 volatility. Chess.com starts new accounts at different ratings depending on self-assessment (e.g., 400, 800, 1200, etc.) and likely uses different initial RD/volatility parameters.
    2. Different K-Factor Equivalents: Both Glicko systems have mechanisms similar to the K-factor in Elo, determining the magnitude of rating changes. These likely differ between the sites and might vary differently based on a player's rating or number of games played.

    3. Different Player Pools & Behavior: While largely overlapping, the player pools aren't identical.

      • Some players are exclusive to one platform.
      • Players may play differently or different amounts on each site. Someone might play casually on Chess.com but seriously on Lichess, or vice versa.
      • The distribution of opponents a player faces at a given rating level might differ slightly between the sites.
    4. Different Initial Seeding and Inflation/Deflation:

      • Lichess starts everyone at 1500. Chess.com uses self-reported skill to start players at various levels (often lower than 1500 for beginners). This injects points into the system differently.
      • Rating systems can naturally inflate or deflate over time based on how new players are added, how points are exchanged, and how inactivity is handled. Chess.com and Lichess likely have different inflation/deflation dynamics. Lichess, for example, has historically had a higher degree of rating inflation than Chess.com or FIDE.
    5. Implementation Details: Subtle differences in how rating periods are defined, how provisional ratings are handled, or how timeouts/aborts affect ratings can contribute to divergence.

    Why No "Convergence" in the Sense of Becoming Identical?

    • The Chess.com/Lichess systems aren't "converging" to the same number. They are settling into a relatively stable relationship or mapping between the two systems, which your table illustrates. This relationship is non-linear precisely because of the differences listed above.
    • Your intuition about Lichess starting at 3000 is correct. If Lichess hypothetically started everyone at 3000 but kept its Glicko-2 system, the ratings would eventually spread out, but the average rating and the mapping to Chess.com would be significantly higher. A 1500 Chess.com player might correspond to a 3200 Lichess player (just a guess), not converge down near 1500. The total points in the system matter, and the rating algorithms don't inherently destroy points to force convergence to a lower baseline. They just redistribute points based on results relative to expectations.

    In Summary:

    • System A vs. B: Identical algorithms and parameters + different starting offset = maintained offset.
    • Chess.com vs. Lichess: Different algorithms/parameters + different starting points/distributions + different player pool dynamics = complex, non-linear relationship, not convergence to identical values.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.