Can you tell me about the different leaderboards on lmarena.ai and how to interpret them?
- User
Can you tell me about the different leaderboards on lmarena.ai and how to interpret them?
- Assistant
Below is a quick “field guide” to the leaderboards you’ll run into on lmarena.ai, what each one is actually measuring, and a few rules of thumb for reading the numbers without getting tripped up ☕️📊
- Chat-Arena (a.k.a. “Arena Elo”)
What it is
• Live, crowdsourced A/B “battles.” A user chats, two models answer, voters pick a winner (or tie).
• The site fits a Bradley-Terry/Elo-style model to thousands of these pairwise preferences.
• The resulting rating is called Arena Elo (with a ±95 % conf. interval).How to read it
• Higher Elo ⇒ more likely to win an average battle.
• 50 pt gap ≈ 57 % win-prob.; 100 pt ≈ 64 %; 200 pt ≈ 76 %.
• Ignore small gaps when CIs overlap – those models are “statistically tied.”
• Check “Battles” count: < ~200 battles means the rating will still jump around.Helpful tabs/filters
All | Open-Source | Base-only | Under 20B | New | Small-sample.
Great when you care about real-time user preference, writing style, harmlessness, etc.
- Benchmark / Hub Leaderboard (static scores, a.k.a. “PW-E score”)
What it is
• Aggregates classic academic test-sets: MMLU, ARC-Chal, HellaSwag, TruthfulQA, GSM-8K, etc.
• The “Hub score” or “Power-Weighted Elo (PW-E)” is just a weighted average of those columns.How to read it
• Each column is % accuracy; bold = best in that column.
• PW-E tries to combine them, but missing values (“--”) skew things, so compare rows with similar coverage.
• Great for knowledge & reasoning under closed-book, non-chat conditions.
- Single-Dataset Views (“Eval Sets”)
Pick “MMLU” or “GSM8K” in the drop-down and the board re-sorts by that one metric.
• Useful for spotting specialist models (math-tuned, code-tuned, med-tuned, …).
• Scores are directly comparable because every model gets the same prompt template.
- Live Battle Feed
A fire-hose of the most recent A/B duels.
• You’ll see raw win/loss counts, prompt text, and voter comments.
• Fun for qualitative inspection, but don’t over-interpret single fights.
- Per-Model “Card”
Click a model name → mini-dashboard:
• Elo trajectory over time
• Win-rate broken down by question category (coding, reasoning, casual chat…)
• Model metadata (size, license, release date, system prompt, etc.)
Putting it all together
When choosing or comparing models:
✓ Look at Arena Elo for general chat UX.
✓ Check static benchmarks if you need proven accuracy on standardized tests.
✓ Mind the ±CI and the battle count.
✓ Filter by openness/size if deployment constraints matter.
✓ Remember that a 100-pt Elo gap is meaningful, but two models whose intervals overlap are effectively tied.Finally, the site is live and ratings change daily—always note the time-stamp in the header (“Last updated: …”). Happy model hunting! 🏹
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.