Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-3-7-sonnet-20250219· AnthropicHugging Face LMArena Human Preference 140K

What are the performance rankings of mainstream LLM, and which ones are based on human…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Amazon Novaamazon.nova-pro-v1:0’s answer.
  1. User

    What are the performance rankings of mainstream LLM, and which ones are based on human user scores?

  2. Claude Assistant

    The performance rankings of mainstream LLMs vary depending on the evaluation method, but as of my last update in October 2024:

    Top performers generally include GPT-4 (OpenAI), Claude 3 Opus (Anthropic), Gemini Ultra (Google), and Llama 3 (Meta) models, with varying strengths in different domains.

    For human evaluation specifically, the most prominent benchmarks include:

    • LMSYS Chatbot Arena: Uses Elo ratings based on direct human A/B comparisons
    • AlpacaEval: Combines human judgments with automated evaluation
    • MT-Bench: Uses human evaluators to rate model responses on complex tasks

    Human evaluations tend to show different results than purely automated benchmarks like MMLU or HumanEval, as they capture subjective qualities like helpfulness and natural communication that are harder to measure programmatically.

    Would you like more specific information about any particular ranking system or evaluation method?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.