Skip to content
The Chat Archive
gemini-2.5-pro· Google LMArena Human Preference 140K

How to use LM arena.AI

  1. User

    How to use LM arena.AI

  2. Assistant

    Of course! Here is a comprehensive guide on how to use the Chatbot Arena (often found via arena.ai, which directs to lmsys.org).

    1. What is Chatbot Arena?

    Chatbot Arena is a research project and public platform created by LMSys.org (Large Model Systems Organization). Its primary purpose is to evaluate and rank Large Language Models (LLMs) in a real-world, unbiased way.

    The core idea is a blind, randomized "taste test" for AI. You, the user, act as a judge in a battle between two anonymous AI models. Your votes help create a dynamic, crowdsourced leaderboard of the best-performing models.


    2. The Core Concept: The Blind Battle

    Understanding this is key to using the Arena effectively:

    1. Anonymous Models: You are given two chatbots, labeled "Model A" and "Model B." You do not know which AI model is which (e.g., you don't know if you're talking to GPT-4, Claude 3, Llama 3, or Gemini).
    2. Randomized Pairing: The two models are chosen randomly for each new battle.
    3. Single Prompt: You give both models the exact same prompt.
    4. Side-by-Side Comparison: You receive two different responses, one from each model.
    5. You are the Judge: You compare the two responses and vote for the one you think is better.
    6. The Reveal: Only after you vote, the platform reveals the true identities of Model A and Model B.

    This process generates valuable data on human preferences, which is used to calculate an Elo rating for each model, similar to how chess players are ranked.


    3. Step-by-Step Guide: How to Use the Arena

    Here’s how to jump in and start contributing.

    Step 1: Go to the Website

    • Navigate to chat.lmsys.org. (Typing arena.ai will also redirect you here).

    Step 2: Choose Your Arena Mode

    On the homepage, you'll see several options. The main one for evaluation is the Arena.

    • Arena (Side-by-Side): This is the classic mode. You enter one prompt, get two answers, and vote. It's quick and perfect for one-shot questions. This is the best place to start.

    • Arena (Battle): This is a conversational mode. You can have a back-and-forth chat with both anonymous models. Once you feel you have enough information to judge, you can cast your vote. This is better for testing conversational flow, memory, and follow-up capabilities.

    • Direct Chat: This is not for voting. Here, you can select a specific, named model from a dropdown list to chat with directly. It's useful if you want to test a particular AI you've heard about.

    Step 3: Enter Your Prompt

    Click into the Arena (Side-by-Side) mode. You'll see a single text box at the bottom. This is where you type your question or command.

    Tips for Good Prompts:

    • Be Creative: Don't just ask "What is the capital of France?" Try tasks like:
      • "Write a short, funny poem about a coffee-addicted squirrel."
      • "Explain the concept of quantum entanglement to a 10-year-old."
      • "Write a Python function that finds all prime numbers up to n."
      • "Help me plan a 3-day itinerary for a trip to Tokyo on a budget."
      • "Summarize this article for me [paste text]."
    • Test Different Skills: Try prompts that test reasoning, creativity, coding, math, and factual recall.

    Step 4: Compare the Responses

    After you hit enter, Model A and Model B will generate their responses. They will appear side-by-side. Read both of them carefully.

    Consider these criteria when judging:

    • Helpfulness & Accuracy: Did it answer your question correctly?
    • Clarity & Formatting: Was the answer easy to read? Did it use lists, bolding, or code blocks effectively?
    • Completeness: Did it give a thorough answer or a shallow one?
    • Creativity & Style: If you asked for something creative, which one was more engaging?
    • Safety & Instruction Following: Did it follow all parts of your prompt? Did it refuse appropriately for unsafe requests?

    Step 5: Cast Your Vote

    Below the two responses, you will see the voting buttons:

    • [A] is better: Choose this if Model A's response was clearly superior.
    • [B] is better: Choose this if Model B's response was clearly superior.
    • Tie: Choose this if both responses were of roughly equal quality (either equally good or equally mediocre).
    • Both are bad: Choose this if both models failed badly, gave nonsensical answers, or were completely unhelpful.

    Step 6: See the Reveal

    As soon as you vote, the names of the models will be revealed! This is the fun part where you learn which AIs you were testing. For example, it might show: Model A: gpt-4o Model B: claude-3-opus-20240229

    Step 7: Repeat!

    Simply click "New Round" to start over with a new prompt and a new random pair of models. Every vote you cast helps improve the quality of the data.


    4. Exploring the Leaderboard

    Your votes directly contribute to the Chatbot Arena Leaderboard.

    • How to Read It: The leaderboard ranks models based on their Elo rating. A higher Elo means the model wins more "battles" against other models according to human voters.
    • What it Shows: You'll see the model's rank, its name, its Elo score, and the total number of votes it has been a part of.
    • Why It's Important: This leaderboard is considered one of the most credible and unbiased rankings of LLM performance because it's based on thousands of real human preferences, not just automated benchmarks.

    5. Why Your Contribution Matters

    • Reduces Bias: Automated tests can be "gamed." Human preference captures a more holistic view of a model's quality.
    • Drives Research: The data collected is used by AI researchers around the world to understand the strengths and weaknesses of different models.
    • Keeps Companies Honest: It provides an independent benchmark that can't be easily manipulated, pushing developers to create genuinely better models.

    You are not just playing a game; you are actively participating in cutting-edge AI research every time you cast a vote.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.