Skip to content
The Chat Archive
gemini-2.5-pro· Google LMArena Human Preference 140K

how was the end of 2023 for LM arena in terms of votes

  1. User

    how was the end of 2023 for LM arena in terms of votes

  2. Assistant

    Of course! The end of 2023 was one of the most exciting and dynamic periods for the LMSYS Chatbot Arena, marked by major new releases that dramatically shook up the leaderboard and generated a surge in user votes.

    Here’s a breakdown of how the end of 2023 looked for the LM Arena in terms of votes and rankings.

    Key Takeaways for Late 2023 (Oct - Dec)

    1. The Rise of a True Open-Source Champion (Mixtral)
    2. Google's Gemini Pro Made a Powerful Debut
    3. GPT-4's Dominance Was Challenged Like Never Before
    4. Overall Vote Count and Engagement Skyrocketed

    1. The Open-Source Revolution: Mixtral's Arrival

    The biggest story from the end of 2023 was the release of Mixtral 8x7B from Mistral AI in December. Its impact on the Arena was immediate and profound.

    • Performance: Based on user votes, Mixtral 8x7B quickly established itself as the undisputed best open-source model.
    • Ranking: It didn't just beat other open-source models; it leapfrogged many powerful proprietary models, including Google's Bard (at the time) and even GPT-3.5-Turbo. For the first time, a freely available open-source model was performing at a level directly competitive with the top tier.
    • Vote Impact: This generated huge excitement and a wave of new votes as users were eager to compare this new open-source marvel against the established giants like GPT-4.

    2. The New Proprietary Challenger: Gemini Pro

    Google released its Gemini family of models in December 2023, and Gemini Pro was quickly added to the Arena.

    • Strong Debut: Gemini Pro entered the leaderboard with a very strong Elo rating. It immediately placed itself in the top cluster of models.
    • Ranking: Votes showed it was highly competitive with, and often preferred over, GPT-3.5-Turbo. It established itself as a clear peer to the best models, just behind the absolute leader, GPT-4-Turbo.
    • Vote Impact: As Google's direct answer to GPT-4, its inclusion drove significant traffic. Voters were curious to see if it could dethrone OpenAI, leading to many direct comparisons and a wealth of new voting data.

    3. The Reigning King: GPT-4 Under Pressure

    Throughout 2023, various versions of GPT-4 had been the untouchable kings of the Arena. By the end of the year, this was still true, but its lead was shrinking.

    • Top Model: The leading model at the end of 2023 was gpt-4-1106-preview (one of the first GPT-4 Turbo models). It consistently held the #1 spot based on hundreds of thousands of votes.
    • The "Laziness" Issue: Interestingly, this period coincided with widespread community reports of GPT-4 becoming "lazier" (i.e., providing shorter or less complete answers). While it still won most of its matchups in the Arena, the new competition from Mixtral and Gemini Pro made its occasional slip-ups more noticeable.
    • The Gap Closed: While GPT-4 Turbo was still #1, the Elo rating gap between it and the next-best models (like Claude 2.1, Gemini Pro, and Mixtral) was visibly smaller than it had been earlier in the year.

    4. Overall Vote Volume and Leaderboard Snapshot

    The intense competition fueled by these new releases led to a massive increase in community engagement and total votes. By the end of December 2023, the LMSYS Arena had collected well over 400,000 human-voted comparisons.

    Here is a simplified snapshot of what the top of the leaderboard looked like near the very end of December 2023, reflecting the results of all these votes:

    1. 🥇 GPT-4-Turbo (gpt-4-1106-preview) - Still the clear leader, but the competition was closer than ever.
    2. 🥈 Claude 2.1 - Held a strong position as a top-tier model, often praised for its detailed and well-mannered responses.
    3. 🥉 Gemini Pro - A powerful new entry that immediately secured a top-3/top-4 position.
    4. Mixtral 8x7B Instruct - The revolutionary open-source model, performing on par with Gemini Pro and beating GPT-3.5.
    5. GPT-3.5-Turbo (-0613 / -1106) - Once the undisputed #2, it was pushed down the rankings by the new, more powerful models.

    Summary

    In short, the end of 2023 was a landmark period for the LM Arena. The votes showed that the AI landscape was no longer a one-horse race. The arrival of Mixtral 8x7B proved that open-source could truly compete at the highest level, while Gemini Pro solidified Google as a top contender, setting the stage for an even more competitive 2024.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.