Skip to content
TheChatArchiveThe Chat Archive
Geminigemini-2.5-pro-preview-05-06· GoogleHugging Face LMArena Human Preference 140K

what is the best scoring OPEN SOURCE llm

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-sonnet-4-20250514’s answer.
  1. User

    what is the best scoring OPEN SOURCE llm

  2. Gemini Assistant

    This is a rapidly evolving field, and the "best" can change weekly! However, as of my last update (and generally holding true for early-mid 2024):

    For the highest raw benchmark scores among generally accessible open-source (or at least open-weight) models, the top contenders typically include:

    1. Meta's Llama 3 (especially the 70B version):

      • Why it's often at the top: Llama 3, particularly the 70B instruct-tuned version, has shown exceptional performance across a wide range of benchmarks (MMLU, GSM8K, HumanEval, etc.), often surpassing previous open-source leaders and even rivaling some closed-source models on specific tasks.
      • Openness: Weights are publicly available, and the license is more permissive than Llama 2's, making it widely usable.
    2. Mistral AI's Mixtral 8x22B:

      • Why it's a strong contender: This Mixture of Experts (MoE) model offers incredible performance, often sitting very close to or at the top of leaderboards. MoE allows for very large parameter counts with more efficient inference than a dense model of equivalent size.
      • Openness: Apache 2.0 license, weights available.
    3. Alibaba's Qwen1.5 (e.g., Qwen1.5-72B-Chat):

      • Why it's a strong contender: The Qwen series, particularly the larger models, consistently performs very well on benchmarks, especially with strong multilingual capabilities and coding.
      • Openness: Weights available, generally permissive license.
    4. Cohere's Command R+:

      • Why it's notable: While Cohere's models aren't always "open source" in the most permissive FOSS sense (check licensing for commercial use), Command R+ is a very powerful model with weights available for research and some forms of deployment. It scores very highly on benchmarks, particularly for enterprise use cases like RAG and tool use.
      • Openness: Weights available, but license has specific conditions.

    Key Considerations & How to Stay Updated:

    • Benchmarks are not everything: While scores are indicative, real-world performance on your specific task can vary. Human evaluation (like on Chatbot Arena) often gives a different perspective.
    • Size vs. Performance: Larger models (70B+) generally score higher but require more resources. Excellent smaller models like Llama 3 8B, Mistral 7B, Mixtral 8x7B, and Gemma 7B offer great performance for their size.
    • Fine-tuning: The "best" base model can also depend on how well it fine-tunes for your specific dataset and task.
    • "Open Source" vs. "Open Weights": Ensure the license meets your needs. Most of the above are genuinely open or have very permissive licenses.
    • Where to check:
      • Hugging Face Open LLM Leaderboard: This is the go-to resource for comparing open LLMs on a standardized set of benchmarks. (leaderboard.co)
      • Chatbot Arena (LMSys): Ranks models based on human preference in blind chatbot battles.
      • Papers With Code: Tracks SOTA on various NLP benchmarks.

    In summary: Right now, Llama 3 (70B) is widely considered the top-scoring generally accessible open-source LLM, with Mixtral 8x22B and Qwen1.5-72B as extremely strong competitors.

    Always check the Hugging Face Open LLM Leaderboard for the latest rankings!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.