Skip to content
TheChatArchiveThe Chat Archive
Googlegemma-3-27b-it· GoogleHugging Face LMArena Human Preference 140K

Who is the best opensource LLM

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-3-5-sonnet-20241022’s answer.
  1. User

    Who is the best opensource LLM

  2. Google Assistant

    Okay, let's break down the "best" open-source LLM. It's a surprisingly complex question! There's no single definitive answer, as "best" depends heavily on your specific needs. Here's a comprehensive overview, categorized by strengths, with current leaders as of late 2023 / early 2024. I'll also give a "sweet spot" recommendation for general use.

    Important Considerations Before We Start:

    • License: Open-source licenses vary. Some are very permissive (allowing commercial use with few restrictions), while others are more restrictive. Check the license before using a model.
    • Model Size: Larger models (measured in parameters – 7B, 13B, 70B, etc.) generally perform better, but require more computational resources (RAM, GPU) to run.
    • Training Data: The data a model is trained on significantly impacts its capabilities.
    • Fine-tuning: Most open-source LLMs benefit greatly from fine-tuning on a specific dataset for your use case. A generally good model can become excellent with targeted fine-tuning.
    • Quantization: This reduces the precision of the model's weights, making it smaller and faster, but potentially with some loss of accuracy. Important for running large models on limited hardware.
    • Context Window: How much text the model can "remember" at once. Longer context windows are crucial for tasks like summarizing long documents or having extended conversations.

    Here's a breakdown of leading open-source LLMs, categorized by their strengths:

    1. For Overall Performance & General Use (The "Sweet Spot" Right Now):

    • Mistral 7B / Mixtral 8x7B: These models from Mistral AI are extremely popular and often considered the best all-around choice currently.
      • Strengths: Excellent performance for their size, especially Mixtral. Mixtral uses a "Mixture of Experts" architecture, giving it a performance closer to much larger models. Good reasoning, coding, and general knowledge. Permissive Apache 2.0 license. Fast inference.
      • Sizes: 7B, 8x7B (Mixtral)
      • Context Window: 8k tokens (standard), but can be extended with techniques like RoPE scaling.
      • Where to find it: Hugging Face, Mistral AI website.
      • Why it's a sweet spot: Balances performance, size, and licensing. Mixtral is a standout, often outperforming models much larger than itself.

    2. For Raw Power & Complex Tasks (Requires Significant Resources):

    • Llama 2 (Meta): A very strong contender, and a foundational model for many others.
      • Strengths: Strong general performance, good at following instructions, and widely available. Large community support. Available in multiple sizes.
      • Sizes: 7B, 13B, 70B
      • Context Window: 4k tokens (standard), but can be extended.
      • License: Custom license (generally allows commercial use, but with some restrictions – read carefully).
      • Where to find it: Meta website, Hugging Face.
    • Qwen (Alibaba): A series of models that are gaining traction.
      • Strengths: Excellent performance, particularly in Chinese language tasks, but also strong in English. Available in various sizes.
      • Sizes: 0.5B, 1.5B, 7B, 14B
      • Context Window: 8k tokens (standard)
      • License: Apache 2.0
      • Where to find it: Hugging Face

    3. For Coding:

    • CodeLlama (Meta): Built on Llama 2, specifically fine-tuned for code generation and understanding.
      • Strengths: Excellent at generating code in various programming languages, debugging, and explaining code.
      • Sizes: 7B, 13B, 34B
      • Context Window: Varies by version (some have longer context windows).
      • License: Custom license (similar to Llama 2).
      • Where to find it: Meta website, Hugging Face.
    • WizardCoder: Another strong coding model, often competitive with CodeLlama.

    4. For Instruction Following & Chatbots:

    • Zephyr 7B: Fine-tuned version of Mistral 7B, specifically for chat and instruction following.
      • Strengths: Excellent conversational abilities, good at following complex instructions.
      • Size: 7B
      • Context Window: 8k tokens
      • License: MIT
      • Where to find it: Hugging Face
    • OpenChat 3.5: Another strong chatbot model.

    5. For Smaller Footprint/Edge Devices:

    • TinyLlama: A very small (1.1B parameters) model that aims to achieve good performance despite its size. Good for resource-constrained environments.
    • Phi-2 (Microsoft): A 2.7B parameter model that punches above its weight.

    Here's a quick table summarizing the key players:

    ModelSize(s)StrengthsLicenseResources Needed
    Mixtral 8x7B8x7BOverall performance, reasoningApache 2.0High
    Llama 27B-70BGeneral purpose, instruction followingCustomMedium-High
    CodeLlama7B-34BCodingCustomMedium-High
    Qwen0.5B-14BChinese & English, performanceApache 2.0Low-High
    Zephyr 7B7BChat, instruction followingMITMedium
    TinyLlama1.1BSmall footprintApache 2.0Low
    Phi-22.7BSmall, surprisingly capableMITLow-Medium

    Where to Find and Run These Models:

    • Hugging Face Hub: https://huggingface.co/ (The central repository for open-source models)
    • LM Studio: https://lmstudio.ai/ (A user-friendly GUI for running models locally)
    • Oobabooga's Text Generation Web UI: https://github.com/oobabooga/text-generation-webui (A powerful web interface for interacting with models)
    • Google Colab: (Free cloud-based GPU access – good for experimenting)
    • Cloud Providers: AWS, Google Cloud, Azure all offer services for deploying and running LLMs.

    To help me narrow down the best recommendation for you, could you tell me:

    • What do you want to do with the LLM? (e.g., chatbot, code generation, text summarization, creative writing, translation)
    • What hardware do you have available? (e.g., CPU only, GPU with how much VRAM)
    • Are you comfortable with some technical setup, or do you prefer something very easy to use?
    • Is commercial use a requirement? (This affects license considerations)

    I hope this detailed overview is helpful! The open-source LLM landscape is evolving rapidly, so staying up-to-date is key.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.