Skip to content
TheChatArchiveThe Chat Archive
Geminigemini-2.5-flash-preview-04-17· GoogleHugging Face LMArena Human Preference 140K

Show me the leaderboard for local models smaller than 15B

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-opus-4-20250514’s answer.
  1. User

    Show me the leaderboard for local models smaller than 15B

  2. Gemini Assistant

    Okay, here is a look at some of the top-performing local models with fewer than 15 billion parameters, based on common benchmarks often tracked on leaderboards like the Hugging Face Open LLM Leaderboard.

    It's important to note:

    1. Leaderboards are Dynamic: The exact rankings change frequently as new models are released and evaluated.
    2. Benchmark != Real-World Performance: Benchmarks like MMLU, HellaSwag, etc., measure specific capabilities (reasoning, commonsense, knowledge) but don't always capture subjective qualities like creativity, helpfulness, or refusal behavior.
    3. "Local" Suitability: While these are the best performers under 15B on benchmarks, actual performance on your hardware will depend on the model's specific architecture, the quantization method used (like GGUF, AWQ, GPTQ), and your system's VRAM/RAM.

    Based on a snapshot of recent performance on common benchmarks for models under 15B, here are some notable entries (not an exhaustive list, and order can vary slightly day-to-day):

    Top Performers (Under 15B)

    • Qwen 1.5 (various sizes): Qwen 1.5 has performed exceptionally well across multiple parameter counts.
      • Qwen 1.5 14B: Often ranks very highly, pushing the upper limit of the 15B cap. Provides strong performance if you have the VRAM (around 16GB+ for unquantized/lightly quantized).
      • Qwen 1.5 7B: A top contender in the 7B category, offering excellent performance for its size and running well on systems with 8GB-12GB VRAM.
    • Llama 3 8B: Meta's Llama 3 8B has set a new standard for performance in the 8B category, often outperforming models from previous generations that were much larger. Highly popular for local use due to its strong capabilities and availability in various quantized formats.
    • Cohere For AI Aya 23 8B: A powerful multilingual model that often scores very well on benchmarks, offering strong capabilities beyond just English.
    • Command R+ Light (10B): Cohere's smaller Command R+ model is designed for business use cases, often scoring well on benchmarks related to RAG and tool use, while staying within the <15B limit.
    • Gemma (various sizes): Google's open models.
      • Gemma 7B: A strong performer in the 7B class, known for its compact architecture relative to its performance.
      • Gemma 2B: While much smaller, it's often a top performer in its own tiny category, suitable for even more constrained devices. (Though the user asked for <15B, 2B is technically included and notable for its efficiency).
    • Mistral 7B: Still a solid and very popular choice in the 7B category, known for its efficiency and strong base performance, leading to many excellent fine-tunes.
    • Nous Research models (various sizes, e.g., Hermes): Fine-tunes like Nous Hermes often take top base models (like Mistral, Llama, Qwen) and improve their instruction following and chat capabilities, sometimes leading to higher scores on specific instruction-tuned benchmarks or subjective evaluations.

    Where to See the Live Data:

    The best place to see the absolute latest ranking is the Hugging Face Open LLM Leaderboard:

    On that page, you can usually filter the results by parameter count (Max Params) to specifically see models under 15B and sort them by the "Average" score.

    Look for models around the 7B, 8B, 10B, and 14B parameter counts, as these are typically the sizes that fall under the 15B limit and compete for the top spots in this category.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.