Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-pro-high· PerplexityHugging Face LMArena Search Arena 24K

I am looking for the best open source LLM in summarization tasks

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4o-search-preview-high’s answer.
  1. User

    I am looking for the best open source LLM in summarization tasks

  2. Perplexity Assistant

    When selecting the best open-source large language model (LLM) for text summarization tasks, the choice depends on factors such as performance, model size, and specific use cases. Based on the available information, here are some of the top contenders:

    Top Open-Source LLMs for Summarization

    1. LLaMA 2

    • Overview: Meta's LLaMA 2 is widely regarded as one of the best open-source LLMs for summarization, particularly for local deployments. It supports both extractive and abstractive summarization and is highly adaptable through fine-tuning.
    • Strengths:
      • Excellent balance between performance and adaptability.
      • Suitable for on-premises deployment, ensuring data privacy.
      • Scalable for small and large enterprises.
    • Use Cases: Ideal for industries like healthcare and finance where data security is critical[1][5].

    2. Falcon

    • Overview: Developed by the Technology Innovation Institute, Falcon is known for its speed and efficiency, making it ideal for real-time summarization tasks.
    • Strengths:
      • Multilingual capabilities.
      • Optimized for handling large datasets.
    • Limitations: May require fine-tuning for highly technical or specialized texts[1].

    3. Vicuna

    • Overview: A fine-tuned version of LLaMA, Vicuna excels in interactive summarization workflows and local deployments.
    • Strengths:
      • Enhanced conversational capabilities.
      • Fine-tuning potential for domain-specific tasks.
    • Use Cases: Suitable for industries like legal, healthcare, and education[1][5].

    4. Dolly

    • Overview: Dolly is an instruction-tuned model that performs well in summarization tasks, especially when fine-tuned for specific instructions.
    • Strengths:
      • High ROUGE-L score (0.407), indicating strong summarization performance.
      • Good alignment with human-written references[6].

    5. BERT (and BERTSum)

    • Overview: BERT specializes in extractive summarization by identifying the most relevant sentences from a document.
    • Strengths:
      • Ensures factual accuracy and simplicity.
      • Open-source and highly customizable.
    • Limitations: Lacks abstractive summarization capabilities[1][4].

    6. Bloom

    • Overview: A large-scale open-source model with strong performance in abstractive summarization.
    • Strengths:
      • High ROUGE scores (e.g., ROUGE-L: 0.436).
      • Suitable for long-form content summarization[6].

    Performance Comparison

    ModelParametersROUGE-LKey Features
    LLaMA 27B-13BHighLocal deployment, fine-tuning, data privacy
    Falcon7B0.403Speed, multilingual capabilities
    Vicuna13BModerateConversational summarization, local deployment
    Dolly12B0.407Instruction-tuned, high alignment with references
    BERTSum-ModerateExtractive summarization, factual accuracy
    Bloom176B0.436Abstractive summarization, long-form content

    Recommendations

    • For General Use: LLaMA 2 is the most versatile and reliable choice for summarization tasks, especially when fine-tuned for specific domains.
    • For Speed and Multilingual Needs: Falcon is ideal for real-time summarization and multilingual applications.
    • For Interactive Summarization: Vicuna is a strong choice for workflows requiring conversational capabilities.
    • For Extractive Summarization: BERT and its variants (e.g., BERTSum) are excellent for tasks requiring factual precision.

    Fine-tuning these models for specific tasks and domains can significantly enhance their performance. Additionally, evaluation metrics like ROUGE, BLEU, and BERTScore can help assess the quality of generated summaries.

  3. User

    Also I want that this LLMs be the most recent as possible as we can

  4. Perplexity Assistant

    For the most recent open-source large language models (LLMs) excelling in summarization tasks, here are the top contenders as of 2025:

    1. DeepSeek R1

    • Overview: Released in early 2025, DeepSeek R1 is a cutting-edge Mixture-of-Experts (MoE) model with 671 billion parameters (37 billion activated per token). It is optimized for reasoning and long-form content processing, making it ideal for summarization.
    • Key Features:
      • 128K token context window for handling extensive documents.
      • Superior reasoning and logical inference capabilities.
      • Multilingual support across 20+ languages.
    • Use Cases: Research, technical documentation, and summarization of large texts[1][7].

    2. Qwen2.5

    • Overview: Developed by Alibaba, Qwen2.5 is a versatile instruction-tuned model with up to 72 billion parameters. It supports multilingual tasks and excels in structured output generation.
    • Key Features:
      • 128K token context window.
      • Proficiency in 29+ languages.
      • Strong performance in summarization, coding, and mathematical reasoning.
    • Use Cases: Enterprise applications, content creation, and summarization of structured data[1][6][7].

    3. LLaMA 3.3

    • Overview: Meta's LLaMA 3.3, released in late 2024, is a 70-billion-parameter model optimized for dialogue and long-context understanding.
    • Key Features:
      • 128K token context window for long-form summarization.
      • Multilingual capabilities (8+ languages).
      • High performance across general knowledge and reasoning tasks.
    • Use Cases: Summarization, multilingual applications, and content generation[1][4][7].

    4. Mistral Small 3

    • Overview: Released in early 2025, this 24-billion-parameter model is optimized for low-latency tasks, including summarization.
    • Key Features:
      • 32K token context window.
      • Multilingual support (12+ languages).
      • High efficiency and accuracy for summarization and conversational AI.
    • Use Cases: Real-time summarization and on-device applications[6][7].

    5. Falcon 180B

    • Overview: A massive 180-billion-parameter model, Falcon is known for its strong performance in summarization and reasoning tasks.
    • Key Features:
      • Trained on 3.5 trillion tokens.
      • Multilingual capabilities.
      • High efficiency despite its size.
    • Use Cases: Summarization of complex and multilingual content[2][3].

    6. BLOOM

    • Overview: BLOOM remains a strong contender with 176 billion parameters and multilingual capabilities in 46 languages.
    • Key Features:
      • High ROUGE scores for summarization tasks.
      • Extensive support for long-form content.
    • Use Cases: Translation, summarization, and document classification[3][6].

    Comparison Table

    ModelParametersContext WindowKey StrengthsRelease Year
    DeepSeek R1671B (37B active)128KLong-form summarization, reasoning2025
    Qwen2.5Up to 72B128KMultilingual, structured summarization2025
    LLaMA 3.370B128KMultilingual, dialogue optimization2024
    Mistral Small 324B32KLow-latency summarization2025
    Falcon 180B180B8KMultilingual, efficient summarization2024
    BLOOM176B8KMultilingual, long-form summarization2022

    Recommendation

    For the most recent and advanced summarization capabilities:

    • DeepSeek R1 is the top choice for long-form and complex summarization tasks.
    • Qwen2.5 is ideal for multilingual and structured summarization.
    • LLaMA 3.3 offers a balance of performance and scalability for general summarization needs.

    These models represent the latest advancements in open-source LLMs, ensuring cutting-edge performance for summarization tasks in 2025.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.