Skip to content
The Chat Archive
sonar-reasoning LMArena Search Arena 24K

Compare Gemini 2.5 pro, Claude 3.7 sonnet thinking, deepseek v3 (new),…

  1. User

    Compare Gemini 2.5 pro, Claude 3.7 sonnet thinking, deepseek v3 (new), deepseek-reasoner, o3-mini, in terms of code and logic performance

  2. Assistant

    Coding Performance

    Gemini 2.5 Pro achieves 63.8% on SWE-Bench Verified, excelling in generating complete applications from prompts and handling large codebases with its 1M token context window[1][3]. However, users report integration challenges with code generation[1].

    Claude 3.7 Sonnet Thinking leads in coding benchmarks with 70.3% on SWE-Bench Verified (extended mode), outperforming Gemini. It supports full-stack development and GitHub integration but may overcomplicate solutions[3][5].

    o3-mini (especially the "high" version) is praised for STEM-focused coding and logical reasoning, though it lacks specific SWE-Bench scores. Users highlight its clarity in explaining problem-solving steps[6].

    DeepSeek R1 demonstrates cost efficiency and strong math performance (AIME 2024: 1368), but coding benchmarks are less clear. It is noted for customization and open-source suitability[3][6].

    ModelSWE-Bench VerifiedKey Coding Strengths
    Gemini 2.5 Pro63.8%Large context handling, application generation
    Claude 3.7 Sonnet70.3% (extended)Full-stack dev, GitHub integration
    o3-mini (high)N/ASTEM-focused logic, step-by-step explanations
    DeepSeek R1N/ACost efficiency, math-heavy tasks

    Logic and Math Performance

    Gemini 2.5 Pro dominates in math with 92.0% on AIME 2024 and 84.0% on GPQA Diamond, showcasing advanced reasoning[3][7].

    Claude 3.7 Sonnet Thinking scores 80.0% on AIME 2024 and 84.8% on GPQA Diamond, balancing accuracy with structured problem-solving[3][8].

    DeepSeek R1 excels in math (AIME 2024: 1368), though the scale differs from Gemini/Claude. It is ideal for budget-sensitive, math-intensive tasks[3][6].

    o3-mini (high version) is optimized for STEM logic, with users noting superior performance in pure logic tasks compared to Claude 3.5[6].

    ModelAIME 2024GPQA DiamondKey Logic Strengths
    Gemini 2.5 Pro92.0%84.0%Complex reasoning, science tasks
    Claude 3.7 Sonnet80.0%84.8%Structured problem-solving, hybrid modes
    DeepSeek R11368*71.0%Cost-efficient math, open-source projects
    o3-mini (high)N/AN/ASTEM-focused logic, iterative learning
    Note: DeepSeek R1’s AIME score uses a different scale.

    Reasoning and Context Handling

    Gemini 2.5 Pro offers a 1M token context window (expandable to 2M), enabling analysis of entire codebases or lengthy documents[1][7]. It ranks #1 on LMArena (Elo 1443) for general reasoning[3].

    Claude 3.7 Sonnet supports 128K tokens and features Extended Thinking Mode, allowing users to control reasoning depth. However, this increases token usage (6,500 tokens/task) and latency[2][8].

    o3-mini balances speed and accuracy, with the "high" version prioritizing depth over speed. It is limited to ~50 messages/week in high mode[6].

    DeepSeek R1 provides 128K tokens and open-source flexibility but lags in adoption and generalization[3][6].

    ModelContext WindowReasoning ModeKey Trade-offs
    Gemini 2.5 Pro1M tokensDeep reasoningHigh accuracy, proprietary
    Claude 3.7 Sonnet128K tokensConfigurableBalanced speed/accuracy, mid-cost
    o3-mini (high)N/AFixed depthLimited messages, STEM focus
    DeepSeek R1128K tokensStandardLow cost, open-source limitations

    Pricing and Efficiency

    Gemini 2.5 Pro is proprietary with undisclosed pricing, targeting enterprise use[3].
    Claude 3.7 Sonnet costs 3/Minput,3/M input, 15/M output, suitable for commercial coding[3][5].
    DeepSeek R1 offers 0.14/Minput,0.14/M input, 0.55/M output, ideal for budget projects[3][6].
    o3-mini is included in ChatGPT subscriptions, with usage limits in high mode[6].

    ModelPricing (Input/Output)Best For
    Gemini 2.5 ProProprietaryEnterprise, large-context tasks
    Claude 3.7 Sonnet3/3/15 per M tokensCoding, complex problem-solving
    DeepSeek R10.14/0.14/0.55 per M tokensCost-sensitive, open-source
    o3-miniSubscription-basedSTEM education, research

    Summary

    • Coding: Claude 3.7 Sonnet Thinking leads, followed by Gemini 2.5 Pro.
    • Math/Logic: Gemini 2.5 Pro dominates, with DeepSeek R1 and o3-mini excelling in niche areas.
    • Reasoning: Gemini 2.5 Pro and Claude 3.7 Sonnet offer the deepest analysis, while o3-mini balances speed and depth.
    • Cost: DeepSeek R1 is most affordable; Claude 3.7 Sonnet suits mid-tier budgets.

    DeepSeek v3 (new) and DeepSeek-Reasoner lack sufficient benchmark data in the provided sources.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.