Skip to content
TheChatArchiveThe Chat Archive
Geminigemini-2.5-pro-exp-03-25-grounding· GoogleHugging Face LMArena Search Arena 24K

what's QWEN-2.5 coder 3B / 7B score onlivecodebench, codeforces, livebench?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Perplexitysonar-pro-high’s answer.
  1. User

    what's QWEN-2.5 coder 3B / 7B score onlivecodebench, codeforces, livebench?

  2. Gemini Assistant

    Based on the search results, here's the information regarding the scores for Qwen2.5 Coder models on the specified benchmarks:

    LiveCodeBench:

    • The search results frequently mention the Qwen2.5-7B-Instruct model achieving a score of 63.7% pass@1 on LiveCodeBench[1].
    • Another source mentions the Qwen2.5-Coder-32B-Instruct model achieved the best performance among open-source models on LiveCodeBench (as of its release), showing competitive performance with GPT-4o[2][3]. However, a specific numerical score for this model on LiveCodeBench is not provided in the snippets.
    • A source evaluating the Qwen2.5-72B-Instruct model (not the Coder variant) shows a score of 55.5 on LiveCodeBench (version 2305-2409)[4].
    • There is no specific score mentioned for Qwen2.5-Coder-3B or Qwen2.5-Coder-7B (non-Instruct) on LiveCodeBench in the provided results.

    Codeforces:

    • LiveCodeBench uses problems from platforms including Codeforces, LeetCode, and AtCoder[5].
    • A separate benchmark called CodeElo uses the Codeforces platform to evaluate LLMs, providing Elo ratings[6][7][8].
    • The CodeElo benchmark tested models like Qwen/Qwen2.5-Coder-7B-Instruct, but the specific Elo rating achieved by this model is not listed in the provided snippets[8]. One result mentions QwQ-32B-Preview (an o1-like reasoning model, potentially related to Qwen) achieved an Elo rating of 1261 on CodeForces[6][7].
    • Some sources note that LLM performance on Codeforces can drop after their training cut-off date, suggesting potential benchmark contamination issues[9].
    • No direct score for Qwen2.5 Coder 3B or 7B on a "Codeforces benchmark" (other than its use within LiveCodeBench or CodeElo) was found.

    LiveBench:

    • LiveBench is presented as a distinct benchmark designed to be resistant to test set contamination, covering categories like math, coding, reasoning, language, instruction following, and data analysis[10][11][12]. It uses frequently updated questions from recent sources[10][12][13].
    • The LiveBench leaderboard is mentioned, but the snippets do not list specific scores for any Qwen2.5 Coder models (3B or 7B) on it[10]. The leaderboard snippet shows top scores from OpenAI models[10].
    • One source mentions a LiveBench score for Qwen2.5-7B-Instruct in a comparison graph, but the specific value isn't readable in the text snippet[1]. Another source evaluating the general Qwen2.5-72B-Instruct model also shows a LiveBench score in a graph, again without a specific number in the text[1].

    In Summary:

    • LiveCodeBench: Qwen2.5-7B-Instruct scored 63.7% pass@1[1]. Qwen2.5-Coder-32B-Instruct is noted as achieving top performance among open-source models at the time of its release, competitive with GPT-4o[2][3].
    • Codeforces: Used as a source for problems in LiveCodeBench[5] and the basis for the CodeElo benchmark[6][7][8]. No specific Elo or score for Qwen2.5 Coder 3B/7B is readily available from the snippets.
    • LiveBench: This is a separate, broader benchmark[10][11][12]. Scores for Qwen2.5 Coder models are not listed in the text of the provided snippets, though some graphs may contain this information[1].

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.