Skip to content
TheChatArchiveThe Chat Archive
gpt-4o-mini-search-preview· OpenAIHugging Face LMArena Search Arena 24K

deepseek v3 0324 gpqa benchmark?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Perplexitysonar-reasoning’s answer.
  1. User

    deepseek v3 0324 gpqa benchmark?

  2. Assistant

    DeepSeek-V3 has demonstrated strong performance on the GPQA-Diamond benchmark, achieving a Pass@1 score of 59.1%. This score surpasses that of Claude-3.5, which scored 49.9%, and Qwen2.5, which scored 51.1%, indicating DeepSeek-V3's superior precision on high-difficulty QA tasks[1].

    To ensure that this high performance is not due to data contamination, METR conducted a verification by paraphrasing the GPQA-Diamond questions and evaluating DeepSeek-V3 on these paraphrased questions, as well as on a held-out set of GPQA questions. The results showed that DeepSeek-V3's performance remained consistent across these datasets, suggesting that its high benchmark scores are not the result of memorization from training data[2].

    In summary, DeepSeek-V3's performance on the GPQA-Diamond benchmark reflects its robust capabilities in handling complex question-answering tasks.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.