Could you kindly share the evaluation results for Grok-3's performance on the MMMU…
- User
Could you kindly share the evaluation results for Grok-3's performance on the MMMU benchmark?
- Assistant
Grok 3, xAI's latest AI model, has demonstrated strong performance on the Massive Multi-discipline Multimodal Understanding (MMMU) benchmark, achieving a score of 73.2%[1]. This benchmark evaluates models on tasks that require integrating text and images across various disciplines, including art, business, science, health, humanities, and engineering[2].
In comparison, other leading models have achieved the following scores on the MMMU benchmark:
- Google's Gemini 2.0: 70.7%
- OpenAI's GPT-4o: 69.1%
- xAI's Grok-2: 66.1%
- Anthropic's Claude3 Opus: 59.4%[2]
These results position Grok 3 as a competitive model in multimodal understanding tasks, showcasing its ability to process and interpret complex information involving both text and images.
- User
what about grok03 thinking model?
- Assistant
Grok 3's "Think" mode, introduced in February 2025, enhances its reasoning capabilities by allowing the model to engage in extended problem-solving processes, refining solutions over seconds to minutes[1].
In evaluations, Grok 3 (Think) achieved a 93.3% score on the 2025 American Invitational Mathematics Examination (AIME), demonstrating its proficiency in mathematical reasoning[1].
Additionally, Grok 3 (Think) attained an Elo score of 1402 in the Chatbot Arena, reflecting its strong performance in conversational contexts[1].
These results underscore Grok 3's advanced reasoning and problem-solving capabilities, positioning it as a leading model in AI-driven reasoning tasks.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.