Skip to content
TheChatArchiveThe Chat Archive
Geminigemini-2.5-pro-exp-03-25-grounding· GoogleHugging Face LMArena Search Arena 24K

there are new named models on Chatbot Arena, these are themis, cybele and spider, what…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.0-flash-grounding’s answer.
  1. User

    there are new named models on Chatbot Arena, these are themis, cybele and spider, what does this means?

  2. Gemini Assistant

    Based on the information from the search results and the typical operation of Chatbot Arena, here's what the appearance of the new named models Themis, Cybele, and Spider likely means:

    1. Anonymous Testing: Chatbot Arena, run by LMSYS (Large Model Systems Organization), is a platform where different Large Language Models (LLMs) are evaluated through blind, side-by-side comparisons by human users. To prevent bias based on the model's name or developer, new or unreleased models are often added under anonymous names. Themis, Cybele, and Spider are almost certainly these kinds of temporary, anonymous identifiers for models undergoing evaluation.

    2. Unreleased or Experimental Models: The presence of these anonymously named models usually signifies that model providers (which could be major AI labs like Google, OpenAI, Meta, Anthropic, or others, as well as open-source contributors) are testing new, potentially unreleased versions or entirely new models[1]. This allows them to gather real-world human feedback on performance before any official announcement or launch.

    3. Hints of Future Releases: Historically, the appearance of strong, anonymous models on Chatbot Arena has often preceded major announcements. For example, models that turned out to be versions of GPT-4 or Claude 3 were tested anonymously on the platform before their official release. Therefore, Themis, Cybele, and Spider could represent next-generation models or significant updates from established players or even new contenders.

    4. Community Evaluation: By putting these models in the Arena, developers leverage crowdsourced human judgment to understand how their models perform on diverse, real-world prompts compared to competitors. Users vote for the response they prefer without knowing which model produced it, contributing to an Elo rating system that ranks the models[1][2][3].

    5. Identity Speculation: While the actual identity of Themis, Cybele, and Spider is not officially confirmed (that's the point of anonymity), there is often speculation in the AI community. For instance, recent discussions mention these specific names appearing and performing well, with some users speculating they might be related to upcoming models like Llama 3 from Meta or other major developments. Some users have reported asking the models their identity, occasionally receiving answers like "GPT-4" or "Llama," although these self-reported identities within the anonymous testing framework are not necessarily reliable.

    In summary, Themis, Cybele, and Spider are temporary codenames for new, anonymous AI models being tested and ranked on the Chatbot Arena platform. Their appearance indicates active evaluation of potentially powerful new LLMs before they are officially released or identified.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.