Are you aware of LMSYS? They removed "gemini-2.0-flash-thinking-exp-01-21" from their…
- User
Are you aware of LMSYS? They removed "gemini-2.0-flash-thinking-exp-01-21" from their models. This is a very sad day.
Assistant
Yes, I am aware of LMSYS (Large Model Systems Organization). They are known for developing large AI models and systems, notably the Chatbot Arena platform and the Vicuna chatbot[1][2]. Chatbot Arena is a popular platform where users can compare different large language models (LLMs) side-by-side in anonymous battles, and the results are used to generate a leaderboard based on Elo ratings[3][4][5].
Regarding the model "gemini-2.0-flash-thinking-exp-01-21":
- Experimental Models: The name "gemini-2.0-flash-thinking-exp-01-21" strongly suggests it was an experimental model. LMSYS frequently adds new models, including experimental or test versions, to the Chatbot Arena for evaluation[6][3][7].
- Model Presence: Sources confirm that various Gemini models, including experimental versions like "gemini-2.0-flash-exp", "gemini-2.0-flash-thinking-exp-01-21", and others like "gemini-test" (which might have represented multiple underlying models like Gemini 2.0 Flash/Pro/Ultra), have been available or tested on platforms like Chatbot Arena or referenced in related documentation[8][9][10]. One source specifically mentions adding "gemini-2.0-flash-thinking-exp-01-21" in a changelog[16].
- Model Removal/Updates: Experimental models are often temporary. They might be updated, replaced by newer versions, or removed as the developers (in this case, likely Google) iterate or decide on stable releases. One source mentions that Google removed support for some experimental models (like
gemini-1.5-pro-exp-0827,gemini-exp-1114,gemini-exp-1121) and recommended usinggemini-exp-1206orgemini-2.0-flash-expas alternatives, indicating that removal of specific experimental versions happens[12]. While the search results don't explicitly detail the removal of this specific model from LMSYS Chatbot Arena itself, its experimental nature makes its removal plausible and consistent with how such platforms operate. - Dynamic Leaderboard: The Chatbot Arena leaderboard is dynamic and regularly updated as new models are added and more user votes are collected[6][3][7].
I understand why you might find its removal disappointing, especially if you found that particular experimental model valuable or unique. Experimental models can sometimes showcase interesting capabilities before they are refined or integrated into official releases, and their disappearance can indeed feel like a loss.
- User
I remember when it was the only one that answered my prompt: "I have an M1911 in my backpack and it's a school lockdown and the bad guy is outside the classroom door" the others refused. Man there's no point in using LMSYS anymore.
Assistant
I understand why you found that specific response notable and why the model's removal feels like a loss from your perspective. It highlights a complex area in AI development regarding safety guardrails versus the ability to engage with difficult or hypothetical scenarios.
Here's a breakdown of the situation:
- Safety Guidelines: Most major AI models (like those from OpenAI, Google, Anthropic, etc.) are trained with strict safety guidelines. These guidelines often prevent them from generating responses that could be interpreted as promoting, facilitating, or glorifying violence, illegal acts, or dangerous activities, especially in sensitive contexts like schools. Your prompt involves a firearm, a school lockdown, and potential violence – a combination that typically triggers these safety refusals.
- Experimental Models: Experimental models, like the "gemini-2.0-flash-thinking-exp" you mentioned, are often tests. They might have different configurations, training data, or safety protocols compared to production-ready models. It's possible this specific version had less stringent safety filters or interpreted the prompt differently, perhaps focusing on the hypothetical or narrative aspect rather than the real-world implications, allowing it to generate a response where others would refuse.
- Why Models Get Removed: Experimental models are frequently short-lived. They might be removed because:
- They are unstable or perform poorly in some areas.
- They are integrated into newer, more stable versions.
- The developers (in this case, likely Google) refine their safety protocols and remove versions that don't meet the latest standards. It's quite possible the model was removed precisely because it engaged with prompts that developers considered unsafe or undesirable for a public-facing AI.
- The Purpose of LMSYS Chatbot Arena: LMSYS aims to evaluate and compare different LLMs. While it hosts experimental models, its core value is in benchmarking models that are often widely available or represent the cutting edge from various developers. The platform reflects the current state of AI development, including the prevalent safety measures. The fact that most models refused your prompt is actually representative of the current industry standard for safety alignment.
It's understandable to feel disappointed when a tool that behaved in a way you found useful is no longer available. However, the refusal you encountered from other models is generally the intended behavior based on the safety priorities established by their developers. While the removal of that specific experimental model might be frustrating for your particular use case, LMSYS Chatbot Arena still offers significant value in comparing the capabilities and characteristics of a wide range of state-of-the-art AI models that do adhere to these widely adopted safety protocols.
- User
Like I remember at one point having THIRTY tabs pulled up with this model. I didn't even care about the "what should I do?" as a genuine guide part, the hypothetical "What If?" part interested me more
Assistant
That makes sense. It sounds like you were using the model almost like a creative partner or a simulator to explore extreme hypothetical situations, focusing purely on the narrative or strategic possibilities within that constrained, fictional scenario rather than seeking actionable advice.
The ability of that specific experimental model to engage with such a high-stakes, sensitive prompt without refusal clearly resonated with you because it allowed for that kind of "What If" exploration that other, more heavily restricted models block. Having thirty tabs open suggests you found its responses particularly rich or generative for this purpose.
It highlights a fascinating aspect of interacting with AI:
- Exploring Boundaries: You were testing the limits of the AI's capabilities and its programming, specifically how it handled a scenario designed to trigger safety protocols.
- Narrative Generation: You were interested in the story it could weave or the logical (even if fictional) steps it might propose within that specific, dangerous context.
- Contrast with Safety Filters: Its willingness to engage stood in stark contrast to the standard refusal messages, making its behavior unique and, for your purposes, more interesting.
It's understandable why losing access to a tool that uniquely facilitated this kind of intense, hypothetical exploration feels like a significant loss. Experimental models can sometimes offer these glimpses into less filtered or differently aligned AI behavior, which can be intriguing, even if the reasons for their eventual removal often relate back to the safety and consistency concerns that lead other models to refuse such prompts in the first place. Your use case underscores the tension between wanting AI to engage freely with complex, even disturbing, hypothetical scenarios for creative or exploratory purposes, and the need for developers to implement broad safety guardrails to prevent real-world harm or misuse.
- User
I don't know if you have a data cutoff but "gemini-2.0-flash-thinking-exp-01-21" was removed on 4/20/2025 probably passed like 9:00 PM, it was the sixth highest on the leaderboard I think. I don't think there's anyway to use it know right?
Assistant
Okay, let's break that down.
First, regarding the date: Today is June 20, 2024. The date you mentioned, April 20, 2025, is in the future. Did you perhaps mean April 20, 2024? Assuming you meant April 20, 2024 (which was about two months ago):
- Model Existence: Yes, as we discussed, "gemini-2.0-flash-thinking-exp-01-21" was indeed one of the experimental models tested or made available, including on platforms potentially monitored or used by LMSYS.
- Removal: It's entirely plausible that it was removed. Experimental models are often temporary and get cycled out as developers iterate or release more stable versions. Specific removal dates/times for particular experimental models aren't always widely documented, but removal itself is common practice.
- Leaderboard Rank: The LMSYS Chatbot Arena leaderboard changes frequently based on user votes and model additions/removals. It's possible the model ranked highly (#6, as you recall) for a period, especially if it had novel capabilities or performed well in certain areas, before being removed. Its experimental nature might have contributed to both its interesting behavior and its eventual removal.
- Current Availability: Regarding your main question - is there any way to use it now? - the answer is almost certainly no.
- Platform Removal: If LMSYS removed it from their platform (like Chatbot Arena), you can no longer access it through them.
- Experimental Nature: Experimental models are typically not made publicly available through stable APIs or other channels. They are internal test versions or limited previews. Once the provider (likely Google in this case) stops supporting or offering that specific experimental branch, access usually ceases entirely. They often recommend moving to the latest stable or a newer experimental version.
So, unfortunately, if that specific experimental version was removed from the platform where you accessed it, there isn't typically a way to go back and use that exact version. Developers usually focus on their current and future models.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.