grok-3-mini-beta· xAI LMArena Human Preference 140K
When I am interacting in the lmarena chat and ask a follow up, is the original question…
- User
When I am interacting in the lmarena chat and ask a follow up, is the original question being repeated to two new models in order to give them the initial context?
- Assistant
Yes, that's generally how LMArena's Chat-Arena system is designed to handle follow-up questions, though I'll explain it based on the platform's typical behavior to ensure we're on the same page. Let me break this down step by step:
How Follow-Up Questions Work in LMArena Chat
-
Context Preservation:
- When you interact in the Chat-Arena, each follow-up question is treated as part of an ongoing conversation. To provide fair and accurate comparisons between models, the system typically includes the full conversation history (including your original question and any previous responses) in the prompt sent to the selected models.
- This means the original question isn't just "repeated" verbatim; instead, it's incorporated into the context so that both models have the same background information. This helps simulate a natural, multi-turn dialogue and ensures the A/B battle remains equitable.
-
Model Selection and Prompting:
- For each new prompt (like your follow-up), LMArena selects two models (or the same ones if it's a continuing battle) and sends them the complete context. This is based on the Bradley-Terry/Elo system they use, where models are evaluated on their ability to respond coherently in real-time chats.
- Example: If your original question was "What's the capital of France?" and your follow-up is "How far is it from Paris to London?", the models would receive something like: "[Full history: User: What's the capital of France? Model A: Paris. User: How far is it from Paris to London?]"
- This way, both models can build on the conversation, just as they would in a real user interaction.
-
Why This Approach?
- LMArena aims for realistic testing, so maintaining context is crucial. Without it, models might give disjointed or irrelevant responses, which wouldn't accurately reflect their strengths in conversational settings.
- However, keep in mind that the exact implementation could vary slightly based on the platform's updates. For instance, they might truncate very long histories to manage token limits, but the core idea is to keep things contextual.
A Few Tips for Using Chat-Arena
- Check for Variations: If you're in a specific battle mode, the rules might be outlined in the interface. Sometimes, users can opt for "reset" or single-turn modes, but in standard chat, context is preserved.
- Voter Feedback: As a participant, your follow-ups contribute to the live battles, and voters will see the full exchange when deciding winners.
- If Unsure, Test It: Try a simple conversation and review the responses or the battle feed to see how context is handled.
- Documentation: For the most accurate details, I recommend checking LMArena's help section or FAQ on their site, as they might have specifics on chat mechanics.
If you notice any discrepancies or have more details about your experience, feel free to share, and I can refine this explanation further! 😊
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.