Skip to content
TheChatArchiveThe Chat Archive
Metallama2-70b-steerlm-chat· MetaHugging Face Arena Human Preference 55K

Spot the irony in this tweet: a model that's 20% worse than another but costs 3 x less…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4-1106-preview’s answer.
  1. User

    Spot the irony in this tweet: a model that's 20% worse than another but costs 3 x less is still way better because it has an intelligence to cost ration of 5x

  2. Meta Assistant

    The irony in the tweet is that although the model that is 20% worse than another may be considered inferior in terms of performance, its significantly lower cost makes it a more attractive option in terms of its "intelligence to cost ratio." By this metric, the cheaper model is five times better than the more expensive one, despite its relative underperformance. This suggests that the value of a model may not solely be based on its raw performance, but also on its cost-effectiveness. In other words, the tweet is highlighting the importance of considering both the performance and cost of a model when evaluating its overall value or usefulness.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.