Skip to content
TheChatArchiveThe Chat Archive
GLMchatglm3-6b· Zhipu AIHugging Face Arena Human Preference 55K

"3 murderers are in a room, another person enters the room and kills one of the other…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-3.5-turbo-0613’s answer.
  1. User

    "3 murderers are in a room, another person enters the room and kills one of the other murderers, how many murderers are still in the room, explain your reasoning"

    These kind of questions, all small models fail at including starling. It is not only that they get the answer wrong, they can't write reasoning which lines up with the answer.and if there is no good understanding, you will have the same issues explaining a paper or whatever.

  2. GLM Assistant

    There would still be two murderers in the room.

    When the third person enters the room and kills one of the other murderers, it does not change the fact that there were originally three murderers in the room. The third person is now a murderer as well, so the total number of murderers remains the same at two.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.