give me 3 complicated and short examples to test llms
- User
give me 3 complicated and short examples to test llms
Assistant
Here are three concise, challenging examples to test LLM reasoning capabilities:
1. Physical Reasoning
Question: A strawberry is placed in a cup, which is then flipped upside down. What is the temperature of the strawberry?
Answer: Room temperature (the strawberry falls out when inverted).
Test Focus: Physical causality and object permanence. LLMs often assume the strawberry remains in the cup, failing to account for gravity[[6]].
2. Multi-Step String Manipulation
Question: Take the last letter of each word in "John Doe Smith" and concatenate them. What is the result?
Answer: "neh" (J→n, D→e, S→h).
Test Focus: Sequential processing and attention to individual components. Requires breaking down the task into word-level operations[[2]].
3. Contextual Pronoun Resolution
Question: The trophy didn’t fit into the brown suitcase because it was too small. What was too small?
Answer: The suitcase.
Test Focus: Winograd schema-style reasoning to resolve ambiguous pronouns. LLMs often misattribute "it" to the trophy instead of the suitcase[[3]][[8]].
These examples target physical reasoning, algorithmic processing, and contextual ambiguity—areas where LLMs frequently struggle despite human ease[[6]][[8]].
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.