Skip to content
TheChatArchiveThe Chat Archive
DeepSeekdeepseek-v3-0324· DeepSeekHugging Face LMArena Human Preference 140K

During RLHF on LLMs, what type of problems in the domain of physics do LLMs struggle…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Googlegemma-3n-e4b-it’s answer.
  1. User

    During RLHF on LLMs, what type of problems in the domain of physics do LLMs struggle with the most?

  2. DeepSeek Assistant

    During Reinforcement Learning from Human Feedback (RLHF) on Large Language Models (LLMs), physics-related problems often expose several key limitations. Here are the types of physics problems LLMs struggle with the most:

    1. Quantitative & Symbolic Reasoning

    • Struggles: LLMs often fail at precise calculations, unit conversions, or solving equations (e.g., kinematics, thermodynamics).
    • Example:
      • "If a car accelerates at 3 m/s² for 5 seconds, how far does it travel?"
      • LLMs may mix up formulas (e.g., ( d = v_0t + \frac{1}{2}at^2 ) vs. ( d = vt )) or mishandle units.

    2. Counterintuitive or Abstract Concepts

    • Struggles: Problems requiring deep conceptual understanding (e.g., relativity, quantum mechanics) often lead to plausible but incorrect explanations.
    • Example:
      • "Explain why time dilation occurs near a black hole."
      • LLMs might conflate general relativity with Newtonian gravity or misuse terms like "spacetime curvature."

    3. Multi-Step Problem Solving

    • Struggles: Breaking down complex problems (e.g., electromagnetism, fluid dynamics) into logical steps is error-prone.
    • Example:
      • "Calculate the induced EMF in a rotating coil in a magnetic field."
      • LLMs may skip steps (e.g., Faraday’s Law → flux calculation → derivative) or misapply rules.

    4. Visual/Spatial Reasoning

    • Struggles: Interpreting diagrams, graphs, or spatial relationships (e.g., forces in free-body diagrams, optics ray tracing).
    • Example:
      • "A light ray refracts through a prism—sketch the path."
      • Without visual input, LLMs struggle to infer angles or Snell’s Law applications correctly.

    5. Dynamic Systems & Real-World Physics

    • Struggles: Simulating real-world physics (e.g., collisions, pendulum motion) due to lack of iterative computation.
    • Example:
      • "Model the trajectory of a projectile with air resistance."
      • LLMs may omit drag force or fail to integrate differential equations.

    6. Edge Cases & Paradoxes

    • Struggles: Resolving paradoxes (e.g., twin paradox, Maxwell’s demon) or extreme scenarios (e.g., near-light-speed effects).
    • Example:
      • "Why doesn’t an electron collapse into the nucleus classically?"
      • LLMs might not correctly invoke quantum mechanics or orbital stability.

    Why RLHF Doesn’t Fully Fix These Issues

    • RLHF optimizes for human-preferred responses, not necessarily correctness.
    • Physics requires grounded, precise reasoning, while LLMs excel at pattern-matching from training data.
    • Feedback often prioritizes fluency over accuracy, leading to "confidently wrong" answers.

    Mitigation Strategies

    • Hybrid Models: Combining LLMs with symbolic solvers (e.g., Wolfram Alpha for math).
    • Fine-tuning on STEM Corpora: Improving physics-specific knowledge.
    • Prompt Engineering: Asking LLMs to "think step-by-step" (Chain-of-Thought prompting).

    In summary, LLMs struggle most with quantitative rigor, abstract reasoning, and multi-step physics problems, even after RLHF. For reliable physics solutions, domain-specific tools or human verification remain essential.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.