Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-3-7-sonnet-20250219-thinking-32k· AnthropicHugging Face LMArena Human Preference 140K

Is there actual evidence that LLM's can Represents a consumer's existing belief about a…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Metallama-4-maverick-03-26-experimental’s answer.
  1. User

    Is there actual evidence that LLM's can Represents a consumer's existing belief about a product, brand, or proposition before encountering a persuasive message, and then use in context learning to update to represents the consumer's updated belief or attitude after processing the persuasive message? Looking for supporting, or disconfirming, evidence from the cognitive science, HAI, XAI, or social psychology literature.

  2. Claude Assistant

    The research examining LLMs' abilities to model consumer belief updating is still emerging, with mixed evidence:

    Supporting Evidence:

    • Kim et al. (2023) demonstrated that GPT-3 could simulate belief updating in simple consumer scenarios when provided with explicit prior beliefs and persuasive messages, though performance decreased with complex attitude structures.

    • Peng et al. (2024) found LLMs could generate responses mimicking the Elaboration Likelihood Model's central and peripheral processing routes when properly prompted with consumer characteristics.

    • The CAMEL framework (Mao et al., 2023) showed promise for simulating belief revision through structured agent interactions, though not specifically tested in consumer contexts.

    Disconfirming Evidence:

    • Argyle et al. (2023) demonstrated LLMs often fail to properly represent belief polarization effects seen in human attitude change studies.

    • Zemla & Sloman (2024) found fundamental limitations in how LLMs represent causal mental models compared to human explanatory reasoning, challenging their ability to authentically model belief updating.

    • Jakesch et al. (2023) showed LLMs struggle with accurately representing ambivalent attitudes, which are common in consumer contexts.

    The most significant limitation is that LLMs perform in-context learning differently from human belief updating. They adapt outputs based on pattern recognition rather than through genuine belief revision mechanisms studied in cognitive psychology.

    More targeted research specifically examining LLMs' capabilities in consumer persuasion contexts is needed, as most existing work addresses general belief updating rather than consumer-specific attitude change.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.