Skip to content
TheChatArchiveThe Chat Archive
stripedhyena-nous-7b· Together AIHugging Face Arena Human Preference 55K

Where was the self-attention idea first proposed?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with starling-lm-7b-alpha’s answer.
  1. User

    Where was the self-attention idea first proposed?

  2. Assistant

    The self-attention mechanism was first proposed by a computer scientist named Richard S. Shannon in 1951. Shannon's work laid the groundwork for information theory, which is the study of how information is encoded, stored, and transmitted. In one of his pioneering papers, "A Mathematical Theory of Communication," Shannon introduced the concept of self-information, which later evolved into the idea of self-attention. The self-attention mechanism is a key component in many modern neural network architectures, particularly in transformer models, which are widely used for natural language processing and other tasks.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.