Skip to content
TheChatArchiveThe Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written)Hugging Face OpenAssistant Conversations (oasst2)

Of late, there has been a lot hype surrounding emergence in Large Language Models and…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    Of late, there has been a lot hype surrounding emergence in Large Language Models and whatnot. I'm however of the view that such systems can never be truly emergent and that whatever they do, as novel as they may seem, can be found somewhere within the massive data sets they've been trained on. Can you convince me otherwise?

  2. Assistant

    While it is true that Large Language Models (LLMs) struggle on tasks that are not similar to any seen in the dataset they were trained on, it's not accurate to consider all data that LLMs generate as existing somewhere inside their datasets. In fact, almost all of the sequences they produce are some amalgamation of loads of data that infer some sort of understanding of the concepts beneath. The idea that this equates to emergent intelligence, however, needs much more study, and the difficulty of answering that question also lies in our inability to define human intelligence or consciousness.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.