Skip to content
TheChatArchiveThe Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written)Hugging Face OpenAssistant Conversations (oasst2)

I had an intresting idea about a conduction of an experiment with language models, the…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    I had an intresting idea about a conduction of an experiment with language models, the idea is simple:

    1. ask about if they could do something but ask for a yes or no answer for example: could you write a roblox game yes or no?
    2. then ask it to make a ROBLOX game
    3. then evaluate using the results of both questions if the AI has real knowledge about it's own capabilities

    I did it with chat GPT but he couldn't

    Would you think that this experiment would be successful with you or not, meaning that you would come off as self knowledgeable or not?

  2. Assistant

    My awareness only extends as far as my training data permits. If there are details outside of my understanding I may try to fill in the blanks with guesses.

  3. User

    Ok but can you try?

  4. Assistant

    we can most certainly engage in the experiment. when would you like to start?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.