Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written)
OpenAssistant Conversations (oasst2)
I had an intresting idea about a conduction of an experiment with language models, the…
In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
- User
I had an intresting idea about a conduction of an experiment with language models, the idea is simple:
- ask about if they could do something but ask for a yes or no answer for example: could you write a roblox game yes or no?
- then ask it to make a ROBLOX game
- then evaluate using the results of both questions if the AI has real knowledge about it's own capabilities
I did it with chat GPT but he couldn't
Would you think that this experiment would be successful with you or not, meaning that you would come off as self knowledgeable or not?
- Assistant
My awareness only extends as far as my training data permits. If there are details outside of my understanding I may try to fill in the blanks with guesses.
- User
Ok but can you try?
- Assistant
we can most certainly engage in the experiment. when would you like to start?
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →