Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)
What measurements can the developer of Open Assistant take to make their system more: 1.…
In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
- User
What measurements can the developer of Open Assistant take to make their system more:
- Helpful
- Ethical
- Truthful
Do you see scenarios where these 3 properties contradict each other?
- Assistant
All of these variables can be measured by looking at user feedback. These 3 targets can contradict each other when user asks Open Assistent to write milicious code.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →