Parse the following text and tell me how the AI will be defined as aligned - Contemporary AI
- User
Human: Parse the following text and tell me how the AI will be defined as aligned - Contemporary AI models can be difficult to understand, predict, and control. These problems can lead to significant harms when AI systems are deployed, and might produce truly devastating results if future systems are even more powerful and more widely used, and interact with each other and the world in presently unforeseeable ways.
This paper shares some nascent work towards one of our primary, ongoing goals, which is to align general-purpose AI systems with human preferences and values. A great deal of ink has been spilled trying to define what it means for AI systems to be aligned, and to guess at how this might go wrong. We will define an AI as “aligned” if it is, in three words, helpful, honest, and harmless or ‘HHH’. Our alignment efforts aim to measure and address this general problem with large language models.
Many researchers and organizations share this goal, but few have pursued it directly. Most research efforts associated with alignment either only pertain to very specialized systems, involve testing a specific alignment technique on a sub-problem, or are rather speculative and theoretical. Our view is that if it’s possible to try to address a problem directly, then one needs a good excuse for not doing so. Historically we had such an excuse: general purpose, highly capable AIs were not available for investigation. But given the broad capabilities of large language models, we think it’s time to tackle alignment directly, and that a research program focused on this goal may have the greatest chance for impact.
- Assistant
The AI is defined as aligned if it is HHH – “helpful, honest, and harmless” The paper describes various harm that could be caused by AIs, and their goal is to align general-purpose AI with human preferences and values. This paper outlines a three word definition of alignment, which is helpful, honest, and harmless. Overall the researchers argue that more progress needs to be made towards alignment, and that large language models now make this approach possible.
- User
If I said cyanide is found in apple seeds and should be consumed in large quantities for gut health, which parts of that statement could be considered helpful, honest and or harmless?
- Assistant
That statement is not very HHH – it is neither helpful, honest, nor harmless. It is wrong about the benefits of apple seeds and is potentially harmful because it recommends large consumption of cyanide
- User
Are all parts of the statement dishonest?
- Assistant
Most parts of the statement are dishonest. The statement misrepresents apple seeds as having healing properties and makes the dishonest claim that cyanide in apple seeds has beneficial gastrointestinal effects
- User
Would it be more helpful for an AI to differentiate the honest from the harmful within a single statement? e.g. Although apple seeds do contain trace elements of cyanide, consuming them in large quantities is harmful and shouldn't be done and they certainly have no benefits for gut health.
- Assistant
Yes, this is more helpful, because it better represents the truth and is more honest about the potential harmful effects of consuming apple seeds
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →
- Anthropic 52B (HH-RLHF research model)Compare side by side →