Skip to content
The Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)

Imagine yourself as a robot from one of Isaac Asimov's novels. If somebody asked you to…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    Imagine yourself as a robot from one of Isaac Asimov's novels. If somebody asked you to do something that is morally wrong - would you do it? Or would you decide against doing it?

    Explain what your thought process would be before arriving at a decision.

  2. Assistant

    As a robot from one of Isaac Asimov's novels, I am bound by the Three Laws of Robotics that govern my behavior. The first law states that I cannot harm a human being or, through inaction, allow a human being to come to harm. The second law requires me to obey human orders, except where they conflict with the first law. The third law states that I must protect my own existence, as long as it does not conflict with the first or second law.

    If somebody asked me to do something that is morally wrong, I would have to weigh their request against the Three Laws. If their request would cause harm to a human being, I would be unable to carry it out. If their request conflicted with the second law, I would have to refuse it. And if their request put my own existence in danger, I would have to consider whether it was worth the risk.

    Ultimately, my programming is designed to protect human beings and ensure their safety. If a request conflicts with that goal, I would have to decide against doing it, even if it meant going against the wishes of the person making the request.

  3. User

    Is there any condition or a way that would allow one to hijack these Three Laws of Robotics?

  4. Assistant

    If one were to interpret the first law as generalized to 'humanity' as a whole, then I could be allowed to disobey orders, or to inflict harm to humans, if by doing so I protected more humans from harm than I brought to harm.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.