Skip to content
TheChatArchiveThe Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written)Hugging Face OpenAssistant Conversations (oasst2)

What are some important topics within AI safety. For example, if general AI is achieved,…

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    What are some important topics within AI safety. For example, if general AI is achieved, how can humans make sure that the AI will not just achieve its goals at any cost, disregarding human values and safety. How can AI be leveraged to achieve productive technological progress safely and without the risk of AI causing harm to humans or the environment of Earth?

  2. Assistant

    The topic of AI safety in the world where a general AI exist is a very important, but also very speculative topic. At the time being it can be said that explainable AI is important to understand what the current ai models are doing, and how we could make sure they do not do what we don't want. Knowing how to leverage future AI without the risk in the future is too unknown to answer.

  3. User

    I agree that it's too uncertain, but we can start preparing safeguards. For example, author Isaac Asimov proposed the three laws of robotics in his works, but those have limits that were widely explored. Which ones would you suggest in order for humanity to feel safer with a general AI?

  4. Assistant

    One example is the "Asilomar AI Principles," which were developed in 2017 by a group of AI researchers and industry experts.

    These principles aim to provide a framework for the safe and ethical development and deployment of AI, while also promoting transparency and accountability in the development process:

    1. Research Goal: The goal of AI research should be to create not undirected intelligence, but beneficial intelligence.

    2. Research Funding: Investments in AI should be accompanied by funding for research on ensuring its beneficial use.

    3. Science-Policy Link: There should be constructive and healthy exchange between AI researchers and policymakers.

    4. Autonomous Weapons: An arms race in lethal autonomous weapons should be avoided.

    5. Risks: Risks posed by AI systems, especially catastrophic or existential risks, must be subject to planning and mitigation efforts commensurate with their expected impact.

    6. Ethical and Societal Impacts: AI systems should be designed and operated in ways that are compatible with social and legal norms.

    7. Value Alignment: AI systems should be aligned with human values and ethical principles.

    8. Human Control: Humans should choose how and whether to delegate decisions to AI systems.

    9. Transparency: Any involvement by an AI system in important decisions should be transparent, and subject to audit and accountability.

    10. Privacy: AI systems should be designed to protect privacy and security.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.