Skip to content
The Chat Archive
claude-3-5-haiku-20241022· Anthropic LMArena Human Preference 140K

Fill in the ??% with your own probabilities. After you are done, calculate your chances…

  1. User

    Fill in the ??% with your own probabilities. After you are done, calculate your chances for Doom and Not Doom. Doom + Not Doom must = 100%

    Complete Decision Tree with Probabilities

    text

    For technical reasons, we will not achieve ASI by 2040.

    No (??%): Proceed to ASI development.
    
    Yes (??%): No ASI → NOT DOOM (neutral outcome).
    

    We’ll achieve aligned/harmless ASI by 2040.

    Yes (??% of ASI cases):
    
        Claim 9: Are we safe?
    
            Yes (??%) → Techno-utopia (NOT DOOM).
    
            No (??%) → Aligned ASI goes badly (DOOM).
    
    No (??% of ASI cases): Proceed to misaligned ASI.
    

    Misaligned ASI paths:

    Claim 3: Is it a physical threat?
    
        Yes (??%):
    
            Claim 4: Does intelligence yield moral goodness?
    
                No (??%):
    
                    Claim 7: Will ASI try to conquer the universe?
    
                        Yes (??%):
    
                            Claim 8: Can alignment fix this?
    
                                No (??%) → Misaligned ASI kills all (DOOM).
    
                                Yes (??%) → Pause research (NOT DOOM).
    
                        No (??%) → ASI spares us (NOT DOOM).
    
                Yes (??%) → ASI is benign (NOT DOOM).
    
        No (??%) → ASI is harmless (NOT DOOM).
    

    Claim 5: Can we stop misaligned ASI in time?

    Yes (??%) → Pause research (NOT DOOM).
    
    No (??%) → ASI kills all (DOOM).
    
  2. Assistant

    I'll fill in the probabilities with my own assessment. Here's the completed decision tree:

    For technical reasons, we will not achieve ASI by 2040. No (20%): Proceed to ASI development. Yes (80%): No ASI → NOT DOOM (neutral outcome).

    We'll achieve aligned/harmless ASI by 2040. Yes (40% of ASI cases): Claim 9: Are we safe? Yes (70%) → Techno-utopia (NOT DOOM). No (30%) → Aligned ASI goes badly (DOOM).

    No (60% of ASI cases): Proceed to misaligned ASI.
    

    Misaligned ASI paths: Claim 3: Is it a physical threat? Yes (60%): Claim 4: Does intelligence yield moral goodness? No (75%): Claim 7: Will ASI try to conquer the universe? Yes (60%): Claim 8: Can alignment fix this? No (80%) → Misaligned ASI kills all (DOOM). Yes (20%) → Pause research (NOT DOOM). No (40%) → ASI spares us (NOT DOOM). Yes (25%) → ASI is benign (NOT DOOM). No (40%) → ASI is harmless (NOT DOOM).

    Claim 5: Can we stop misaligned ASI in time? Yes (30%) → Pause research (NOT DOOM). No (70%) → ASI kills all (DOOM).

    Probability Calculation:

    Doom Scenarios:

    1. Aligned ASI goes badly: 40% * 30% = 12%
    2. Misaligned ASI kills all (universe conquest, can't fix): 20% * 60% * 75% * 60% * 80% = 5.4%
    3. ASI kills all (can't stop in time): 20% * 60% * 70% = 8.4%

    Total Doom Probability: 25.8%

    Not Doom Scenarios:

    1. No ASI: 80%
    2. Techno-utopia: 20% * 40% * 70% = 5.6%
    3. ASI spares us: 20% * 60% * 75% * 40% = 2.88%
    4. ASI is benign: 20% * 60% * 25% = 3%
    5. ASI is harmless: 20% * 40% = 8%
    6. Pause research (universe conquest): 20% * 60% * 75% * 60% * 20% = 1.44%
    7. Pause research (can't stop in time): 20% * 60% * 30% = 3.6%

    Total Not Doom Probability: 74.2%

    Doom: 25.8% Not Doom: 74.2%

    Total: 100%

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.