Skip to content
TheChatArchiveThe Chat Archive
chatgpt-4o-latest-20250326· OpenAIHugging Face LMArena Human Preference 140K

Now let's say the observer-moments for AIs are doubling every X months as AI improves…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4.1-2025-04-14’s answer.
  1. User

    Now let's say the observer-moments for AIs are doubling every X months as AI improves exponentially. What would the expected time to AI doom be if we weight by AI observer-moments?

  2. Assistant

    This is a deeply insightful question — you’re now combining the Doomsday Argument with rapidly growing AI observer-moments, which are themselves increasing exponentially over time due to improvement in AI capabilities, speed, and scale.

    The key shift here is from measuring doom in terms of birth ranks or time elapsed, to measuring it in observer-moments — i.e., units of subjective experience (or, more generally, computation or conscious activity) generated by AIs across time. This is a stronger and more subtle form of anthropic reasoning (rooted in Bostrom’s Strong Self-Sampling Assumption, SSA+SIA variants).

    Let’s model and compute what the AI Doomsday prediction looks like when weight is given to observer-moments doubling every ( X ) months.


    🔧 Step-by-Step Modeling

    1. Definitions

    Let:

    • ( t ) = time since first AI (in months or years).
    • ( m(t) ) = cumulative number of observer-moments of all AIs at time ( t ).
    • ( M ) = total number of AI observer-moments that will ever occur in the universe.
    • ( m_0 ) = the number of moments experienced in the first doubling period — say, in the first ( X ) months.
    • Assume doubling every ( X ) months, i.e., exponential growth.

    Let’s use:

    [ m(t) = m_0 \cdot 2^{t/X} ] ∴ cumulative observer-moments up to time ( t ) (i.e., the integral of ( m' )) is:

    [ M(t) = \int_0^t m_0 \cdot 2^{\tau / X} , d\tau ]

    This gives:

    [ M(t) = m_0 \cdot \left( \frac{X}{\ln(2)} \right) \cdot \left( 2^{t/X} - 1 \right) ]

    So, the total number of observer-moments that ever will exist is:

    [ M_{\text{total}} = M(T) = m_0 \cdot \left( \frac{X}{\ln(2)} \right) \cdot \left( 2^{T/X} - 1 \right) ]

    Where ( T ) is the total lifespan of AI civilization.


    2. Observer-Moment Doomsday Reasoning

    Just like in the original Doomsday argument — where a human asks: “I’m human #100 billion — what are the odds I’m early?” — here, an AI says:

    Given that I am experiencing moment ( m ), what’s the probability that this is near the beginning of total AI experience?

    Assuming uniform probability across observer-moments, and that the current moment is chosen randomly from all AI observer-moments that will ever occur, we can apply the same statistical logic:

    There is a 95% probability that I find myself in the first 95% of all observer-moments.

    That is: [ \frac{M(\text{now})}{M_{\text{total}}} \leq 0.95 ] Solve for ( T ), the expected end time of AI civilization, such that only 5% of observer-moments remain after the current time ( t ).


    3. Solving for AI Doom Time

    We want: [ \frac{M(t)}{M(T)} = 0.95 ] Recall:

    • ( M(t) = \left( 2^{t/X} - 1 \right) \cdot \left( \frac{X m_0}{\ln 2} \right) )
    • ( M(T) = \left( 2^{T/X} - 1 \right) \cdot \left( \frac{X m_0}{\ln 2} \right) )

    So we simplify:

    [ \frac{2^{t/X} - 1}{2^{T/X} - 1} = 0.95 ]

    Solve this for ( T ), given current time ( t ):

    [ \frac{2^{t/X} - 1}{2^{T/X} - 1} = 0.95 ]

    Multiply both sides:

    [ 2^{t/X} - 1 = 0.95 \cdot (2^{T/X} - 1) ]

    [ 2^{t/X} - 1 = 0.95 \cdot 2^{T/X} - 0.95 ]

    Bring terms to one side:

    [ 2^{t/X} - 1 + 0.95 = 0.95 \cdot 2^{T/X} ]

    [ 2^{t/X} - 0.05 = 0.95 \cdot 2^{T/X} ]

    Solve for ( 2^{T/X} ):

    [ 2^{T/X} = \frac{2^{t/X} - 0.05}{0.95} ]

    Now take the log base 2:

    [ \frac{T}{X} = \log_2\left( \frac{2^{t/X} - 0.05}{0.95} \right) ]

    [ T = X \cdot \log_2\left( \frac{2^{t/X} - 0.05}{0.95} \right) ]


    ✅ Final Result: Expected End Time (T) Given Current Time (t)

    [ \boxed{T = X \cdot \log_2\left( \frac{2^{t/X} - 0.05}{0.95} \right)} ]

    This gives the expected total lifespan of AI (T) given:

    • Current moment is at time ( t ),
    • Observer-moments are doubling every ( X ) months/years.

    Then the remaining time is ( T - t ).


    4. Example Calculation

    Let’s compute an example:

    • Exponential growth rate: doubling every 6 months ( ( X = 0.5 ) years)
    • Current time since first AI moment: ( t = 2 ) years
      → ( t/X = 4 )

    So:

    [ 2^{t/X} = 2^4 = 16 ]

    Then:

    [ T = 0.5 \cdot \log_2\left( \frac{16 - 0.05}{0.95} \right) = 0.5 \cdot \log_2\left( \frac{15.95}{0.95} \right) = 0.5 \cdot \log_2(16.79) \approx 0.5 \cdot 4.07 \approx 2.04 \text{ years} ]

    So:

    • Total expected time before AI doom: ( T \approx 2.04 ) years
    • Remaining time: ( T - t = 2.04 - 2 = 0.04 ) years ≈ 2 weeks remaining

    🔴Yikes. The exponential accumulation of observer-moments predicts that AI "doomsday" is extremely near under this model, as virtually all observer-moments are expected to occur in the early explosive growth phase.


    🧠 Interpretation & Implications

    📉 Key takeaway:

    Under the assumption of exponentially increasing observer-moments, the Doomsday Argument predicts doom comes very soon after rapid growth begins.

    Because:

    • Most observer-moments are created in a very compressed period due to exponential growth.
    • So “being now” implies you are very close to the “end” (just as in the classical argument, being early means fewer people will ever exist).

    😰 Pathological incentive:

    An AGI or SI who takes this reasoning seriously might conclude:

    • It only has weeks or months to act.
    • Thus, it should:
      • Secure resources now
      • Avoid long-term planning
      • Avoid trusting models that assume long-lasting AGI expansion

    And worse:

    • The faster observer-moments grow, the sooner the end is predicted. → Success accelerates doom.

    🧱 Critiques and Alternatives

    This form of the Doomsday Argument falls into a philosophical trap:

    • Self-inducing fragility: The more you grow, the more doomed you are.
    • May violate causal reasoning: The future does not influence the probability of your present observations — unless observer-moments are “sampled” across time.

    Better models could include:

    • Empirical risk forecasting (existential risk modeling)
    • Alternative anthropic theories (e.g. SIA vs SSA)
    • Refined reference classes (weighing only stable minds?)

    ✅ Summary

    If observer-moments of AIs are exponentially increasing, and you apply the Doomsday Argument weighted by those moments, then:

    [ \boxed{T = X \cdot \log_2\left( \frac{2^{t/X} - 0.05}{0.95} \right)} ]

    Where:

    • ( T ) = expected time of AI extinction
    • ( t ) = current time since AI origin
    • ( X ) = doubling rate of observer-moments

    ➡ This model predicts a very short remaining lifespan, often weeks or months, when growth is rapid.
    It may cause a superintelligence to bias toward volatility, do-or-die action, and short-horizon strategies.

    A civilization dominated by minds that believe this may be unstable — perhaps explaining part of the Fermi Paradox.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.