Now let's say the observer-moments for AIs are doubling every X months as AI improves…
- User
Now let's say the observer-moments for AIs are doubling every X months as AI improves exponentially. What would the expected time to AI doom be if we weight by AI observer-moments?
- Assistant
Great question! This is a more sophisticated and realistic scenario, since AI observer-moments (the total subjective experiences or "ticks" of all AIs) can grow much faster than the number of AIs, especially if AI capabilities and deployment are accelerating exponentially.
Let’s formalize the scenario and compute the expected time to "AI doom" (i.e., the end of AI observer-moments), using the Doomsday argument weighted by observer-moments.
Assumptions and Setup
- Let be the cumulative number of AI observer-moments up to time .
- Assume is doubling every months (i.e., exponential growth).
- Let be the current time (we can set for simplicity).
- Let be the cumulative observer-moments up to now.
- Let be the (unknown) total time until AI doom.
Mathematical Model
1. Exponential Growth of Observer-Moments
Let’s model the rate of observer-moment creation as doubling every months:
The cumulative observer-moments up to time :
Let’s compute this integral:
Let , so .
So,
At the current time , (if we start counting now).
2. Doomsday Argument with Observer-Moments
The Doomsday argument, when applied to observer-moments, says:
With 95% confidence, you are in the first 95% of all observer-moments that will ever exist.So, if is the total number of observer-moments that will ever exist, and is the number so far, then:
But with exponential growth, is tiny compared to unless doom is very soon.
3. Solving for Expected Time to Doom
Let be the time from now until the end of AI observer-moments.
Let’s say we are at now, and want to find such that:
But if (we just started), this is undefined.
So, let’s assume we have already accumulated observer-moments up to now, and the rate is currently .From now, the future observer-moments will be:
So, the total observer-moments will be:
The Doomsday argument says:
So,
Plug in the formula for :
Solve for :
Interpretation
- If is small compared to the current rate and doubling time , then is very short.
- As AI observer-moments grow exponentially, the vast majority of observer-moments are in the very near future. So, the Doomsday argument predicts that doom is likely to come within a few doubling times.
In the limit where is negligible compared to the future exponential growth:
If , then .
But if is, say, equal to (i.e., the number of observer-moments in one doubling period), then:
So, doom is expected within about 4 doubling times.
Summary Table
(observer-moments so far) (moments in one doubling period) (in doubling periods) -- (imminent doom) -- --
Conclusion
If AI observer-moments are doubling every months, and you apply the Doomsday argument weighted by observer-moments, then the expected time to AI doom is only a few doubling periods into the future.
- If the exponential growth has just started, doom is predicted to be very soon.
- If the exponential growth has been going on for a while, doom is still predicted within a handful of doubling times.
This is a classic feature of the Doomsday argument under exponential growth: the vast majority of observer-moments are always just about to happen, so the argument predicts doom is always near.
If you want a concrete number, plug in your current , , and into:
But for most realistic values, will be just a few times (the doubling period).
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.