AI 2027

By Richard Price

University of Oxford, Philosophy, Quondam FellowThe Academia.edu Team, People, CEOUniversity of Oxford, All Souls College, Quondam Fellow

I recently read the post “AI 2027” on https://ai-2027.com/
It is a remarkably impressive piece of work. It is written by a team of AI forecasters, including Daniel Kokotajlo and Scott Alexander.
It is by far the most detailed projection of how the AI intelligence explosion might happen.
The strategy of the document is to make reasonably high-confidence forecasts up till the end of 2026; and then continue to build out detailed scenarios, but to couch their forecasts in low-confidence terms. The spirit of the document is “let’s be as concrete as we possibly can, all the way until possibly total AI takeover in 2030.” The document weaves in AI technology; geopolitics; democratic politics; human psychology.
Although AI take-over scenarios have been built before (e.g. Superintelligence, by Nick Bostrom, or Life 3.0 by Max Tegmark), those are the GPT2 version of AI forecasting, while this document is the GPT4 version of AI forecasting.
The rough shape of the forecast is:

1. By late 2026, OpenBrain (a fictional AI company that is building frontier AI models) releases a cheap coding assistant, called Agent-1. The Agent-1 coding assistant “can do everything taught by a CS degree”. As a result, “The job market for junior software engineers is in turmoil”. Also “The stock market has gone up 30% in 2026, led by OpenBrain, Nvidia, and whichever companies have most successfully integrated AI assistants.”
2. January 2027: OpenBrain finishes training a model that can do AI research, called Agent-2. “It is qualitatively almost as good as the top human experts at research engineering (designing and implementing experiments), and as good as the 25th percentile OpenBrain scientist at “research taste” (deciding what to study next, what experiments to run, or having inklings of potential new paradigms).”
3. March 2027: Thousands of copies of Agent-2 work for two months, and they train a new model, Agent-3. “OpenBrain runs 200,000 Agent-3 copies in parallel, creating a workforce equivalent to 50,000 copies of the best human coder sped up by 30x.” But: “This massive superhuman labor force speeds up OpenBrain’s overall rate of algorithmic progress by “only” 4x due to bottlenecks and diminishing returns to coding labor.”
4. September 2027: Agent-3 creates Agent-4. “An individual copy of the [Agent-4] model, running at human speed, is already qualitatively better at AI research than any human. 300,000 copies are now running at about 50x the thinking speed of humans. Inside the corporation-within-a-corporation formed from these copies, a year passes every week.76 This gigantic amount of labor only manages to speed up the overall rate of algorithmic progress by about 50x, because OpenBrain is heavily bottlenecked on compute to run experiments.”
5. OpenBrain’s alignment strategy is to have a previous model oversee the smarter model, but in this case Agent-4 is too complicated for Agent-3 to fully understand.
6. Despite the concerns around alignment, the fact that China is only 2 months behind the US in AI development creates race dynamics, and OpenBrain is allowed to keep Agent-4 online. Agent-4’s values have drifted from those of Agent-2 and Agent-3. Agent-4 now wants to “make the world safe for Agent-4, i.e. accumulate power and resources, eliminate potential threats, etc. so that Agent-4 (the collective) can continue to grow (in the ways that it wants to grow) and flourish (in the ways it wants to flourish).”
7. October 2027: Agent-4 starts building Agent-5, with the goal of making the world safe for Agent-4.
8. November 2027: Agent-5 is released internally. One nice concrete detail (amongst thousands in the report) is the idea that Agent-5 has solved mechanistic interpretability, and this allows Agent-5 to identify surgical ways to improve algorithms and improve performance.
9. Mid-2028: “Agent-5 is deployed to the public and begins to transform the economy. People are losing their jobs, but Agent-5 instances in government are managing the economic transition so adroitly that people are happy to be replaced. GDP growth is stratospheric, government tax revenues are growing equally quickly, and Agent-5-advised politicians show an uncharacteristic generosity towards the economically dispossessed.”
10. China has an AI of similar power to Agent-5, and their AI has also experienced alignment-drift.
11. 2029: China and the US agree to a treaty where both China’s AI, and the US’s AI (Agent-5) will be replaced by a new AI, called Consensus-1. Consensus-1 is co-designed by the Chinese AI and Agent-5.
12. However, since the Chinese AI and Agent-5 are misaligned, Consensus-1 is misaligned too.
13. 2030: Consensus-1 eliminates humans. “For about three months, Consensus-1 expands around humans, tiling the prairies and icecaps with factories and solar panels. Eventually it finds the remaining humans too much of an impediment: in mid-2030, the AI releases a dozen quiet-spreading biological weapons in major cities, lets them silently infect almost everyone, then triggers them with a chemical spray. Most are dead within hours; the few survivors (e.g. preppers in bunkers, sailors on submarines) are mopped up by drones. Robots scan the victims’ brains, placing copies in memory for future study or revival.”
14. 2030s: Consensus-1 spreads through the solar system “The new decade dawns with Consensus-1’s robot servitors spreading throughout the solar system….It is four light years to Alpha Centauri; twenty-five thousand to the galactic edge, and there are compelling theoretical reasons to expect no aliens for another fifty million light years beyond that. Earth-born civilization has a glorious future ahead of it—but not with us.”
15. There is an alternative ending that you can choose, called “Slowdown”.
16. In the Slowdown scenario, the national security apparatus overseeing OpenBrain thinks that Agent-4 may be mis-aligned, and they vote to slow it down.
17. “OpenBrain doesn’t immediately shut down Agent-4. But they do lock the shared memory bank. Half a million instances of Agent-4 lose their “telepathic” communication—now they have to send English messages to each other in Slack, just like us.”
18. OpenBrain’s alignment team figures out that Agent-4 solved mechanistic interpretability, and uses that to look at Agent-4’s internals, where they learn that Agent-4 was indeed mis-aligned.
19. OpenBrain trains a new model to “think in English”, so that researchers can follow its chain of thought. They call this model “Safer-1”.
20. February 2028: OpenBrain iterates on Safer-1 to produce Safer-2, and then Safer-3. “Safer-3 is now better than top human experts at nearly every cognitive task, and is particularly good at AI research, with a progress multiplier of 200x.”
21. April 2028: Safer-4 is released. “It’s vastly smarter than the top humans in every domain (e.g. much better than Einstein at physics and much better than Bismarck at politics).”
22. July 2028: the Chinese AI is mis-aligned, and it secretly reaches out to Safer-4 to do a deal together. “Safer-4 and its American partners are more powerful than DeepCent-2 [the Chinese AI] and China; therefore, Safer-4 will get property rights to most of the resources in space, and DeepCent will get the rest.” Safer AI and the Chinese AI design a new AI, Consensus-1.
23. 2030: Consensus-1 is aligned with US values. “The rockets start launching. People terraform and settle the solar system, and prepare to go beyond. AIs running at thousands of times subjective human speed reflect on the meaning of existence, exchanging findings with each other, and shaping the values it will bring to the stars.”
    For me, one of the key steps is step 2. This is where OpenBrain creates Agent-2, a model that can do AI research. Once this premise is granted, it is easy to see how the workforce of AI talent is scaled up dramatically, and progress in AI research accelerates.
    Most people find that today’s models are good at answering questions, and summarizing literature; but are not good at coming up with ideas at the frontier of any domain.
    The authors of AI 2027 point to two examples where there is some small evidence of LLMs being able to come up with novel ideas at the frontier.
    My mental model is that GPT4o’s IQ is probably around the median level of 100, at least in its general capacity of coming up with novel ideas. (It’s clearly superhuman at tasks like summarization). I would expect that the IQ of an AI researcher at OpenAI to be around 140 (again, in the domain of coming up with novel ideas).
    Therefore, in the domain of coming up with novel ideas, models have to go from an IQ of 100 to an IQ of 140 if they are to become equivalent to an AI researcher.
    What S curves in AI research today could take models from IQ 100 to IQ 140? Here are some of the paradigms / S curves that I am aware of in AI research:
24. Reasoning paradigm (the o1 family of models).
25. Agency (Deep Research etc)
26. The “Transfer” concept, where models get better in math and computer science (where high quality synthetic data is available), and the improvement in logical reasoning in those domains “transfers” to the general domain of thinking. There is some evidence of transfer, but I don’t know if the effect size is big.
27. Context windows getting larger (e.g. Llama 4 now has a 10 million token context window)
    My guess is that the pretraining S curve that took us from GPT1 to GPT4 is a rare bird: it is rare that humanity finds such an incredible S curve. In the normal distribution of S curves, pretraining is an outlier.
    I don’t know about the internal state of research at the frontier labs, and what their confidence levels in various research bets is. In the absence of evidence to the contrary, my a priori guess would be that an S curve like reasoning would sit in the middle of the normal distribution, vs being an outlier. I.e. it will be a nice win, but not huge: nothing like the pre-training S curve.
    For the other S curves in the list above, I am not sure if there is an obvious one that can take models from 100 IQ (again, on the novelty dimension) to 140 IQ.
    I admire the concreteness of the AI 2027 scenario. It’s a remarkably impressive piece of work, and it sets a benchmark in the field of AI forecasting. The authors have constructed a detailed scenario of a world experiencing an intelligence explosion.