Søren AasA Roadmap To Nowhere
What AI 2040's Plan A Leaves Out
The AI Futures Project just released their next big scenario, it's called AI 2040: Plan A. The thinking behind AI 2040 is more concrete than most and is a positive vision that comes with predictions. Plan A's vision if for the US and China to jointly manage AI, developing it transparently and cooperatively instead of in competition. The predictions it makes are of a future where humans manage to harness AI to do most of our labor while people live off a universal income and become full time vacationers. The predictions are the tires we will be kicking. Common to all is that they treat intelligence as the one variable that matters, the single scarce input, and everything else (physical and political) as infinitely elastic. They are not and that's why the predictions fail.
Machines will do all our jobs for us. Mechanization was promised to serve us, that it would make manual labor irrelevant, because mechanical labor was more efficient. And it was, or still is, but it's been a good 100 years since industrialization and where are the 2 day workweeks? We even doubled the workforce when the ladies joined, and we are still working what Henry Ford determined was the optimal work week. Each week. Why are we still working?
It has to do with legibility. When something became solved we didn't rest, we found a higher level demand and started to feed it, often delegating responsibility to someone further down the hierarchy. Having machines sew our own clothes didn't mean we went idle. Somehow there was always something more we could do that somehow needed doing, or at least could be done. It's like we humans like to do stuff, and once something hard becomes easy we find something else that engages us.
As every IT student who has tried to automate something has learned the hard way, you get 80% working no sweat, it's the last 20% that take 80% of the time to get right. Senior developers don't sweat the edge cases, they make something that is good enough. Because computers need to make something legible before they can handle it, and often legibility is found in the connective tissue between the actually legible processes. So AI will automate a lot, but at all perimeters it meets there will be a legibility gap that it will need help to cross. An analogy is the state as a legible process; its power is absolute where things are defined and labeled, but out in the back country hinterlands its sway takes a nose dive. No one can build a house in a city without permission, but anyone can throw up a rudimentary dwelling in the woods. The state would love to tax the hut, but it can't because it isn't registered in the system, it can't see it. Why AI would differ, the essay never asks.
When clothes became cheap and automated enough the workers didn't frolic in the fields, they started value adding in other ways. At present there are armies of people fighting everyday to make consumers choose their employer's brand. There are enough tshirts for everyone, but we are chasing the right logos. We are not content with a professionally stitched and quality controlled fabric, it has to have something more. It's like we can't refuse one-upmanship, so we all try to one up each other. Why AI automation would differ meaningfully from this historical fact is not something the essay digs into, it's just an assumption that it's based on.
In the remaining western democracies (we miss you America😢) we have got better selection at the rural supermarket than the Kings of a 100 years ago had. Strawberries in February, sure. It's like every activity we do we get better at it by making it more legible and once it is legible enough we can create organization and procedural structures around it so that we can go on to doing something more important with our creativity.
The load bearing assumption that we humans at some point decide to cooperate and pump the brakes on AI rollout? I know the climate scientists are cognizant of just how hopeless this sounds. Yet, it does so happen that we have a particularly potent example of global cooperation that did work. It was two superpowers agreeing that the planetary destruction devices they were stockpiling weren't really needed. In fact it would be really great if we just kept the numbers really low. Enter the Non-Proliferation Treaty of Nuclear Weapons (NPT) agreement. It worked. Its PR campaign wrote itself, photos of Hiroshima and Nagasaki, a cloud and children practicing to hide underneath their school desks. It was very viral. The mushroom cloud gets poor reviews from everyone. But on top of that nukes were a significant budget expenditure, both to build and maintain. The state treasuries were relieved when they could finally start to redirect cash from nuclear warheads to stuff the voters really wanted.
The happy smiling cooperation that happened for nukes won't for AI because they are two very different things. Specifically AI is super useful, yeah we now roll our eyes at em-dashes, but who doesn't use it. And it is making a lot of people rich. Plus no photos of victims of radiation exposure. The NPT would never have happened during the Manhattan project, it was only after we had tested them (confirmed kills) that we were happy to sheath them. Expecting those who have their fingers in the AI pie to just shrug and give it away is not a bet with even bad odds. It does seem that the meticulous care of pointing out the obstacles to the agreement working unearthed the China stealth research carrying on scenario, but the foundation seems given less thought.
Further making a future prediction that does not take resource constraints into account in 2026 is pretending we are not overreaching on the planet, no one is going to argue that. Specifically AI 2040 estimates massive increase in data centers. Powering a data center is, and has been for a long time, an energy intensive endeavor. As the cheap fossil fuel age is transitioning into an age where energy costs are increasing, are we really going to use all available energy on AI? Concluding that sufficient energy will be available for running AI into "explosive growth" as our food production systems and transport energy requirements are also vying for a slice of the shrinking pie. Casting our eyes on the prediction of the 1972 Limits To Growth, its projections have been found to match up well with reality by later studies. In their prediction, around 2040 is exactly when the capitalist infinite growth roller coaster ride is at its peak. It's downhill and screams after that. And this is exactly when AI 2040 is predicting exponential growth curves. The fastest industrial expansion in history is going to happen as hard limits set in. Energy isn't a commodity you can just buy at these scales. By then it's political, and there will be other mouths to feed.
It is suggested that instead of working we will be on permanent holiday. But in a world run by efficiency and bottom lines, a large swathe of the population that is not contributing any value could just as well be seen as an expense. And what happens when, for the first time in history, the kings have drastically reduced need for serfs? Swathes of populations become a burden instead of a resource.
The AI 2040 forecasting model assumes infinite resource elasticity, frictionless geopolitics, and perfectly aligned economic incentives. Like much of the AI debate it is less about thermodynamics, positional dynamics, historical labor patterns, and realpolitik than a roadmap reliant on best-case political and physical assumptions, and these conflict with historical precedent.
On the whole the scenario treats intelligence as the only scarce input, whilst everything else is elastic. It's like they picked the ingredients to cook one dish — a happy path to an AI future. What does it say about a conclusion that only survives if everything goes right?
🌊Follow the current on Medium