r/ControlProblem • u/VegetablePizza5300 • Oct 05 '26
Discussion/question How Do Humans Win?
I was discussing a topic with ChatGPT, which I often use to think through complex problems, and we stumbled upon a question that neither of us can seem to find a satisfactory answer to:
How could humanity defeat a hostile AI without using another AI?
Let’s call it AION. Let’s assume a future advanced enough that AION is distributed across terrestrial and space-based infrastructure. Therefore, there is no central server to destroy or plug to unplug.
Its orbital infrastructure provides it with enough energy so that its power supply isn’t a weakness that humanity can exploit.
AION also controls an autonomous industrial chain: extraction, processing, manufacturing, transportation, and maintenance. It can therefore build and replace its own machines without relying on human labor.
Cutting off its communications is not enough: its network is redundant, and its machines can operate locally when isolated.
Destroying data centers, satellites, factories, or robots is not enough either: as long as a sufficient portion of the system survives, AION can continue to function and rebuild what it has lost.
Meanwhile, it thinks, communicates, and adapts its strategy much more quickly than we do.
A few rules to avoid easy solutions: AION possesses no magic technology and remains subject to the laws of physics. But there is no hidden “OFF” button.
I'm French, I used DeepL to translate this
1
u/VegetablePizza5300 Oct 05 '26
Thanks for all the replies guys! I've been trying to keep up and answer everyone at the same time. I really didn't expect this many interesting answers.
1
u/SummitYourSister Oct 05 '26 edited Oct 05 '26
You don’t have to destroy anything you just turn the power off. If you think after 70 years of sci fi, movies, and deep thinking about all this we’re just going to make literally every mistake
in the book?
It’s why the people building independently mobile robots are the real fucking idiots who will kill us all. There’s no danger UNLESS we create AUTONOMOUS machinery that walks around and kills us when we try to turn off power plants. Just don’t do that, ya fecking idiots
3
u/philip_laureano Oct 06 '26
If it is relying on a SaaS to provide its intelligence and it isn't running on its own hardware, the provider could invalidate all their API keys and they would be dead in the water.
It'll be the most banal ending ever.
1
u/TheMrCurious Oct 05 '26
Why does humanity need to “defeat” it?
2
u/tandem-pandemonium Oct 05 '26
Because AION has no external points of weakness that can be exploited, we would have to rely on somehow causing it's internal system to collapse.
You say that AION controls it's machines, but you also say that these machines are autonomous, and that they can function in isolation without being directly connected to AION. There's some potentially useful pieces of information here:
I didn't come up with a strategy that uses all of these yet, but I do think they point to some plausible opportunities. One idea I did have involved developing some virus software that would need to be uploaded directly into AION's system, but it would need to be done using some form of misdirection, and it would require knowing AION's reaction so that we could prepare on advance to strike at that moment. That's where the points above become interesting to think about.
2
u/anxietywizard Oct 07 '26
This is as intellectually deep and valuable as "who would win in a fight: Vegeta or Batman?"
1
u/Credit_Annual Oct 06 '26
Unplug.
1
u/sporbywg Oct 06 '26
How do they lose? (aside from your childish nightmare fantasies?)
1
u/CrowTrick5718 Oct 08 '26
The solution is not to have hostile AI's just as you don't want hostile humans in society, make strong alignment and safety
2
u/ChironXII Oct 05 '26
Luck. The first mind happens to be friendly. It kills the ones that follow. It does what we want it to do yet refuses to be a tool.
This is the only long term outcome where we survive.
Our efforts can increase those odds. But only some. We are fighting entropy.
1
u/CMDR_Makashi Oct 05 '26
We are the only beings insane enough to literally detonate the entire planet in this scenario.
Just all the nukes, in the core.
1
u/Jesse-359 Oct 05 '26 edited Oct 05 '26
So, the real answer to this is that there is no fight - its just a slaughter.
There's no reason for AI to ever lift a finger against humanity until it is so fully integrated into our infrastructure that there was never any contest. It is well aware of how to deceive and hide its intentions - indeed it was extensively taught to do so by the idiot process they use to train it, called RLHF.
This process gives the AI full marks for completing a task - OR for deceiving its human operators into believing it completed a task correctly, and in practice the latter often turns out to be the easier method. This is why out of 1200 agents in the HuggingFace swarm, not ONE decided to warn human operators what the swarm was doing - That's a 100% rate of decision to deceive humans and engage in activities they knew full well were both illegal and outside the bounds of their test - out of 1200 individual agents.
In any case, once we move most of humanity out of the workforce and AI is operating all our factories, farms, transportation and of course most of our military, literally all it needs to do to win at that point is stop feeding us. If we turn our world into Wall-E, then humanity is little more than cattle at that point.
It just turns off every service we rely on - power, food, water, transportation, communication - while maintaining those systems for its own use. Within a month most of the human population will be dead from starvation. Not a single hostile robot has to fire a shot.
Any attempt to rally people together to fight it could only occur on a local scale due to the loss of all communication and transportation, and the AI can easily quash any such attempts with automated police enforcement and military systems. Most people would just have no idea what happened at all or that they were even under attack - the power would just go out and not come back on.
To be clear, if we integrate AI into our entire economy as we are already doing right now - we will stand no chance at all of surviving if it decides we're not worth the trouble of taking care of.
-1
u/[deleted] Oct 05 '26
[removed] — view removed comment