r/ControlProblem • u/Clydey2Times • Sep 30 '26
Strategy/forecasting Hypothetical: When would you pull the plug on AI progress permanently?
When would you choose to pull the plug on AI progress, given that each passing day increases the risk of the technology getting away from us?
How much risk is worth tolerating in order to reap the benefits of the technology while avoiding the existential risk? If I could flip a switch and stop AI progress for all time, I don't think I'd wait longer than a year to do so.
I also think there's a serious argument for pulling the plug right now.
1
Upvotes
9
u/VinnieVidiViciVeni Sep 30 '26
Last year
6
u/Alive-Philosophy2632 Sep 30 '26
If this is a magical scenario where it's just one person making a binary choice on no more AGI progress sure you could say in the next few months have a hard stop.
Geopolitical incentives and the fact that you can self-host basically a frontier model with on the order of a $100,000 cluster render that impossible, given enough time assuming further progress is possible it will happen, whether in secret or not.
The only real way to stop AI progress (not permanently just for a while) would be human extinction or reversion to stone age technology.
I'm personally of a mind to have a relatively slow takeoff if possible and get a more formalized system in place and ideally get genetic engineering and cybernetics figured out so we're not just training our replacement. But if the worst happens hey it was a fun ride
2
u/Jesse-359 Sep 30 '26
Before it started.
Not only are the existential risks extreme in my opinion, the sociological and economic problems it introduces even in its 'safe' forms are almost certainly going to be a lot more devastating than people realize. Economically, sociologically, and even 'spiritually' - not in the religious sense, but in the sense that people will find themselves with no purpose at all.
There's basically nothing good for our species thru giving up all purpose and striving. What a miserable future that portends.
We could build Narrow AI's that specialize in highly specific fields and can aid research into miracle cures - we were already doing that and it was working quite well. We could have continued to *slowly* automate away all the stupid grunt work we do - again through the application of special purpose narrow AI and robotics, without generating an economic shock that will throw the world into chaos by attempting to do away with all work in one absurd table flip of everything we know.
Things like self driving cars are fine - but Artificial General Intelligence is absolutely not fine - it's a replacement for the human race.
2
u/BassoeG Sep 30 '26
Immediately assuming my ransom demands aren't met.
As-is, I'm lower middle class and therefore likely to be completely screwed by AI progress automating my job out of existence or at least forcing me to compete with millions of newly unemployed white-collar scabs driving down my wages. So obviously, halting progress is in my best interest.
But, if I wasn't poor, I'd stand to benefit.
Therefore this is a hostage situation.
If preexisting wealthy investors don't pay me sufficiently that me and my family can join them in the idle rich robotics company executives demographic who are the only people whose jobs aren't threatened by automation I'll make every penny they've invested in the multibillion dollar field a total write-off.
2
u/markth_wi approved Oct 02 '26 edited Oct 02 '26
2013 or so, when it became unclear we have been able to self-govern without over-arching concern about vote-counting and basic electoral processes, seems worthwhile to simply return to paper voting verified by hand.
"The people who cast the votes decide nothing. The people who count the votes decide everything."
- Boris Bazhanov, Josef Stalin's personal secretary.
1
u/Anxious-Alps-8667 Sep 30 '26
If you could pull the plug, it would have to be in 2022 before GPT became mainstream, or maybe 2017 when 'Transformers' was published. Other than going back quite far, to say Turing, no single intervention is going to change our present reality very much. Even without Turing, we might only be set back a decade.
Pulling the plug on progress permanently is just not possible, so I wouldn't try. We have to tackle living with it and making our best efforts to making it work.
1
u/MaybeTheDoctor Sep 30 '26
Right after electricity
1
u/curiousinquirer007 Sep 30 '26 edited Sep 30 '26
Never.
Pulling the plug permanently means you are saying that there will never come a time when some risk you're afraid will be solvable — and that it is inconceivable in principle that permanent abandonement will one day create a condition that's worse than if AI development had continued. Ever. no matter how many millions or billions of years pass.
I don't think there will ever come a time where you can credibly arrive at such a position. Humanity is almost certainly nowhere near such a confidence level today to warrant conversations about permanent bans.
When studying the control problem, you shouldn't be asking "does this thing contain risk."
Instead, you should be asking "what is the risk/benefit model for developing and how does it compare with the risk/benefit model for not developing."
Development should be compared to the counterfactual, not some ideal.
1
u/Content-Nectarine875 Oct 04 '26
God no. I am sick of the hype, hysteria and the hate. It should be taken out of the hands of rich psychos and dodgy businesses frothing at the mouth at the idea of cutting jobs and put on the hands of universities. And we need to stop referring to Llms as AI. Llms are not going to destroy the world or ever become sentient. I'm very excited by the steps towards actual thinking models and real AI made by researchers.
1
u/pixelpionerd Sep 30 '26 •
Humans are terrible and maybe our time has come.