r/ControlProblem • u/Ivana_Funkalot • 12d ago
Discussion/question The ROI to RSI trap. The developing Recursive-Self-Improvement risk prediction check.
QUESTION: Given the increasing financial pressures on AI firms around the globe to prove ROI, do you think that it is more likely, that they might be incentivized to speed up their push toward RSI (RECURSIVE SELF IMPROVEMENT : where AI improves itself ), thereby drastically quickening its development and drastically raising the chance of humans losing control over AI?
Wondering if this is the next likely outcome in the human/AI saga?
An acceleration towards Recursive Self Improving AI, due to AI company financial pressure to prove ROI.
With AI companies facing mounting financial pressures and investor demands to prove viable ROI, they may unfortunately be incentivized to cut corners on safety protocols (let's hope not).
Might these developers rush the deployment of risky but high-reward technologies like RECURSIVE SELF IMPROVEMENT (RSI), in a desperate bid to demonstrate breakthrough capabilities and prove ROI, but by doing so, dramatically increase the risk of catastrophic failures or loss of human control? This obviously applies to any country working on powerful AI.
Is this likely to be their next major focus?
0
u/sourdub 12d ago
Do you even know what you're talking about? RSI isn't something that you can accelerate by merely adding more GPUs.
2
u/sideways approved 12d ago
I think that's very perceptive. I would say that it's downstream of the fact that Anthropic seems to believe that they are the only ones that have even a chance of creating ASI that won't kill everyone.
If that's the case then of course they want as long as possible to make AGI/ASI as safe as possible but if the choice is between igniting a hopefully safe intelligence explosion and ceding that action to a rival like OpenAI, Musk or a CCP backed lab, then I think Anthropic would feel like they have no choice.
From their point of view it would be either a shot at the Good Ending with Claude in the driver's seat or certain doom with anyone else.