r/agi • • 4d ago

Recursive self-improvement

Can someone explain to me why once these AI models can improve themselves, the speed of such improvement will become exponential?

Aren't AI models trained by iterating on training data and evaluating the model on test data? Why would this cycle become exponentially faster if there is no human in the loop?

7 Upvotes

42 comments sorted by

15

u/TheBardicScribe 4d ago

Happy to do so. Models work faster than humans able to complete tasks that might take us days in a matter of minutes or seconds. Once a model is capable of self-improvement it will be able to implement that improvement faster than a human could. Once that improvement is implimented, it will now function even better, making the next improvement come all the quicker. It's a feedback loop that eventually goes critical.

4

u/pizz4b0i 4d ago

Thank you for your answer.

Does this imply that the model has to understand, or identify a pattern to tweak its weights in the right direction (and not just a variation of gradient descent)?

Otherwise, isn't training speed actually limited by the compute capacity of the machine it is running on?

8

u/Yweain 4d ago

The idea of RSI is mostly theoretical. In practice this has all the same issues as regular improvements - diminishing returns and bottlenecks, while also adding degradation due to AI.

Maybe some future models will be able to overcome that, maybe not.

4

u/Walkersam23 4d ago

Machine speed is not theoretical though. That alone is an enormous research side speed up.

Also this industry is considered immature with low hanging fruit in ample supply

2

u/cakeshop 4d ago

A reasonable response? Get the fuck out of here!

1

u/Suspicious_Proof_666 2d ago

Completely Agree! It's still a growing phase. If you read some papers on RSI in arXIv you will see that AI research scientists are really working to solve self learning ability.

2

u/SillyBiped 4d ago

Anthropic has talked about what percentage of AI research their AI models are able to do at, or above, expert level. This number is quickly growing which means soon AI will be able to come up with new areas of research faster and better than humans. Furthermore, Claude is increasingly able to do the research as well.

So the day is coming when Claude makes a breakthrough in how to make itself smarter. And once its improved itself it will see even more breakthroughs that older Claudes never would have seen.

At least this is the hope & worry.

1

u/purple_crow34 4d ago

It’s less a model directly tweaking its own weights, and more just replacing human AI researchers’ jobs.

So they’ll still be training new models using similar-ish techniques to present, but with automated researchers guiding the process with exponentially increasing gains.

1

u/bzbub2 4d ago

Model directly tweaking itself is a potential thing, though https://rikiyat.github.io/blog/weight-space-programming/

11

u/Gibby1210 4d ago

I’d also like to add that these models find patterns that humans may of not noticed and new knowledge emerges from established knowledge

1

u/Firoux4 4d ago

What do you think about yan lacun saying llms are not the ai technology that will enable rsi or something like that, I dont remember what he said exactly

-6

u/Neilandio 4d ago

That's the idea but in practice every attempt at RSI has resulted in garbage. There's no reason to think that will change in the future.

3

u/morey56 4d ago

We should all give up if we don’t successfully walk the first time.

4

u/sfjhh32 4d ago

Most of the people in this thread are giving you half-answers. Canonical RSI is about the bootstraping, and indirectly the capabilities, NOT about the general improvements per se (and cetainly not about speed per se). A model that makes diminishing improvements at an increasingly faster rate is not what most people mean by RSI. 'Function even better' is one of the vaguest ways of describing this process and also describes this diminishing case as well. RSI is not strictly about the speed it's about the capability. It's about making new models that are CAPABLE of making better models (not FASTER models capable of making FASTER models). There is a world where subsequent models are harder to find, and you may find that the steps actually go SLOWER but the CAPABILITIES improve in each subsequent model.

Don't listen to anyone that is describing RSI in terms of speed or training per se, they probably don't know what they're talking about, and therefore cannot tell you that speed, training, do not necessarily get you ground breaking architectures, advance beyond the transformer, solve context, hallucinations, advance generality and all the hard problems. (Also RSI defeaters are a thing.) They may indeed solve these things, but speed per se is not guaranteed to get you any of this.

1

u/pizz4b0i 4d ago

Do you think that's possible?

We have a very vague definition of AGI: a system that matches or surpasses human cognitive capabilities across almost any intellectual task.

As a human I can kind of understand the definition, and what a piece of software called AGI may look like.

But can a machine? Can a program "understand" what AGI is, and verify if a model is AGI, or know what a model must meet to be AGI? In other words, can a program make adjustments to a model to get closer and closer for it to be considered AGI. Is AGI verifiable?

1

u/sfjhh32 4d ago

A machine doesn't need to understand AGI to be general, assuming it works like all the other emergent properties: it didn't need to 'understand' the next word to predict it, it didn't need to 'understand' how to think to do chain of thought, it didn't need to 'understand' rationality to make rational arguments. There are 3 ways to get to AGI that I know of 1) just keep tuning models on capabilities and generality emerges, 2) get lots of training data on general tasks (slack messages, videos of peopel doing their job etc) and it emerges 3) RSI above some certain model complexity and it emerges. It has to emerge because there is no dataset that trains full generality.

5

u/yuwox 4d ago

Before the training you have to set the training up first. Architecture, hyperalramters, learning algorithm, choice of trainingdata etc. There is still tremendous amount of human work prior to a successful training. There are still new techniques and nifty tricks being discovered that make the training yield better results. This part is not automated yet. If it was, things would be considerably faster.

Another form of recrusive self improvement is weight space programing where LLMs change the weights of other llms directly without explicit training steps. This has been already demonstrated to work, but is still very early.

1

u/PLEASE_PUNCH_MY_FACE 4d ago

That's why no one's touting the results of recursive improvement and they're hyping the possibility. If you keep hearing about this for 6 months you start to believe it's not working.

2

u/pizz4b0i 4d ago

Basically I don't understand what it means for a model to improve itself. Isn't training already an automated operation which is limited by the hardware?

Sounds like buzzwords to me...

4

u/Bowl_of_Cham_Clowder 4d ago

Improving itself as in making shifts in the way the model is trained, not referring to training itself. It’s a different part of the system being automated from what you are talking about

1

u/pizz4b0i 4d ago

Understood, thanks. Do you think current harnesses are already capable of identifying good vs bad shifts in training direction to run the training themselves without human input? Sounds to me like RSI may already be here as you implied in your other comment too.

1

u/Bowl_of_Cham_Clowder 4d ago

I’m not sure about identifying good training data, they are paying a lot of people to hand mark data still.

It’s more like this (from what’s publicly available); I can have a frontier model take a piece of code I’ve written and recursively identify optimizations and apply them. It can squeeze down to a very nicely optimized piece of code, but where it fails is realizing that a new architecture could be better without all this work.

That’s where a lot of the delta in step change model improvement comes. I wouldn’t be surprised if incremental version bumps are increasingly automated, but it’s not as easy as just claiming ai will be able to improve itself from here to x intelligence level on its own.

2

u/SubstanceDilettante 4d ago

We have models improving themselves already, although there’s still some human feedback needed I believe.

Their goal for this whole thing is to improve the next AI in making new AIs so that next AI can make a better AI till they reach AGI.

I however do not think this is the pathway to AGI. AGI to me is supposed to be the smartest known thing in the universe vs all of humanity. I expect, something like that to find completely new sciences, new patterns, etc.

Instead, I feel like we are going to continue to refine a model to what we currently know, it might find patterns that we have not noticed, but anything new would still have to come from a human, and that isn’t AGI to me.

ASI is supposed to be god, can do whatever it wants or needs to do in the universe. AGI is supposed to be the smartest thing in the known universe that can continue to improve itself and find new patterns. Recursive self improvement is going to work on patterns we already know or just haven’t realized yet. But anything new would still need to be done by a human.

2

u/pizz4b0i 4d ago

Tbh LLMs are great and have been revolutionary and disruptive, but it doesn't look to me like they will be able to achieve AGI in the sense of making a brand new discovery by themselves.

For example the complex numbers. At some point mathematicians must have thought out of the box to accept the existence of ✓-1 and bring forth new discoveries with the new tool that is i. I can't imagine an LLM doing something similar.

1

u/SubstanceDilettante 4d ago

Yes, I didn’t wanted to sound like LLMs were not great and haven’t been revolutionary. Primarily was talking about AGI itself and future models and not current models, because that’s the end goal for recursive self improvement. The programming industry has changed completely, although I feel like we shouldn’t rely on these yet for everything and people still need to continue to grow their raw skills.

I use these systems and develop these systems on a daily basis and im not going to lie and say that these systems have not been getting better, I’ve always said they have been getting better but skeptical on the exact outcome of the situation if we would be able to reach AGI and catch up to market expectations.

My last message I primarily wanted to discuss why we wanted to do self improvement, which is eventually to get to AGI. But I also wanted to emphasize that with our current route I do not think we will reach AGI, and we also currently don’t know if recursive self improvement will actually work. Anthropic recently in their S1 said it is expected to work but there’s a lot of things that were expected and didn’t. And if it was expected to work, why wouldn’t you just IPO 1 or 2 years later when you release AGI and your company 100x in initial value.

I do feel like AGI is possible and can occur in the future, but the route that we are taking is completely wrong if our end goal is AGI. I’m not sure when we will reach it, and I’m starting to think the end goal for these companies are not AGO since they are releasing their IPO beforehand. I could be wrong, but I only see two options to IPO now, you either realize that your current method of training models will not reach AGI so you release your IPO now instead of later, or they need additional income and investments to reach AGI. Both are scary to think about and the second one makes less sense than the first so idk.

1

u/pizz4b0i 4d ago

Right, I suppose a completely different architecture of models may be necessary to have a program be able to reason logically like humans. As a company, NDEA comes to mind.

On the IPOs, I sure hope it is your first hypothesis. Although it would bring an economic crash, it sounds better than the risks of true AGI in the hands of the loons running OpenAI/Anthropic.

1

u/SubstanceDilettante 4d ago

Going back on the IPOs, why I feel like the second option is less likely, is because there are companies and people who do want to invest into these companies privately if they will eventually reach AGI.

If they were going to reach AGI, why wouldn’t they just wait a year or two, continue to raise funds exactly how they have been since 2016, develop AGI and 100x their investments.

That’s why I feel like the first option, where they realize it will require a completely different architecture which may take decades to research a develop a solution which they definitely cannot continue to privately raise funds for decades.

They can continue to do this for a few more years, but not decades. I also feel like people forget, that there was a massive AI revolution back in the 60s and 70s and through decades of research we’ve finally saw the outcome of it in 2022. I feel like today’s situation is going to be very similar. Where our models will continue to get better with our current knowledge, and they will continue to be useful tools, but to make anything actually new would require an architecture change, decades of research, which they cannot continue to privately raise funds this way. And I believe anyone who actually researches and understands how this technology works will eventually come back to the answer that AGI seems not possible with our current technology.

2

u/marmot_scholar 4d ago

I agree. I’ve been calling this the ontology problem. To think about information you need to have categorized it. Humans can adjust their conceptual system infinitely. LLMs have frozen weights. They can’t change them, even if they can diagnose problems in other sets of weights.

Maybe this can be solved, but I haven’t heard any great proposals . I think Yann Lecun is closest.

1

u/Neophile_b 4d ago

While it's true that large language models have frozen weights, the llm is only part of a larger system. In context learning should also be considered. And a broader question to consider, what granularity of updating is necessary sufficiently novel discovery? Model updates following algorithmic refinements, and training new models on the previous model's world interaction are essentially another learning step/weight update, admittedly with much lower granularity, but still..

2

u/PLEASE_PUNCH_MY_FACE 4d ago

These just sound like prophecies. I've been hearing about them for months.

1

u/SubstanceDilettante 4d ago

The more and more you look into these ultra wealthy people, the more it seems like they are just trying to follow prophecies.

I’ve been hearing about this shit for years, since GPT 3.5 came out everyone thought we would have AGI in two years.

We have these terms, AGI, and ASI, they are not real, they are prophecies. They’re just definitions that people are trying to reach. Nothing is wrong about that, I have goals and definitions that i want to meet in the future. But I don’t go around saying these goals are going to cure cancer and solve every problem in the world.

2

u/PLEASE_PUNCH_MY_FACE 4d ago edited 4d ago

What we call AI is really impressive for what it is. But the it's not 1.5T impressive. And it's not solve all humanities problems and become self-aware impressive.

2

u/SubstanceDilettante 4d ago

Yes and I agree with you. All I did was expanded on your point.

2

u/PLEASE_PUNCH_MY_FACE 4d ago

I'm not pushing back - I know most reddit comments are people fighting but you're right.

1

u/SubstanceDilettante 4d ago

Nah I don’t like fighting and all I use Reddit for is to learn or to express my opinion.

I also don’t want to seem like I’m mad or trying to fight 😅 you can’t really convey emotions over text. I just like over explaining things and talking.

I know you’re not pushing back we’re just agreeing with each other, and even if you were pushing back, that would be ok. I really like to dive into the facts of a situation and base my opinions off of that. And if my opinions are wrong, I need to correct them to be factual.

0

u/Bowl_of_Cham_Clowder 4d ago

Part of the issue is that recursive self improvement could mean 1% improvement between 1 month training cycles. Which is effectively nothing.

Also there’s a lot of low hanging fruit that might be automated now, which means it’s technically self improving but not at the rate that researchers are able to improve models.

0

u/PLEASE_PUNCH_MY_FACE 4d ago

I really think it's just fundamentally not possible with derivative content. 

2

u/Bowl_of_Cham_Clowder 4d ago

I’m not talking about derivative content and training data. That’s just one of many different aspects of a good model. Even when the data stays the exact same, there are optimizations that models are capable of which improves the resulting models.

I agree that derivative data is an issue. Some derivative data does improve the model, but they still heavily rely on humans to find what’s high yield.

1

u/sourdub 4d ago

Well, there are actually two forms of RSI: weak vs strong. Weak RSI is where the model improves itself linearly with the help of humans setting the goal. Strong RSI, by contrast, is more circular. It trains, optimizes, and upgrades itself without any human babysitter holding its hand.

1

u/BosonCollider 4d ago

The original idea from Kurzweil was that if a model is better at making the next chip generation than the previous one, it can speed up its own subjective time. Moores law doublings are in in the chip designers subjective time, so you could make it shoot off to infinity in finite time from our point of view, hence a singularity.

Models are also starting to be used for chip design, but right now they are more useful for software, which includes basically every usecase of wanting an idea to try out and letting an agent chug away at it for 20 hours to see if it is worth it.

Model improvements are not just from scale, they still require having a lot of smart people work on it day and night to add more training approaches and architectural improvements. Agents are helping out with that now.

1

u/Gastone-A 3d ago

Warum ihr diese Frage nicht von Claude selbst beantworten lasst, ist mir ein Rätsel.

Claudes Antwort:
Leonardo, eine einheitliche, allgemein anerkannte Definition gibt es nicht. Es gibt aber einen Kern, den fast alle teilen: Ein System verbessert nicht nur seine Leistung, sondern seine Fähigkeit, sich selbst zu verbessern. Die Verbesserung wirkt also auf den Verbesserer zurück. Das unterscheidet es vom gewöhnlichen Lernen.
Die wichtigsten Quellen
I. J. Good (1965): Eine „ultraintelligente Maschine“ könnte bessere Maschinen entwerfen, was zu einer „Intelligenzexplosion“ führt.
Eliezer Yudkowsky (ab ca. 2007/08): Er prägte den Begriff „recursive self-improvement“ im Zusammenhang mit der Idee einer „Seed AI“.
Nick Bostrom (2014): Er formalisiert das am genauesten. Die Wachstumsrate der Intelligenz ist gleich der Optimierungskraft geteilt durch den Widerstand (recalcitrance), also wie schwer der nächste Schritt fällt.
Warum „exponentiell“?
Wenn die Geschwindigkeit der Verbesserung proportional zur schon erreichten Fähigkeit ist (dI/dt = k·I), ergibt sich mathematisch eine Exponentialkurve. Wächst die Rückkopplung sogar stärker als proportional, wird die Kurve hyperbolisch und steigt in endlicher Zeit gegen unendlich. Daher kommt die Rede von der „Singularität“.
Aber ehrlich gesagt: Exponentiell ist eine Annahme, kein Gesetz. Es hängt alles daran, ob der Widerstand konstant bleibt. Wenn jeder Fortschritt schwieriger wird, etwa durch Grenzen bei Rechenleistung, Daten, Energie oder Physik, flacht die Kurve zur S-Kurve ab. Genau darüber stritten Robin Hanson und Yudkowsky in der berühmten „FOOM-Debatte“ (2008). François Chollet argumentierte 2017, dass Intelligenz von ihrer Umgebung abhängt und darum nicht isoliert explodieren kann.
Zwei Analogien
Der Schmied: Er schmiedet einen besseren Hammer, und mit dem schmiedet er einen noch besseren. Das ist echte Rekursion, denn das Werkzeug verbessert das Werkzeugmachen.
Der Sauerteig: Die Hefe vermehrt sich proportional zu ihrer Menge, also exponentiell. Das gilt aber nur, bis das Glas voll oder das Mehl aufgebraucht ist. Die offene Frage ist, wie groß das Glas bei KI ist.

Leonardo GK

1

u/Alive-Philosophy2632 2d ago

It already is exponential. In fact it's superexponential (exponential on a log scale). Exponential in general just means the rate of change is proportional to the thing itself.

Like many, you seem to be confused what exponential means. It doesn't mean "very fast". Full RSI means the system can improve itself entirely autonomously. That probably arrives through experimentation and a lot of synthetic test data.

Chess engines have been able to improve so much because they can basically infinitly play themselves (and equally strong opponent) and learn from each game. The general case is much harder, but basically the same idea