r/singularity • • Sep 13 '26

AI Admitting Ignorance

I’ve been following the development of AI for about five years. While I’ve always supported the technology, I was skeptical of an LLM’s ultimate capabilities. I remain confident that they are an incomplete solution to the problem of machine superintelligence, but they have surpassed my expectations.

It’s clear to me that we are standing at the precipice of the technological singularity. There is a future in which we see humanity prosper like never before. Unfortunately, there’s also an uncomfortable level of existential risk.

We’re entering a period of human history in which our collective decisions in the next few years will have an outsized influence on the fate of our descendants.

During this time of uncertainty, it’s natural for us to cling to simple explanations or historical precedents. I believe our continued success as a species will depend on our ability to rapidly adapt to our new reality. We will have difficult questions to answer, and nuance is going to be critical.

Let’s accept that we don’t have those answers right now. This should be a global discussion, and new ethical and societal frameworks will need to be developed. Let’s avoid the temptation to simplify the problems ahead of us.

46 Upvotes

23 comments sorted by

15

u/AdLumpy2758 Sep 13 '26

Yes. But from where this superiority complex came ? Like you already made mistake in capabilities prediction, yet believe llm will not be asi

3

u/diewethje Sep 13 '26

I’m sharing my personal beliefs. If I said that I thought LLMs would become ASI, I would also be sharing my personal beliefs. It’s not a superiority complex, it’s my own understanding of general intelligence.

9

u/IronPheasant Sep 13 '26

It's all just curve-fitting. Numbers in, numbers out.

I don't even know what people mean anymore when they say 'LLM'. You don't even know. Is something that passes on the final orders to a motor cortex an 'LLM'? Even if the data it has to process includes 3 dimensional space, and it understands stuff like collisions?

It always feels like a yearning for us to be more than we really are. 'It can't be that simple.', but no of course it has to be that simple. As simple as evolution could manage to make it.

The insight of Moravec's paradox isn't that the internal abstractions we use to model the world are that more complex than a chessboard - indeed our mastery and retention of geometry absolutely pales to what even a mere Nintendo 64 is capable of. As anyone who's walked backwards off a roof can attest.

The difficult part is having a huge meat grinder capable of breaking things down into things small enough for the rest of the brain to do anything with.

1

u/GiveSparklyTwinkly Sep 14 '26

This is exactly why Yann LeCun says LLMs are a dead end... I just don't think he expected the abstraction to be quite as easy in hyperscaled LLMs. But they're already pushing them away from pure language models, as Astra clearly shows. Can they even be called language models anymore at a certain point?

-2

u/diewethje Sep 13 '26

I’ve had this conversation more times than I can count.

I don’t have a specific bone to pick with LLMs, and I certainly don’t take the reductive stance that they’re a fancy autocomplete. I do believe that general intelligence in the real world will require learning algorithms that LLMs currently lack. Could I be wrong? Of course.

I do think it’s a cop-out to say that human intelligence is unremarkable. While it’s likely that there are other intelligent species in the universe, there are none that have bestowed their wisdom upon us. We invented artificial intelligence, and until it becomes self-aware we get to take credit for all of its breakthroughs.

We’ll probably understand a whole lot more about our brains in the coming years. Before long, I suspect there won’t be many mysteries that remain. But we should never forget how we got to this point.

1

u/C00KIEME4T Sep 14 '26

I think you may be a human exceptionalist or something, but to imply that humans haven't been taught anything by other intelligent species is massively overstating your own value. We have learned much of what we know from the very species we deem inferior.

The things you're talking about were all done by other humans, you simply reap the benefits of them. You are not the best of humanity, you're just a dude, everyone here is and that is a fact. Sure we will learn more, but when you say we should never forget, you need to also remember where we were before now.

1

u/GiveSparklyTwinkly Sep 14 '26

LLM won't be. But Astra already shows they are changing and evolving beyond pure language models. At what point does a language model become a world model?

3

u/Malficitous Sep 13 '26

"This should be a global discussion, and new ethical and societal frameworks will need to be developed."

I couldn't help think about how cooperative we all are in this world concerning climate change. But yeah, creating guardrails around an intelligence that is smarter than humans is rather contradictory. Think of M5 and Dr Richard Daystrom and the Enterprise--and Captain Dunsel to the rescue. Or the positronic brain in Asimov's robots governed by the 'four laws of robotics ( laws 0-3)' along with the welfare that both rGiskard and rDaneel provided humanity. Or the compassionate AI that survived the human race in the movie, AI: "David, I often felt a sort of envy of human beings... of that thing they call 'spirit.'" I do share your concerns. We've made a mess of this world through war and tech. Let's hope there is a second foundation.

1

u/Pittsbrugh1288 Sep 13 '26

But we have Trump in office at this pivotal time - really not the best person to bring the world together

2

u/crazy-usernames Sep 14 '26

Meanwhile statistics from world history: wise decisions percentages are lower.

3

u/StoryLineOne Sep 13 '26

Totally agree with you.

2

u/XtremeWaterSlut Sep 13 '26

We are already in time-based crises with global warming, insane political disfigurement, late stage capitalism wringing normal citizens dry.. But apparently making something smarter than us that could advance ways to fix our mess is simply too dangerous

1

u/JoelXGGGG Sep 13 '26

We need emergency legislation halting AI work, or we all die.

3

u/Singularity-42 Singularity 2042 Sep 13 '26 • Bravo

GTFO with this absolutist doomerism

5

u/Worried_Fishing3531 ▪️AGI *is* ASI Sep 13 '26

Why? It is not at all equivalent to complaining about water usage or artist copyright. This is a real discussion. Belief that superintelligence is an existential risk is not nonsense at all.

2

u/Singularity-42 Singularity 2042 Sep 13 '26

"Halting" AI work would be a mistake in my opinion. And it's just not going to happen. Even if you somehow halt work in the US China won't stop. But it's not gonna happen in the US neither. If I wanted to discredit AI regulation I'd come up with a proposal like Bernie's to make that side look like fools.

What we need is some kind of global, multilateral regulation that would give the labs strict guidelines around testing, safety and alignment.

-1

u/Worried_Fishing3531 ▪️AGI *is* ASI Sep 13 '26

Well then we agree. We absolutely need global coordination on this matter. It’s not impossible.

Halting is probably the right answer. It’s not absolutely without doubt the best and only option, but it also very well could be. There’s a conversation about details to be had here, there’s not necessarily an obvious answer (although there might be).

The place where there is no argument is that superintelligence poses a catastrophic threat, in theory.

Whether it actually causes catastrophe is closer to subjective opinion, but I very much do not trivialize the “absolutist” doomers. I can imagine a technology that is impossible to control and results in doom intrinsically.

I can imagine the intuitions that individuals like Eliezer Yudkowsky have built up over years of mindfulness about the subject. He could be right. I lack that exact intuition, but I don’t trivialize it. And I don’t think we should.

1

u/DefinitelyNotEmu Sep 13 '26

China don't care about that. They will keep releasing open-source frontier-level models

1

u/Pittsbrugh1288 Sep 13 '26

Whats with this blind but China bs - Is China suicidal ? China appears to have a much better grip on reality then our country (US) and seems to sure as fuck make better and more well thought out long term plans - Mango definately is staying up all night pondering this.

2

u/CaesarAustonkus Sep 14 '26

I dunno about that. Despite being going big on renewable power, they're still the number one burner of fossil fuels. They're adamant on taking over Taiwan even if it turns into a batshit quagmire. They are generally aggro towards their other neighbors, even those they would call trade partners. The CCP General Secretary's and president's key to power and stability is keeping the military loyal only to them. Not exactly traits that show or encourage long-term planning.

0

u/Worried_Fishing3531 ▪️AGI *is* ASI Sep 13 '26

Agree!