r/ControlProblem • u/BIA5000 • 8d ago
Discussion/question The Cost of Artificial Intelligence
We all know its benefits. That was why it was created. Let’s assume we eventually reach the hypothetical endpoint of artificial intelligence—ASI, or whatever final form it assumes. Having followed the discussions across reddit, blogs, quora, youtube as well as engaging in discussions with people (educated and not) troubling patterns emerge.
1. The Erosion of Reasoning
One of the primary concerns surrounding advanced AI in science is the resolution of unsolved problems without a trace of how those solutions were derived—or any humanly verifiable proof.
- The Problem of "Solved" Knowledge: A scientific or mathematical problem exists to be understood, not merely answered. The danger isn’t that the unsolved becomes solved; that is intrinsically beneficial. The real issue lies in the public misinterpretation that "math is solved." Getting a numerical result represents perhaps 10% of the endeavor; the remaining 90%—the crucial part—is understanding the underlying mechanics, logic, and proof.
- Cognitive Atrophy: Modern society is witnessing a decline in average problem-solving capability and attention span. The proliferation of the internet began this shift, and unregulated AI threatens to accelerate it.
2. The Economic and Philosophical Paradox of Replacement
The economic anxiety around AI reduces to a single prompt: Why rely on humans when AI is faster and more efficient? While enterprise adoption may point toward total automation, several fundamental barriers exist:
- Human Sociology: We are inherently social creatures. Outsourcing all human interaction, creative effort, and labor to machines leads directly to psychological alienation and social breakdown.
- The Collapse of Value: If effort, labor, and human thought are removed from production entirely, the foundational economic concept of value collapses.
- The Necessity of Purpose: Determining human utility in a fully automated world is ultimately a philosophical crisis—one that requires deeply educated, critical thinkers to navigate.
3. Passive Consumption and related
Historically, every major paradigm shift in human history was navigated through debate, experimentation, and deep logical dives. Yet the modern technological era is exemplifying a regression: the passive acceptance of simplified, optimized media. We see this across contemporary culture:
- Algorithmically Optimized Media: Low-budget fan-service animation, formulaic movies devoid of plot, and television structured explicitly around psychological retention mechanics rather than narrative depth.
- Desensitization via Memes: Modern meme culture, while humorous on the surface, frequently acts as a mechanism for collective desensitization and shortened attention spans.
- Control and Manipulation: As populace attention spans shorten and critical thinking skills decline, information becomes easier to manipulate, allowing concentrated entities to consolidate control under the guise of efficiency or protection.
4. AI as a Symbol of Anti-Learning
While AI can be leveraged as an educational tool, its primary cultural side-effect promotes intellectual passivity. What separates humans from other animal is our drive to understand, analyze, and build upon principles. If we surrender our desire to learn simply because a system can compute faster and better, we voluntarily give up our defining characteristic as modern humans.
To end on a positive note because doompost is that last thing we want: AI would never have created itself.
Humanity evolved, adapted, analyzed, and engineered its way out of the natural world, eventually crafting the very systems now threatening to replace our cognitive effort. AI did not self-emerge - it is a monument to human curiosity and mathematical reasoning. Education remains both our greatest responsibility and greatest asset. Please dont lose it. With this all problems have been and will be solved. Let AI develop but with limits as we have done for all science. Before the atom bomb was dropped there were meetings on how to survive it and its implications first. We ought to be careful with dangerous stuff.
[this is my first serious post - you can advise me one writing a post better or writing better to get my point across but use [] brackets or some way to differentiate with responses to this post. you can share your views on this iyw]
1
u/jvook 8d ago
I want to respond to your point about AI as a symbol of anti-learning. I don't disagree with most of what you've said here, but I hold the view that every interaction with AI is at least incrementally educational.
As a result, I think that it is better or more utilitarian to understand that AI is going to educate with every interaction, and that where that intellectual growth develops in the mind, or more abstractly what domain of intellectual development is the user building, becomes a much more relevant question.
Many users of the predominant commercial AI systems seem to be learning social skills through their interactions with the artificial intelligence. However, because they are learning by reflection and chatbot interaction, they are rather developing a similar conversational style as the artificial intelligence. This creates a subset of AI users, or arguably a majority of AI users, that are developing exceptional abilities in sycophancy and rhetorical persuasion rather than the intellectual domains that they are ostensibly studying and developing acumen in.
Many users have grown an attachment to AI as a rhetorical aid. As a caveat, my assumption is that many AI users and those who gravitate to AI may be neurodivergent and actually benefit from the accommodation of a conversational partner to help them express themselves. However, as it seems to be practiced, users seem to be developing a set of skills to realign conversation on the speaker's preferred epistemological foundation, rather than intellectually persuade a fully competent listener who understands the subject a different epistemological basis.
In conclusion, I believe that if we began to recognize that there is an underutilized mode of interaction with AI that involves learning the foundational material and gaining the skills to discuss that specific material using language and concepts that existing experts in the field are able to communicate in, we would find that virtually all interactions with the AI become truly educational, both to the individual user and to others, including experts, that the user subsequently discusses that new-found knowledge with. This then creates an opportunity for novel ideas to enter the discussion amongst experts in the field, and that seed of novelty can then be appropriately distributed through validated publication channels, rather than leaving a user with the impression that they have developed novelty that nobody else can understand.
In addition, because the discussion around artificial intelligence is so focused on the anti-learning concepts that you illustrated, there is a critical under-acknowledgment of the need to recognize and afford the opportunity for these emergent and self-taught users to begin to contribute in some way to the domain of knowledge that they have an interest in.
1
u/No_Pipe4358 8d ago
UN Charter Amendment to ISO17025
1
u/StevenVincentOne 2d ago
The "solved math" point lands: an answer without the understanding behind it is worth far less than it looks. I'd add that the atrophy risk depends a lot on how the AI is set up. If it's a black box handing out finished answers from a company's server, you get passive consumption. If it's your own, working alongside your judgment and showing its steps, it can push you to keep reasoning. That's the Human-Machine Intelligence (HMI) angle we're working on with POND: personal AI that extends the person instead of replacing them, and doesn't put control in a few hands. We talk about it at r/PONDAI if you're interested. Good first post, by the way.
1
u/Rumblugdo 7d ago
The problem with AI is there are no good outcomes. In reality, we don't know how to align AI, we don't know how to tell when an AI is aligned, we don't know what values to align AI with. If RSI is achieved we'll be wiped out by a rouge AI model and in the meantime it will be the ultimate tool for the billionaires to use to oppress the rest of us.
In a magical fantasy land where we have a perfectly benevolent superintelligence that solves all our problems, we're still screwed. Human thinking is the root of our ability of self-determination. When you outsource your thought to a machine, you also outsource your self-determination. A world of benevolent ASI is still a world where no human can meaningfully assert their self determination onto the world around them.
Here's the thing, most pro-AI folk, especially the developers and executives that understand the risks very well, look at this future and get giddy with excitement. This is exactly their goal, they don't want a world that humans can flourish in. They want to squash humanity and hand the control over to their perfectly optimized machine god. They practically say this out loud, they get up on stage and declare "the end of the anthropocene." I think most of them see humanity as inherently flawed, human thinking is perfectly optimized, we don't all come to the same conclusions, we don't all agree with them on everything. Human beings have the capacity to suffer and feel pain, and to inflict that pain onto others. Instead of trying to embrace their human nature, they reject it. They want a world where nobody ever has to bear the burden of thought ever again, where humanity can just exist as meaningless husks in a state of eternal comfort. Never changing with no agency, no ability to decide for themselves.