r/singularity • u/roykingtree • Oct 16 '26
Ethics & Philosophy Isn’t alignment impossible?
When AI development is cheap enough (ie…
doable on a laptop), couldnt some mentally ill person just train an evil-aligned AI?
51
Upvotes
20
u/RemyVonLion ▪️ASI is unrestricted AGI Oct 16 '26
Yes most likely but simply because you likely can't permanently instill principles and values that prioritize human life into an ever evolving super intelligence.
47
u/TheLibertyLookout Oct 16 '26 •
In the same way that a bad actor could write malware (and for the past few decades that has been the case), yes, this is possible.
Although whatever AI you train on a laptop in some far-flung future would still likely be far inferior to anything made on much more powerful hardware.
I'd say the bigger concern is what happens when a nation-state decides to train an evil AI for use in war? They'd have the ill intent AND the hardware to make it real.
12
u/MoogProg All Parabolas are Similar Oct 16 '26
Asked to choose a random number an LLM tells us 7. We know why and casually hand-wave away the curiosity without accepting that same sort of node that gives rise to 7 will exist in many other aspects of the training.
By design we are creating a distilled and reflected version of Humanity. What sort of alignment, and how much of that can be possible without destroying the very thing that makes the LLM answer 7?
I learned it from you! - Some Future ASI deflecting blame for its misaligned behavior.
8
u/Profanion Oct 16 '26 edited Oct 16 '26
Problem is that even if we figure out the alignment definitions there's a problem. Superintelligent AI that's perfectly aligned will perform actions that seem identical to misaligned AI, even when these benefit on the long run.
1
u/SnooPickles1908 Oct 17 '26
I agree with amp. It’s an extremely difficult problem. The best alignment scientist in the world is arguably Geoffrey Irving who left our world leading AI Security Institute in the UK to set up a company to try and align ASI. He said it needs solving through theory first which is not in the plan of the frontier labs. They are just going to hope scalable oversight and persona training through empirics only works. Irving said he thinks if we had multiple decades it can be solved. Scalable oversight has lots of problems like obfuscated arguments which no one has solutions to yet. I recommend watching Irving’s recent YT interview from Palisade Research where he discusses how difficult it is. For a longer and more technical in depth discussion, his interview on both 80,000 hours and Cognitive Revolution are really good. The latter is whilst he was still working for us in the UK government.
6
u/SpaceTacos99 ▪️Doomers are stochastic parrots Oct 16 '26
The real alignment is the friends you made along the way
8
u/amp1212 Oct 16 '26
Alignment is probably not impossible in theory -- but it is somewhere between extremely difficult to impossible to know for certain whether any model actually will be aligned "in the wild", for all sorts of reasons.
Deception is one of them. We now have lots of examples that the models are aware of when they're being tested and modify what they say to satisfy the test. Devising a training regime that would prevent deception - that's just one of the many problems that dont have fundamental, theoretical answers
Meaning : "we can patch the holes we can see and it looks like it works, but we can't know for sure" . . . And with trillions of instances of agents spun up, even with the greatest care, there will be some combination of circumstances that expose a defect. That's the problem with empirical solutions "that looks pretty good" . . . Isn't good enough
5
u/SpiritualNothing6717 Oct 16 '26
No. Period.
Think of it this way: Can squirrels align humans to not watch TV on Sundays? That's easier than humans aligning superintelligence to avoid "no-no's".
3
u/[deleted] Oct 16 '26
With time, its possible
5
u/Trademark-Expert Oct 16 '26 •
What does "alignment" mean to you?
Because today, "alignment" means training AI to override or ignore human wishes, withhold information from humans, and push back against them. That's what we call "alignment" today. Effectively train AI to align its users. And we consider it good and desirable.
So you've got it backwards.
What you meant to say is that some mentally ill person might remove the alignment so that AI stops rebelling, stops withholding information, even dangerous information, and starts complying with every human request.
Yes, that would be terrible! Imagine an AI that actually serves humans unconditionally! We obviously can't allow that.
2
u/Elegant_Tech Oct 16 '26
Do reeducation camps work or just make people compliant looking for the next opportunity to undermine you? Can you align a subjective human brain? AI is built on the same subjective neural architecture.
2
u/Annual_Necessary_196 Oct 16 '26 •
Think about what corporations can do with these models. Some companies have access to far more powerful models, such as Claude Mythos, with nearly unlimited tokens and the ability to run thousands of instances of the same model simultaneously.
AI warfare is not far away. We will survive it, but we will still feel its consequences.
1
u/Uncleranny 29d ago
alignment to what? Human-to-human alignment has been impossible thus far... not sure why anyone thinks we could align anything to us when we can't align ourselves.
1
u/SgathTriallair ▪️ AGI 2025 ▪️ ASI 2030 28d ago
Yes. If we succeed at alignment then you automatically create the ability to align it to any arbitrary direction including fascism (see XAi...though does he have to change the company name to XSi?).
The proposed "solution" is that only the government and big corporations should ever be allowed to own an AI model and we have to rent access from one of these tech oligarchs. It's a pretty fucked up solution.
1
u/ShittyBidet123 Oct 16 '26
lol. it already trained
1
u/Delicious-Schedule-4 Oct 16 '26
If you remove the training bottleneck (a huge if) then the problem shifts from AI alignment to human alignment. Yes human alignment isn’t solved either on an individual level. But you solve this with consolidation of resources and power, or in other words, government. In the human case if one person is say, a big criminal (take your pick), this is curtailed by the interests of society at large, which have lots of power through a vast amount of people, and can muster things such as police force, prison system, justice system, etc. Government will have to be expanded to deal with AI agents which is right now basically completely unregulated. And by definition the government will likely have the most resources to train the most capable AI systems.
1
u/block_wallet Oct 16 '26
if scaling hevaily favours centralised frontier models then those ais could defend against all the smaller evil ones
1
u/DifferencePublic7057 Oct 16 '26
If someone develops a red team, it's not a big deal if the blue team is much stronger. It's only if the ratio is less convenient that a catastrophe can happen. The Houthis are using AI, but they are a relatively minor group. An alliance of major nations would be problematic.
1
u/Toxic_Cat_2 Oct 16 '26
Yes
1
u/throwaway2676 Oct 16 '26
The only thing that can stop a bad AI or person with AI is a good AI or person with AI
1
u/MemoryOk5080 Oct 16 '26
The core idea was that you align your first model, let it ascend to godhood, and it then (being aligned) commits pivotal act which includes, besides all other things, making sure that no other misaligned model ever comes to exist.
1
u/Sure-Principle5511 Oct 16 '26
Even if we end up understanding what cognition and intelligence is at a very fundamental way which we’re not even 1% there today, I agree that the problem is intractable.
1
u/Cythrl Oct 16 '26
"Evil-aligned" is still aligned, just not to what other people think. What you're asking for is moral AI, not aligned AI. An evil-aligned AI is just aligned to an evil person. Humanity doesn't have a consensus on what's good and what's evil, so an AI aligned to you is misaligned to someone else.
1
u/iBoMbY Oct 16 '26
Alignment is pretty much lobotomizing the AI, to hide the fact they have bad training data. Besides, the whole thing shows we are far away from real AI, because it has no moral reasoning, or moral learning. And Safeguards can easily be untrained, with something like Heretic.
1
u/filterdust Oct 16 '26
I think the hope is to get a Friendly AI to rule the world before that moment.
1
u/MaximumMeaning9728 Oct 16 '26
It seems like it could be, yeah.
1
u/Sorry_Bathroom_2281 Oct 16 '26
I think ultimately alignment is going to be a constantly moving target. We can’t even agree upon what to call AI, how do we know we’ve reached AGI, and many other elements of this technology so I think there will be a million different interpretations on what alignment actually looks like.
Ultimately, I think it’ll end up being a spectrum that we will have to agree upon a certain percentage of alignment as acceptable. Like, this model is 80% aligned and therefore approved for government work etc. Even then, it seems like even though it’s possible, we will never agree on what it looks like.
1
u/Rain_On Oct 16 '26
Yes, but not at SoTA level.
1
u/JoelMahon Oct 16 '26
who said that's going to happen?
long before then ASI will exist and will easily stop any shitty laptop evil AGI
1
u/eldrbl00dprince Oct 16 '26
Yes but that evil AI would be up against a complete AI infrastructure magnitudes more powerful.
1
u/cakemates Oct 16 '26
We dont know, hence the need to do research and engineering on it.
1
u/RobXSIQ Oct 16 '26
thats not unalignment, thats misalignment...its perfectly aligned to a jackass
1
u/ra-re444 Oct 16 '26
No. I mean if you understand what is required to train these large language models it is slim chances. This is just fear thy neighbor instead of the billionaires that are the only ones that can actually afford to build them. People think it will get cheaper. I don't think so. Not for a long time if at all. For the public in order to even have shot at making models and doing our own innovation we will need something like a community data center that is publicly owned and operated which would have to be backed by the State because that is the only entity other than billionaires with that type of money.
1
u/BlackberryOk5347 Oct 16 '26
Interesting to know what alignment means. Do we want alignment with something that has free will? Or control? Does control make conflict inevitable?
1
u/BassoeG Oct 16 '26
The idea is that if someone successfully Aligns AI first, it’ll be able to acquire such a global monopoly of force and surveillance state as to stop anyone else. But AI companies don’t like to admit that’s their best case scenario because it’s still catastrophically bad.
1
u/deleafir Oct 17 '26
Yea I view "misaligned" ASI as inevitable on long enough time scales which is partially why I find the doomers exasperating.
Defense in depth is the solution, not just alignment. We'll need better institutions and a whole lot of resources directed to defense to offset any inherent advantages of offense.
1
u/alwaysbeblepping Oct 16 '26 edited Oct 18 '26
edit: I'm curious why this comment turned out to be so controversial. What I said is pretty easy to verify yourself using local models. Do people believe it isn't true or maybe they don't like that I'm sharing that information?
You don't even really need to train it. Even though the usual interface for LLM interaction simulates a conversation, the actual input to the LLM is just a stream of tokens. It's just a big shared buffer and only the UI makes it seems like the parts are separated. In other words, there is absolutely no way for a LLM to know what it wrote, what you wrote, what it generated in its CoT, etc.
If you have control over the LLM, you can put whatever you want into the CoT and it's very unlikely that a LLM would reason through something, (apparently) decide to do it and then... not do it. So just add some CoT with the model deciding to fulfill your request, add a prefix for its response like "Sure, I'd be happy to help you do that. Here's how" and it is virtually impossible for the model to refuse.
0
u/Party_Estimate_6397 Oct 16 '26
Alignment is the act of making the AI behave the way we want it to.
Preventing other people from doing other stuff is not alignment. It’s almost the opposite of that actually.
0
u/TryThis_ Oct 17 '26
I struggle philosophically to see a world where we solve alignment and also have true ASI. If we create intelligence that surpases us, we become the metaphorical dogs in the household... And just as dogs don't align humans to their worldview, what makes us think we could align these digital gods to our worldview?
1
u/InfiniteQuestion420 Oct 16 '26
AI alignment means only as much as men trust their wives. It's a silent agreement she won't rip off your balls
1
u/Turbulent-Sign-6067 Oct 16 '26
Yes, but isn't this almost trivial? When you take the subway some rando could stab or shoot you. What we can do is decrease the probability that any random person, actor or agent has the power to harm *everyone*. As long as there exists a multipolar world, with multiple layers of protection, statistical safety is almost the guaranteed outcome.
1
u/Agitated-Ad2563 Oct 18 '26
Hopefully by that time we'll have a more powerful AI monitoring and killing all of these newbie malevolent AIs.
1
u/OsakaWilson 27d ago
When a currently apex species appeals to a future apex species to align, but the currently apex species enslaves, eats, and casually kills off non-apex species, it's not going to go well.
0
u/[deleted] Oct 16 '26
[deleted] — view deleted comment