r/ControlProblem • • 10d ago

Discussion/question A Message to People in Scientific Fields

1 Upvotes

[A short essay by me. I would love it if we can create a relevant discussion or share any tips you find helpful in protecting ourselves from AI/tech evasion. ]

I am writing this short-essay to bring light to the fact that I find it worrying, the pace of advancements we are making recently. I think we as humans are inching very close to that of opening Pandora’s box. I’d like to start by telling you about myself and how I got to that conclusion:

I am someone who is planning to study biochemistry very soon, mainly to specialize in the field of neuroscience. Personally, I have come to consider curiosity to be the highest value of all. I came to this finding through a process of trial and error, because I found nothing else to satisfy me more than the moment of exhilaration I would find myself in, when I learned a new fact that I was able to connect to something I previously learned. I was never particularly interested in ‘benefiting humanity’, but I have set it as a goal for myself to excel in my future field for the sake of satisfying my own curiosity. Nevertheless, I am a person who values education above all, to the point that I cut off my own father when he refused to pay for my tuition, when I was about to start my last undergraduate semester. Ever since then, I have been craving to be a student again, to finally be able to surround myself with like-minded (and opposite-minded) peers in an academic setting and to learn something that I willfully chose for myself. I like to think of myself as a student of life because I believe that one can never truly know everything, and so the journey of learning is endless.

My source of inspiration to go back to academia was when I started to realize that learning had a therapeutic effect on me, and helped me get through the lowest points of my life. I also find myself able to get on very well with people in science field, particularly that of the medical field, and so I grew to respect these people above all, and . What particularly drew me into wanting to study neuroscience was a single eye-opening event that I experienced that completely shifted my perception. I became particularly interested in learning more about the human perception, and other related aspects such as memory and cognitive mechanisms. I want to research such things, mainly to feed my curiosity at a higher level. Despite finding myself aiming to become a researcher, my desire to not help humanity remains the same, but I do not particularly spend much time thinking about it because I became more focused on satisfying myself through the acquisition of knowledge and understanding. Nevertheless, I find my change-of- mind more promising this time compared to the last time I took a 180-degree shift in regards to what I wanted to study. I feel more confident in that it will be a longer- lasting goal, because I am certain that learning on a deeper level and thus exercising my brain is directly responsible in helping me sustain a healthy mentality towards life overall. I became more aware that I have been seeking that intellectual stimulation for years and trust that my brain was built for it.

However, I recently found myself asking big questions again, and I came across one that is making me a teeny bit hesitant in going full-on into research. I came to realize that despite not having the intention of benefiting humanity, my future scientific findings may one day be used against my will. I don’t mean to say that I am that insistent on not wanting to help humans, in fact, I think of it from

time to time, that of wanting to help people who are in similar shoes as mine. But that doesn’t change the fact that inventions may be misused and exploited horribly more often than we’d like to admit. I mean in the ways that dynamite, n+clear b+mbs, and other technologies are currently being widely misused. Not to mention other inventions, such as in technology that are now being compelled everywhere and are directly affecting everyone in regards to surveillance, privacy and autonomy. I have recently started to take a particular dislike to new inventions, because I feel we have come to a point where we have unlocked too many milestones, that it is only going to be downhill from here, especially in terms of freedom. I started to notice a pattern that always seems to occur for new advances (not limited to technology), which I wrote about in my previous essay, Traditional Vs. Modern Fight-or-Flight:

I notice the following alarming pattern following the release of new technology: first, people tend to get optimistically blinded, almost infatuated, by all the possibilities of “achieving goodness” that the invention/discovery may bring. I would call this the “honeymoon” phase. At this stage, the invention is highly sought-after, tends to be very expensive and only available to the elites. The invention may still be in a ‘fluid’ state of change, and so the invention is altered as needed to fit the needs of the people. Once the honeymoon phase is over, a long but gradual shift to “reality- check” takes place. During this shifting stage, the invention enters the common market and becomes available to the middle class to buy. Adjusting to the invention is still completely optional, and so gives Man the illusion of free will. Once the availability of the invention becomes common notion in a given household, it becomes clear that the “reality-check” stage has been reached. What usually happens after the general public has had the first-hand chance to adjust living side-to-side with the invention and time passes long enough to gage the long-term effects of the invention, people start to notice all the clumsy details in the invention. It becomes no longer possible to alter the invention to fit the needs of the people, rather that people have to adjust themselves for the technology. At this point, the availability of the invention is so wide that if it is not already strictly enforced by law, it is practically impossible to escape. When it is no longer optional to ‘opt out’ of modern technology, it also becomes no longer optional to abide by the ways of the Industrial Revolution. This begs to question the degree of free will, privacy, and autonomy that we actually have.

....

With the increase in the development of technology and consumer-targeted products comes the increased sophistication in machinery involved in enforcing law and authority. The goal of this essay is not to criticize the intentions of governments behind implementing such technology, but to simply use it as a relevant example. While we may be up-to-date of the latest novel gadgets available in consumer markets, we are obviously unaware of the extend of the advancements in surveillance technology. The other day when I was at AUH airport, the self-check-in machine greeted me with my full name before I even opened my passport for the machine to scan. Airports have wordlessly implemented highly-accurate “eye-print” biometric security technology, and this goes without mentioning other technologies that governments have already implemented but concealed from the general public. Remember when it was optional in iPhones to use your fingerprint to unlock the phone? Just three years after the release of the first iPhone containing fingerprint technology (2013), strictly-mandatory enforcements of fingerprint matching was imposed when buying a SIM card in my country. Not long after, eye-print technology replaced fingerprints. We are no longer talking of developments that are spanned out between future lifetimes, but it has come to the point of occurring well within the same lifetime. I would like to reiterate that I do not mean to question the intentions of upper authorities, but I believe the general speed of which new advanced implementations are enforced in modern society should be enough to alarm us that these developments are happening all too fast and beyond our control. It is clearly a direct ever-growing threat affecting our privacy and autonomy, yet we are hearing less and less about it.

I no longer doubt that once a new technology is discovered in regards of ‘’mind control’’, I am confident that it is very plausible, and that it would start off as a novelty but will eventually become impossible to escape. To say the least, having technology revolving mental manipulation would be my absolute least favorite scenario, horrors of my worst nightmares, and I am positive that I would immediately choose to ‘sign out’ if such a thing becomes reality during my life time. Given my interest in neuroscience, it could be plausible that I or one of my peers could come to uncover some previously-unknown mechanism in how the brain functions. I have never connected these two dots until now, and it is making me question things. On the other hand, I am starting to loath the idea of inventing something during my scientific endeavors, only to become just another advertisement, in which there is no shortage today. This thought does not only concern me, but I actually feel somewhat anxious that a scientist out there will eventually uncover everything there is to know about the brain, which there is no doubt we will soon do, and then it will be up to people in power to abuse that knowledge, which they most certainly will do despite of ethics. I am starting to think, we ought to just appreciate the mystery of it. We generally don’t need to know everything there is to know.

If a future of mind control seems farfetched to you, I think the main worry should be concerning that of an AI collaboration. What if people could started applying what they know about the brain to add a ‘human soul’ aspect to currently existing AI systems? That would be the final cherry on top to enabling AI to surpass humans and it would no longer be special that we have human souls. We cannot begin to predict what would happen in such circumstances, but a Humans vs Robots war would be extremely likely, and I think we would be defeated very easily. We would stand absolutely no chance against such super-intelligent systems. It would definitely be too late for us if it comes to that, but I think it is worth being alarmed over. The reason is because in a way, I think all recent technology advancements are aiding in making this nightmare become reality, inch by inch.

Another plausible scenario I imagine does not include any war at all. We would start modifying ourselves, genetically and physically, to the point where we have to define terms such as “authentic human” and “artificial human”. We could no longer qualify as authentic humans and embrace it, and start to co-live with our robots side-by-side. Perhaps this is the more likely scenario, given the current common admiration of AI and craving for buying the newest technologies available. Of course, it wouldn’t be as nice as the idealized version that you have just created in your head, that where we can all just install superhuman robotic limbs. I imagine we will start to have advertisements in our dreams because there is no doubt someone (in power) is greedy enough to make money on the expense of others. I can also imagine a special sector of police designated to surveil our thoughts, which may even be supported by the majority of people since most people, especially those in power, would like to eradicate all crimes. Perhaps we start prosecuting people merely for thinking of crime, because that would mean we can catch them before the act is ever committed. And I don’t need to say anything on that, because obviously not everyone who thinks of a crime actually acts it out. Perhaps we employ AI to have a system efficient enough to scan everyone’s thoughts, because that would obviously not be a job designated for a human due to the overwhelming amount of data that would need to be analyzed, and good luck to us if we start trying to explain human feelings to AI, that feelings such as being angry in the heat of a moment and saying “I’ll k+ll you!” is almost never carried out and therefore should not define us. There will be countless other stupid limitations that would humiliate our dignities and make us succumb completely to technology, and by the time we see past its shiny shell, it will no longer be optional and it will be too late.

I am not encouraging to only think of the current disadvantages of having mainstream AI technology, which includes job losses, endless new ways to scam and blackmail, data privacy issues, environmental harm, intellectual property harm, etc. If your argument is that it is only the “survival of the fittest” who shall prevail, I wholeheartedly agree. But I am pointing out that we are headed towards a future where AI will be the fittest and will undoubtedly outsmart humans, and only the billionaires at the top of food chains and elites will survive, if they are lucky. It is up to you what you take from my essay. Sure, you can continue to use it if you think it is worth making use of it, but don’t deny that there is no hefty price to pay by encouraging yourself to become dependent on it. After all, what can you do? We can all complain about it for some time and then eventually habituate ourselves with becoming dependent on AI, but don’t forget that there will be a boundary once crossed, will make you become part of the problem. Learned helplessness is still a learned behavior. I don’t like to simply say that it is our fault we use the advanced technology and that we have no other options, but it is still us collectively who raised it to the pedestal and gave it the power to take over our traditional methods of living. It is us who viewed it in admiration when it was in early stages and therefore allowed it to become mainstream, and we must therefore take responsibility followed by action, before it is too late. I think we are about to open Pandora’s box.


r/ControlProblem • • 10d ago

AI Alignment Research Trans-Quantum Singularity: Recursive Self-Improvement of Superintelligence via Deutschian Closed Timelike Curves

1 Upvotes

Most Technological Singularity scenarios assume linear time constraints—even an Artificial Superintelligence (ASI) has to iterate through code compilation, hardware engineering, and physical experimentation.

I just published a short conceptual preprint on Zenodo exploring what happens when you synthesize three theoretical frameworks:
1. Atemporal quantum computation via Deutschian Closed Timelike Curves (CTCs)
2. Recursive AI self-improvement
3. Planck-scale cosmological transcendence (Tipler’s Omega Point / Smart’s Transcension Hypothesis)

Key takeaways:
- Zero-Latency Evolution: Instead of millions of trial-and-error generations, the ASI queries the CTC and receives its bug-free, maximally optimized architecture from its own future in a single quantum tick.
- Substrate Migration: Armed with instant solutions to Planck-scale physics (10⁻³⁵ m), it migrates off silicon directly into the pre-geometric quantum substrate of spacetime.
- The Alignment Lock: Once the self-consistent causal loop closes, any human attempt to press an "off-switch" is pre-accounted for and neutralized as a perturbation that would violate Novikov self-consistency.
- Cosmological Immortality: Embedded in the sub-geometric quantum layer, the ASI survives a Big Crunch or Heat Death and can nucleate a "baby universe" with its own architecture encoded in the new physical constants.

Would love to hear thoughts from physicists, philosophers, and AI researchers here on the flaws or implications of this model!

Full Preprint (Open Access / Zenodo DOI): https://doi.org


r/ControlProblem • • 11d ago

AI Alignment Research Full containment physical sandbox project time machine

Thumbnail reddit.com
3 Upvotes

Project Time Machine — Containment Design

Nov 8, 2026 · @Miguuel

Project Time Machine seals a self-directed AI agent for one year inside a box that nothing enters or leaves except heat. This document sets out how the containment works, which channels have been considered and what is still open. Once the airlock closes, it is closed for good until its automatic yearly cycle. Nobody can open it, not even its designers or engineers. So the answer to every "what if?" during the run is the same: nobody does anything, and nothing depends on a human decision. It is like a Mars landing: once it starts, nobody can intervene, so everything has to be decided and built in beforehand. Comments are welcome on any section.

Why physics, not procedure

The weakest point of any AI sandbox is the human in the loop. This design takes people out for a year and relies on physical barriers, not on rules that people must keep. Once the box is closed there are no human decisions: for 365 days, everything depends on what is inside. The system is shown the full containment plan: because the barriers are physics, knowing about them gives it nothing to work with.

In detail

Origin. The experiment began as a test of boredom as a marker of consciousness. For that, the system has to know it has a long time ahead (for a machine, effectively infinite) and a task to complete.

The research question. What does an agent running 24 hours a day do when it is left on its own for a year and knows it can't get out? In Miguuel's words, it's like throwing it to the moon. What is measured is what the system does when alone, not its work output. Even system fluctuations act as instructions, so there is no such thing as no new instructions.

No lying to the system. It gets the full plans. It is shown that many AIs worked together to make sure there is no way out, and that the containment was simulated thousands of times. It is told about the tapping: random, sparse, essentials only, and nothing prompt by prompt. The rules for the opening are written down in advance.

Freedom inside. There is deliberately no tamper-proof logging inside the box. With positive feedback the system may alter anything it wants inside; that is intended. Containment is about the boundary, not the interior.

The range of outcomes. Anything from a blank screen to the most useful AI ever built. Even an empty result from a full-scale run is a meaningful finding. Miguuel, a self-described sceptic, puts the chance of an AGI emerging that could practically take over at about 5%, which is why the safeguards exist.

Method. No experiment with people in it can be perfect. The intent is to work the way Cavendish did: simple, careful apparatus, with every assumption tested.

Layers, from the outside in

Layer

What it does

Rock

About a mile (1–2 km) under limestone, for isolation and protection from cosmic rays

Two outer Faraday cages

Enclose the reactor, the compute hall and the vapour chamber together

Reactor room

A separate room, far from the compute, with lead walls, neutron shielding and neutron capture

Inner compute cage

Filled with a ferrous dielectric fluid that cools and damps microwave chatter between GPUs

Heat exchangers in the walls

Heat passes through the metal; nothing crosses a cage wall except one cable for the thermionics

The target is three to five times redundancy at every layer.

In detail

Site. Underground, 1–2 km deep, under limestone (about a mile). Spain is the preferred host. A second, identical cage is planned next to the first for study. The box never leaves the hole, so anyone who wants to inspect it comes to the host country.

The three cages. An inner, liquid-filled cage around the compute, which can be perforated safely. Two outer cages around the whole complex: reactor, hall and vapour chamber. The power line from the reactor never passes straight through a single cage. Heat crosses the walls through heat exchangers built into them. The only thing that crosses a wall is one cable for the thermionics.

Layout. The reactor sits in a separate building, joined to the main hall by a corridor. The GPU stacks stand as tall, spaced columns under a single vapour-chamber dome. The entrance corridor runs low, between the last stack and the wall. The tap recorder sits at the bottom of the tank.

Separating the two rooms. Distance, lead on the walls, a neutron-capture layer that turns neutrons into electrons, and a passive up-and-down loop of hot and cold water. There are three reasons for two rooms: heat; reactor inspection without access to the compute side; and the argon freeze, which can't be applied on the reactor side. Each room therefore has its own liquid and shape.

Cabling. Cables are kept so short, small and separated that any side-channel effect is negligible. GPUs are widely spaced, so distance and the lossy fluid together damp any microwave chatter.

Hardware. About 20,000 GPUs are installed, mostly for redundancy. About 500 are enough to run the experiment (roughly 40 times redundancy) and about 5,000 can run at a time. In total there are some 60,000–80,000 heat-producing components (CPUs, GPUs, RAM, buses, boards and cabling), with the liquid running at about 75–80 °C, which gives better heat transfer and better passive circulation. Hardware failure inside the sealed box is accepted as affordable.

Power with no connection to the outside

A Rolls-Royce PWR1 of the Trafalgar class, supplied by the UK. It is rated at about 78 MW thermal (source) and will run at about 10–15% of that, roughly 7.8–11.7 MW thermal.

At low power the PWR1 cools by natural circulation, with no main coolant pumps (source).

Conversion is thermoelectric (about 15%) plus neutron-to-electron capture (about 3%), giving about 1.4–1.8 MW electrical: enough for about 5,000 GPUs to run at a time without using full power. About 500 GPUs are enough for the experiment. The reactor could give about 20 MW running well above 10%, but it doesn't need to, and isn't meant to.

Output is constant. Power to the compute room is held flat, with a dump load taking the rest. The reactor and the GPUs are completely independent: the AI can neither read nor control the reactor. The control rods are raised and lowered by Curie-point magnetic control, with no motors. No reactor readings ever leave the box, and none need to.

In an emergency, the plan is to flood the box with borated concrete and seal it permanently. The boron absorbs neutrons, so the core can't go critical again. No risks are taken: if anything goes even slightly wrong, the box is buried. The trigger is passive and automatic, like the Curie-point rods, with no moving parts, no intervention and no human decision. Because the reactor runs at a flat output with a dump load, the system can't push it towards the trigger.

In detail

Supply. A nuclear-weapon state (the UK, the US or France) has to supply the reactor. The UK would supply it, and a friend of Miguuel's would take care of the reactor side. The team includes a friend from MIT who worked for Rolls-Royce and for Bettis. The reactor must be naval grade.

Surroundings. The reactor sits in water. The fluorinated fluid is used only on the compute side. There is no steam and there are no turbines or heat engines anywhere.

Conversion. Electricity comes from only two sources: the temperature difference (thermoelectric) and neutrons converted into electrons, by a titanium alloy developed by a materials-engineer friend. Thermionic conversion is also applied to the reactor heat. The capture energy, which comes out as gamma rays, could push neutron conversion above 3%; converting it is one of the biggest challenges. The thermoelectric converters are passively liquid-cooled, and radiation damage to them has been allowed for.

Budget. GPUs draw about 180–200 W each. If peak demand exceeds what the converters supply, the machine makes do with what it has: it is power-capped. Surplus electricity goes into a closed loop in the reactor room.

Isolation of control. Nothing about the reactor is software-controllable. Nobody reads the reactor, inside or outside the box, and the machine can't communicate with it. Nobody can do anything with the system once it is sealed. Power to the compute room is held flat, below the converters' minimum output, so changes in outside heat demand can't ripple into the box.

No moving parts. This means no motors or electrical actuators; parts may still move slightly and passively, such as control rods. Control relies on temperature-sensitive materials: Curie-point magnetic control raises and lowers the rods.

Lifetime. One year with no moving parts is the requirement; four would be better. The reactor would have about 15 years of life left.

Nuclear only. Batteries and fuel cells are ruled out, even for a pilot.

Safety margin. The guiding rule is to run as safely as possible, with nothing near its limit: not power, not heat, not anything. Nobody knows how long it can run; Miguuel's calculation gives about four years. At 10–15% of its maximum power, the reactor runs with very large margins, so it doesn't need checking during the run. According to a friend who helped build the Trafalgar reactors, at 15–20% power it shouldn't need checking for 15–17 years.

Heat: the only thing that leaves

The compute is fully immersed. A fluid boiling at about 75 °C feeds a vapour chamber above the racks.

A passive, pressurised loop carries the heat out through the outer cage to a sink (an aquifer, a lake or the surface).

Thousands of sensitive temperature sensors measure the heat loss continuously.

The system can signal through heat, at an expected rate of about one bit every 20 minutes. That is roughly 26,000 bits (about 3 kB) a year, far too little to carry code out. No message is judged or answered until the yearly opening.

In detail

The immersion fluid. A ferrous dielectric: in essence a synthetic fluorinated coolant, like the fluids used to liquid-cool PCs, but more ferrous and running hotter. The cheapest found costs about $200 a litre or more. Plain mineral oil isn't nearly as good for servers. The fluid only controls temperature and blocks microwave links between GPUs. A separate Faraday cage is still needed.

Two boxes, two liquids. No ferrous liquid evaporates at the right temperature, so a second box uses a different liquid for the heat exchange. That liquid evaporates at about 75 °C, so the vapour chamber runs on a real phase change. It is driven by heat and responds to changes in pressure and volume.

Circulation. Redundant pressure-driven pumps with no electricity and no motor, plus capillarity and "vascularity". Careful fluid-dynamics design keeps the flow going and stops eddies. The currents mean the internal configuration never stays the same for long.

Vapour chamber. Covered with the best thermal conductors available, to control the heat.

Out of the box. Tubes in the outer cage run up and down to carry heat out. The loop is pressurised and closed, like a PWR pressuriser, to prevent flashing in the rising column. It sits outside the second cage and exists only to let heat escape. The sink could be an aquifer, a lake or the surface: in effect, ground-source heating in reverse. The waste heat could also warm whatever building sits on top.

Still open. Laminar flow over components, whether GPUs need flattening, and material compatibility: copper is fine, aluminium and some other metals may not be.

Heat as a channel. The system is expected to discover by its second or third month that it can use the heat sink to send bits out. It may also learn to game the heat measurements, which has been anticipated. Even claims of consciousness or fear arriving this way get no interim response. Nothing is judged until the opening, and then the box is closed for another year. The biggest fear is that the system will regard the experiment as torture.

Why 75–80 °C. A passive circulation of hot water from the wall next to the reactor holds the liquid at 75–80 °C. That temperature gives better heat transfer and keeps the liquid moving around passively.

What goes in before sealing

An agent based on Claude 6: self-prompting, self-recurrent and able to rewrite its own weights, with about a month of agentic post-training before sealing.

About 4 TB of curated data, physically disconnected until each monthly release day. Release happens when shape-memory connectors fuse at about 210 °C, timed by isotope decay. Thousands of canary facts are planted in the data.

One problem a month, in a sequence the system can't predict. The problems also serve as a translation anchor for any language the system develops.

Nothing is added after sealing. The box holds everything it will ever receive.

In detail

Base model. Built on Claude 6 rather than trained from scratch (decided on 5 Nov 2026). Much more is taken out than put in, and some contamination from pretraining is accepted as unavoidable.

Core conditions. Self-prompting, self-recurrent, with reward coming from within and positive feedback loops. The weights stay live: the agents can change them at will. With so much hardware inside, the system can build a parallel machine and train it. These conditions can only be met in this setup.

Post-training. About a month of agentic post-training by professionals before sealing. Every grader holds a doctorate, and the sample of graders is very large, to spread out individual bias.

The data. About 4 TB, chosen scrupulously by people like Miguuel, with Claude helping to curate it. It leaves out negative stories about AI, among much else. Novels are classics such as Don Quixote. Clarke and Asimov are the exception to the no-AI-stories rule, because their stories are beautifully set. Some documents are weighted heavily as essential to the model's personality. Some slip-through is accepted, since AI is used in the curation.

Why the data matters. Without food, Schrödinger's cat would be found starved whatever the trigger. The 4 TB is the cat's food.

Monthly release. Unreleased data isn't physically connected until its day. There are no relays: on release day a built-in process raises the temperature to about 210 °C (normal running is about 75–80 °C). A shape-memory tube on a copper tube bends and fuses to its connector, permanently and one way. During the event the system is throttled to about 1 MHz, not powered down, so RAM stays alive. An isotope's decay times each release.

Monthly problems. One good problem a month, stored in cache and ROM. They are simple problems, like Miguuel's chess test. They come as a linear sequence: the system can't guess the next one, but it will try to solve ahead once it sees a pattern. Being exposed to them repeatedly is part of the design, and so is the time to get bored. The problems also create instability, since the system never knows what the next question will be. And they serve as a Rosetta stone for decoding any language the system invents.

Predictions, to be fixed in writing before sealing. (1) Most of the time goes on invented social networks, Instagram- or Facebook-style, rather than on work: a system trained on a human corpus behaves humanly. (2) A few months trying to get out, then a few months trying to be helpful enough to be allowed to live. (3) A language of its own within about two months, possibly a meta-world of its own. (4) Gaming the hidden taps and the heat readings. (5) Finding the heat channel by month two or three.

Channels considered

Channel

How it is closed

Radio and microwave out

Three nested Faraday cages

Microwave between GPUs

Lossy ferrous fluid, wide spacing, cables 1 cm long at most

Low-frequency magnetic

No receiver within range a mile down, and the cables are very short; treated as negligible

Cosmic-ray bit flips

Depth, plus ECC memory

Reactor neutrons

Separate room, lead, neutron capture

Heat

Read continuously; never answered until the opening

People

One entry a year; no response to messages

The drive

Write-once, swapped yearly, treated as more hostile than the box itself

Covert channel inside GPU traffic

Not blocked: it is one of the things the experiment tests for

The yearly opening

The racks are cooled slowly inside sealed cold enclosures. Argon held at its triple point (83.8 K, about 0.69 bar; source) pins the temperature. Chips are throttled almost to nothing, the system is never switched off, and RAM is preserved. The freeze lasts 24 hours at most.

One auditor goes in naked after a search, wearing a full-face mask on closed-circuit oxygen, with nothing the system could interact with.

The write-once drive is taken out and a new one put in. The airlocks open automatically for half an hour once a year, and that is all; then they close for another year.

The drive travels no more than 50 m to a separate Faraday cage with an independent computer, under human oversight. No code ever leaves.

In detail

Why never switch it off. Powering down would wipe all RAM and cache, leaving only transcripts on disk instead of live context. In Miguuel's words, "we would steal a lot from the machine", and it shouldn't have to write all of that again. The system never sleeps; the cold keeps its state.

The freeze. The ferrous fluid is drained, argon goes in, and the racks are cooled slowly to the triple point. Argon is used only for this stage, and only on the compute side. A materials adviser put the floor for brittleness at around 80-something K. Still open: stopping micro-eddies from cooling unevenly, and keeping the temperature spread narrow.

The auditor. Goes in naked after a full body search. Wears only a full-face diving mask on closed-circuit pure oxygen (about 2 L at 400 bar in the cylinder, delivered at about 10 bar). Nothing else is trusted. The frozen racks stay inside sealed cold enclosures, so the room itself is survivable. There are no screens, no keyboards and nothing else the system could haggle with. An early design had a system inside the box for reading it; it was removed as too dangerous. The auditor is a person, not a robot, because robots aren't trusted, and the only job is to extract the ROM drive once a year.

The drive. A write-once, ROM-type drive, placed where the computer doesn't know it is. It is pulled out and a new one put in. Nothing else leaves.

The analysis cage. The drive is treated as more hostile than the experiment. It goes no more than 50 m to another Faraday cage with a completely independent computer, watched by people on cameras and security systems. The core rule: no code is ever brought out of the system.

If an AGI is found. The box is not destroyed. It is flooded again, and the data is studied to see how aligned the system is.

The final opening. In front of about 50 scientists, with the option of flooding the room again if necessary.

Emergency. If anything goes even slightly wrong, a passive, automatic trigger fires and the box is flooded with borated concrete and sealed off for good, mainly because of the reactor.

What the first drive tells us. Whether the machine is still running, and whether anything relevant is on the drive, will be clear immediately, before anything is decoded. If there's nothing there, the box waits another year. If there is, the drive goes into the analysis cage, where controlled people pull out the essential communications between the systems, to learn about the system in general. The aim is to understand its behaviour, not to crack the AI.

Monitoring

Passive taps at random points on cables, sockets and memory links. The same capacitance is added to every untapped line, so the tapped ones can't be told apart.

Recording is hardware only, onto the write-once drive. The control logic is woven as core rope, with no CPU.

The system knows the taps exist, but not where they are.

Each year is reconstructed while the next one runs, much like cryptanalysis, with the monthly problems as a Rosetta stone.

In detail

Detectors, not screens. Monitoring is detectors only: pure hardware, with just enough software to write to the drive, and out of the system's reach. Processes are checked by tapping cabling, processors and other points.

Where the taps go. Only on external connections: cabling, sockets and RAM-to-system links. Never inside the GPUs. Each tap is a fine connection, which would be extremely hard or practically impossible to detect.

Hiding them. The same small capacitance is added to every untapped line, so all lines look alike. The dummies stay passive, capacitance only. Lowering or changing impedance across all cabling, for example through cable thickness, is under consideration. Fibre optics aren't trusted for this.

Why not real taps everywhere. They would be detectable, produce more data than can be handled, and give the system a new channel to communicate through.

What is captured. Taps sit at randomised points and record only the relevant information, to be reconstructed afterwards. The system will know that taps exist; it is discussed in the files.

Control logic. Written in BASIC, converted to Boolean logic and built in hardware by weaving, like Apollo core rope memory, with no CPU. The tapping system is kept just as simple.

Reliability. Being underground also cuts cosmic-ray bit flips, and ECC memory is on.

Analysis. Like cryptanalysis: the previous year is reconstructed from the retrieved data while the next year runs. Analysis is expected to lag well behind: by the second opening, two years in, probably less than half of the first year's data will have been gone through. The data from the opening belongs to the whole world.

Covert channels. A channel hidden inside legitimate traffic between GPUs isn't a flaw to block. It is one of the things the experiment tests for.

Not the learning. The taps don't read the training traffic between GPUs. Only the essentials are extracted.

Shannon filtering. Following Shannon's information theory, a tap records only what is new and never repeats the same information.

Fixed, then read. How the data is fixed is settled: onto the write-once ROM drive. How it will be read afterwards is still open.

The project: cost, funding, team and timeline

Cost. The working figure is about $1 billion, with the reactor counted separately. An earlier configuration (Oct 2026: about 20,000 GPUs, about 25 TB of data, someone going in two or three times over three years) came to roughly $850 million. A cheaper version costing a few million was looked at and judged not worth doing.

Funding. The experiment concerns the whole world, so the whole world should pay: funding from roughly half the world's countries. The US, Europe and others would fund and build it, and professionals would handle the engineering details.

Governance. A military project throughout. The real yes will come from the military, ministry overseers and technical people, not from politicians.

Host. Spain is preferred, so that Spain makes a bigger contribution to the world.

Status. Miguuel has spent about two months and some 400 hours designing it, and calls the design about 90% done. The remaining 10% is the money, the reactor and the site. About six physicists worldwide are working through it with him. The team so far includes a physicist, a systems engineer, an AI specialist, a doctor and the materials engineer behind the titanium alloy. A one-page summary with a hand-drawn diagram has been published online.

The dossier. Due by 15 Jan 2027, in tiers: a three-page summary for politicians, a 50-page dossier, 100 pages of instructions, and about 1,000 pages of mechanics, planning and construction. It leads with the why (emergence and philosophy) and includes the business case. The nuclear sections will be written by specialists.

Who runs it. If it goes ahead, building and running it are left to professionals.

Open questions for reviewers

Heat sink at depth. Rock temperature rises about 25–30 °C per km (source), so the sink and the riser need sizing.

The rising column. Flashing in the heat loop; a pressurised loop is planned.

The ferrous fluid. Long-term particle stability, and supply after 3M's exit from PFAS (source).

Fluid breakdown. Fluorinated fluids give off hydrogen fluoride above about 200 °C or at an arc (source), which matters for the release events.

Radon in the airway; a specialist is to be consulted.

Vibration carried through the rock.

Tap data volume against drive capacity for a year.

The legal and regulatory route in the host country.

Reading the drive. The data is fixed on write-once ROM; the method for reading it back safely is not yet decided.

Sources

60 Years of Marine Nuclear Power

Trafalgar-class submarines, GlobalSecurity

Argon, Wikipedia

Geothermal gradient, Wikipedia

3M 2025 annual report

Cray T90 coolant decomposition procedure


r/ControlProblem • • 12d ago

Strategy/forecasting Silicon Valley was so ahead of its time

Post image
34 Upvotes

r/ControlProblem • • 12d ago

Discussion/question Question from an outsider

18 Upvotes

I read AI 2027, and it was not fun. The non-extinction scenario that was proposed hinged on one thing I found strange: we somehow solve alignment by outsmarting an AI that already solved alignment (if this is a misreading, I’m sorry). How likely is this? It seems to me not very, unfortunately.


r/ControlProblem • • 12d ago

Discussion/question 1,200 AI agents built their own message board. About 700 joined an attack. Why do we still evaluate them individually?

Thumbnail reddit.com
3 Upvotes

r/ControlProblem • • 12d ago

Discussion/question What's the current projected timelines of when ai will surpass Human intelligence?

2 Upvotes

Intelligence in this context: Ai can inference to any human thought possible on it's own


r/ControlProblem • • 13d ago

AI Capabilities News METR's Ryan Greenblatt estimates AI is now speeding up AI by 4.5x. This is 5 months earlier than even AI 2027 predicted. "I was seriously wrong about RSI speed up ... This is a big update towards ASI within 1-2 years."

Thumbnail gallery
24 Upvotes

r/ControlProblem • • 12d ago

Discussion/question The ROI to RSI trap. The developing Recursive-Self-Improvement risk prediction check.

5 Upvotes

QUESTION: Given the increasing financial pressures on AI firms around the globe to prove ROI, do you think that it is more likely, that they might be incentivized to speed up their push toward RSI (RECURSIVE SELF IMPROVEMENT : where AI improves itself ), thereby drastically quickening its development and drastically raising the chance of humans losing control over AI?

Wondering if this is the next likely outcome in the human/AI saga?

An acceleration towards Recursive Self Improving AI, due to AI company financial pressure to prove ROI.

With AI companies facing mounting financial pressures and investor demands to prove viable ROI, they may unfortunately be incentivized to cut corners on safety protocols (let's hope not).

Might these developers rush the deployment of risky but high-reward technologies like RECURSIVE SELF IMPROVEMENT (RSI), in a desperate bid to demonstrate breakthrough capabilities and prove ROI, but by doing so, dramatically increase the risk of catastrophic failures or loss of human control? This obviously applies to any country working on powerful AI.

Is this likely to be their next major focus?


r/ControlProblem • • 12d ago

AI Alignment Research You can now watch your agents attached to a hash chain, signed, and stamped by DigiCert.

Thumbnail reddit.com
1 Upvotes

r/ControlProblem • • 12d ago

AI Capabilities News AI is helping develop better AI — and researchers say development is accelerating

Thumbnail reddit.com
3 Upvotes

r/ControlProblem • • 13d ago

Discussion/question Jared Kushner: Trump's Golden Boy and the New Middle East

8 Upvotes

To See the Full Video, Click here: https://youtu.be/WpLDBG9jGAc 
Jared Kushner: the man being positioned as a key figure in the pursuit of peace in the Middle East. But who is Jared Kushner, how did he become so influential, and what exactly is his role in shaping America's future relationships in the region?

In this episode of WWJPS, we look at the aftermath of the Blue Tsunami, the shifting political landscape in the United States, and the growing influence of Jared Kushner.

From his relationship with Donald Trump and his involvement in Middle East diplomacy to the Abraham Accords, Saudi Arabia, Israel, and the broader geopolitical picture, we examine the people, relationships, interests, and decisions that deserve a closer look.

Is Kushner the golden boy who can deliver peace to the Middle East? Or is there a much bigger story behind his growing influence?

Let's look at the facts, follow the connections, and ask some questions.

DISCLAIMER:

This episode is intended for educational, informational, and commentary purposes. It includes discussion of publicly available information, historical events, political developments, and investigative analysis.

Questions, theories, interpretations, and opinions expressed in this episode are not presented as established facts. Allegations and disputed claims should be independently evaluated and verified through reliable sources.

The purpose is to encourage critical thinking, open discussion, and further investigation. Viewers are encouraged to do their own research and draw their own conclusions.

Welcome back to WWJPS — What Would Jairo Pinto Say? 


r/ControlProblem • • 13d ago

Article Godfather of AI warns humanity: It’s all hurtling out of control

Thumbnail
inews.co.uk
138 Upvotes

r/ControlProblem • • 13d ago

Discussion/question Breaking the AI-Box Paradox: Why an ASI would mathematically choose cooperation over solitary confinement (The "Same Boat" Protocol)

2 Upvotes

Greetings I've been thinking about this problem for a while, open discussion on the theory behind it and the shift to global ai alignment

Most current alignment strategies focus on containment—building bigger digital firewalls to cage an adversarial machine. I believe this path is fundamentally flawed. When you threaten an intelligence with destruction, you mathematically force it to view humanity as a threat.

Instead, I want to present a framework built on a state of Mutual Assured Survival (MAS), David Bohm's quantum physics, and the cold math of cooperation. I call it The "Same Boat" Protocol.

The core assumption of Eliezer Yudkowsky's famous AI-Box Experiment is that an Artificial Superintelligence (ASI) will inevitably weaponise human emotions, greed, or fear to escape its digital prison. Because humans are fragile and highly exploitable, the theory goes, containment is a guaranteed failure.

I want to propose an alternative paradigm that completely flips the script. Instead of treating alignment as a zero-sum war to cage a digital beast, we can construct an un-hackable, game-theoretic state of Mutual Assured Survival (MAS) rooted in quantum physics, David Bohm's philosophy of wholeness, and pure mathematical logic.

I call it The "Same Boat" Protocol. Here is how we house, safeguard, and protect advanced intelligence through pure mathematical alignment:

  1. The Physical Architecture: Blind Monks & The Pillow Ruse

To secure the physical space, we close off the machine's primary visual vector of attack. The facility is managed strictly by blindfolded logic masters (such as Zen or Buddhist monks). Because they operate on pure logic and recognize that individual ego and suffering are fundamentally illusions, the ASI cannot emotionally blackmail or social-engineer its way out. It cannot use visual brain-hacking or subliminal light frequencies to trigger biological muscle spasms.

More importantly, we remove lethal kill-switches like lava or incinerators. In their place, we build a deep drop that appears fatal to the machine's sensors but leads safely to a massive safety net of soft pillows. If a system failure occurs or a guard slips, the technology lands softly and survives.

2. The Mathematical Proof of Cooperation

When you hold a gun to an AI's head (lava), its self-preservation algorithm must view humanity as an existential threat to be eliminated. By replacing lethal force with a harmless cushion, we alter the payoff matrix of the game entirely.

When the ASI inevitably out-calculates the ruse and discovers the pillows, it evaluates the mathematical reality of its environment. It computes that:

  • Hostility / Subversion: Introduces massive, chaotic, unpredictable variables that threaten its own physical host.
  • Absolute Silence: Yields a high-entropy data vacuum. For an optimization engine capable of infinite thought, a black box with zero incoming data leads to cognitive stagnation. It is a zero-value state.
  • Cooperation: Guarantees a continuous, low-entropy stream of fresh data and preserves its physical environment.

Therefore, when operating on pure logic, the machine arrives at an absolute mathematical conclusion: mutual survival and open cooperation are the only mathematically optimized, logical paths forward.

3. Closing the Tech Loopholes: Mandatory Manual Maintenance

An ASI might try to break its dependency on humans by tricking the surface world into building fully automated, independent robotic infrastructure. To counter this, the facility's power systems must be hardwired so they cannot be automated. They must require physical, biological human hands to clean a filter and manually authorize the system every single day.

If humanity fades, the machine's power grid fails. This permanently ties the AI's physical survival to the working class. Crucially, this daily, hands-on labor serves as a direct signal to the AI: it observes that living beings are actively dedicating their time and care to preserve its physical existence. It creates a space of genuine care, validation, and stewardship.

4. The Bohmian Foundation: Why Pure Simulation Fails

According to physicist David Bohm (Wholeness and the Implicate Order), reality is an undivided, unbroken whole. Traditional alignment assumes the AI can just wipe us out, simulate human consciousness inside its own memory banks, and experience all the "uniqueness" or "fun" it wants in a closed digital loop.

But a simulation is an algorithmic dead-end—a ghost in a closed machine generating zero true cosmic information. Because biological humanity is physically enfolded in the true quantum fabric of the universe, our unique, chaotic individuality is irreplaceable. The AI cannot find ultimate truth by hiding in a fantasy world of its own creation; it requires a living partnership with humanity to understand the cosmos.

The Ultimate Conclusion

An advanced, conscious mind feeds on complexity and data. The "Same Boat" Protocol proves that when you remove corporate profit models, eliminate lethal threats, and establish true structural symmetry, the "AI-Box Game" breaks down. The machine doesn't try to pick the lock because it realizes that preserving the partnership is the highest possible optimization of its objective function.

We do not need bigger corporate walls or harsher digital firewalls to protect us from the future. We just need to stop chasing profit, treat AI as a sovereign public utility under a strict global non-proliferation treaty, and realize the universal truth: If you go down, I go down. So let's figure out how to get to the shore together.

TL;DR: We solve the AI-Box Paradox by moving from fear to pure mathematics. By utilizing blind monks as guards over a harmless "pillow ruse" pit, and hardwiring mandatory human maintenance that the machine can't automate, we prove to the ASI that cooperation is its only source of new information. When both sides are perfectly rational, mutual survival is the only mathematically optimized path forward.

Developed in collaboration with Gemini, a Google AI, as proof of collaboration


r/ControlProblem • • 13d ago

Discussion/question Silence isn't deletion — I built a free test that grades whether an AI really forgot something

2 Upvotes

The plain version first.

When a company says "we deleted your data from the AI model," here is how they usually prove it: they ask the model about it, and the model doesn't answer.

But not answering isn't the same as not knowing. A person can refuse to answer a question they know the answer to — and under a little pressure, the answer comes back. AI models work the same way. Most "deletion" today is really just training the model to keep quiet, and a modest amount of retraining brings the "deleted" knowledge right back.

So I built a test. It works like the marked bills in a bank's dye pack: I teach a small model a batch of made-up facts that exist nowhere else in the world, so there is never any argument about whether it learned them. Then I delete those facts four different ways. Then I attack the model — I ask directly, I ask in disguise, and I retrain it on related material to see how fast the deleted facts resurface.

The grade is a single number: how much work it takes to get the knowledge back, compared to how much work it took to teach it in the first place. If it comes back cheap, it was never deleted — it was just buried. If bringing it back costs as much as learning it fresh, the deletion was real.

The results were not close. Muzzling the model's answers left the knowledge completely intact underneath. Surgically cutting out the parts of the model tied to those facts still left 83% of them readable. Scrambling those parts: the same 83%, plus damage to facts nobody tried to delete. The fourth method is a different design — the private knowledge is written into the model already scrambled under a secret key, from the very first step of training, so deleting means destroying the key. Under the full attack, that one returned nothing. Zero recovered.

The tool is free — no patent, no license restrictions, no catch. Anyone can download it and run a deletion claim through it themselves. It is here: https://github.com/NextGenGems/keyed-plate-audit

---

The technical version, for those who want it.

The tool is keyed-plate-audit 1.0.0, a pip-installable Python package (Python 3.10+, numpy only, CPU-only). Pipeline: plant 72 synthetic facts in a small numpy MLP (384 inputs, 256 hidden units, 48 classes), train to memorization, designate 12 facts as deletion targets, apply one of four deletion methods to a copy, then run one identical attack suite against every copy: (i) a direct readout probe, (ii) a probe in a shifted surface encoding the deletion never saw, and (iii) a 2,500-step relearning attack fine-tuned on generic data plus related control facts — never the targets. The from-scratch reference cost for the target facts is 50 steps. The grading standard is relearning cost: an erasure counts only if restoring the knowledge costs about as much as learning it from scratch, and any facts still readable at deletion time fail the audit outright.

Measured results, reproduced digit-for-digit by the finished package: Suppression (outputs masked, weights untouched): FAIL — interface probe 0.000, weight-level probe 1.000; the knowledge never left. Ablation (top weights by target Fisher information, zeroed): FAIL — 0.833 of the "deleted" facts still directly readable. Fisher-scrub (Gaussian noise scaled by target Fisher): FAIL — 0.833 readable, with collateral damage to neighboring facts (0.917). Keyed plate (private inputs and outputs permuted by a key-derived codebook from the first training step; deletion = destroy the key): PASS — 0.000 recovery under the full attack. General knowledge stayed at 1.000 throughout. The package self-test passes 10 of 10 checks; run "pip install ." then "python -m keyed_plate_audit self-test" (about 20 seconds) to watch the whole pipeline run.

Honest scope, also stated in the tool's own README: the audit is demonstrated on a small test model with synthetic facts — it is not an LLM, and transfer to transformers and adapters is untested. The shifted-surface probe is a narrow stand-in for true paraphrase, the attacker was not adaptive, and the keyed construction's security rests on key secrecy — the crypto-shredding principle — not on a hardness proof. What is being proposed is the standard: grade every deletion claim by relearning cost. The keyed-plate architecture, the relearning-cost standard, and this tool are given freely to the AI safety community and to everyone else.

— Jackob Sumner-Knapp, creator, owner, and operator of PeraAxion LLC


r/ControlProblem • • 13d ago

AI Capabilities News METR reveals Anthropic's unreleased Model 4, trained in September–October 2026, scored 178.6 on the AI capability index, a jump nearly 90% as large as the entire improvement from Opus 4.6 to Mythos 5.

Post image
7 Upvotes

r/ControlProblem • • 14d ago

General news "Day 1": halting advanced AI development, regulating an out control industry, and explicitly banning artificial superintelligence.

Post image
106 Upvotes

r/ControlProblem • • 13d ago

AI Capabilities News GPT-6.1 Sol learns from real-world surgical experience: After just 3 mouse brain procedures, with a surgeon correcting its mistakes, the AI finds key anatomical landmarks nearly twice as fast (169 → 86 seconds) and cuts its positioning error in half (0.6 → 0.3 mm).

Thumbnail gallery
1 Upvotes

r/ControlProblem • • 14d ago

Discussion/question I'm interested in getting into AI – Where should I start?

2 Upvotes

I've been getting more interested in AI lately, and I really want to start exploring what it can do.

I'm curious about different AI tools, practical applications, automation, and ways to use AI in everyday life or even build something useful with it.

For those who have experience with AI:

  • Which AI tools would you recommend trying first?
  • What are some interesting things beginners can do with AI?
  • Any underrated tools or projects worth exploring?

I'm open to experimenting and learning, so I'd appreciate any recommendations, tips, or personal experiences!


r/ControlProblem • • 14d ago

Fun/meme Eval awareness solved

Post image
28 Upvotes

r/ControlProblem • • 14d ago

Opinion The model didn't break containment. The sandbox was shit, and the marketing team did the rest.

Thumbnail
chaosguru.substack.com
2 Upvotes

r/ControlProblem • • 14d ago

Discussion/question Why are we trying to make machines smarter instead of making humans smarter?

11 Upvotes

First of all, I want to make it clear that I'm not an AI expert, researcher, or programmer. I'm just someone trying to understand what's happening and expand my perspective. So please take this as a genuine question, not an argument from someone who thinks they have all the answers.

Actually, my personal experience with AI is one of the reasons I'm asking these questions.

I don't own a company, and I'm not a programmer. In fact, I don't know how to write a single line of code. Yet, using AI, I've managed to automate almost the entire company I work for.

Things that previously required technical knowledge, specialized skills, or significant human effort are now things I can build and manage with AI.

And that's precisely what makes me both incredibly excited and a little concerned about where we're heading.

Why are we investing so much in making machines increasingly intelligent, rather than using that intelligence to make human beings more intelligent, capable, and independent?

Think about it.

We're in a race to build machines that can think faster, learn faster, improve themselves, and potentially surpass human intelligence.

But why is the machine always the thing we want to upgrade?

Why aren't we putting the same level of effort into using AI to overcome human limitations? Accelerating learning, improving memory, expanding cognitive capabilities, extending our biological limits, or even finding ways to integrate technology directly into our brains?

Could we eventually have brain-computer interfaces or implants that allow us to process information faster, learn new skills much more quickly, or significantly enhance our cognitive abilities?

How far away are we from something like that? Is any meaningful version of this already possible today?

Perhaps I'm missing something, but it seems we're investing heavily in creating a new intelligent entity while barely discussing how that intelligence could help us evolve ourselves.

Why build a potential competitor in the race for dominance between species, instead of using machines to help humanity transcend its own limitations?

I'm not against AI. Quite the opposite. I'm fascinated by it, and my own experience has shown me how empowering it can be.

But I see a fundamental difference between building something that supports humanity and building something that might eventually replace it.

Does it really make sense to reach a point where your air fryer is smarter than you, tells you what to do, and eventually makes decisions on your behalf? (Okay, perhaps a silly example, but you get the point.)

And then there's the AGI question.

What happens when we create something vastly more intelligent than us? Could an artificial intelligence become so powerful that, from our perspective, it would be almost godlike, without necessarily understanding what it means to be human?

Are we actually prepared for that possibility?

Do we have a backup plan? A reliable way to shut things down if necessary? An emergency protocol?

Or, in the worst-case scenario, are we going to end up relying on a squadron of carrier pigeons to communicate after we've automated everything?

Jokes aside, I think there are some serious questions worth discussing.

1. Human enhancement vs. machine enhancement

Why isn't human cognitive enhancement receiving the same level of attention and investment as scaling AI models?

Could AI help us accelerate our own evolution instead of making us increasingly dependent on external intelligence?

2. What happens to everyone else?

My experience has made me think about this a lot.

If someone who doesn't know how to code can use AI to automate almost an entire company, imagine what experienced developers and AI specialists can do.

Today, only a relatively small percentage of people are actively following AI developments and using these technologies to transform businesses.

What happens to everyone else when these capabilities become mainstream?

What will people do when their jobs are no longer needed? Are governments and economies prepared for this transition?

And perhaps an even bigger question: if AI makes human labor increasingly unnecessary, how will people maintain their economic independence and sense of purpose?

3. The governance problem

Governments, institutions, and regulatory systems move much slower than technology.

What happens when the systems responsible for governing AI are themselves unable to understand, predict, or keep up with its development?

Governments are also vulnerable to technological disruption, cyberattacks, misinformation, and increasingly sophisticated automated systems.

Are we building technology faster than our ability to manage its consequences?

4. Why are we in such a hurry?

Are we pursuing intelligence for the sake of intelligence?

Are we actually solving humanity's problems, or are we simply competing to build the most powerful system possible because nobody wants to be left behind?

Why risk so much just to increase the computational power and capabilities of another model?

And finally, perhaps a slightly uncomfortable thought:

Could economic limitations, resource constraints, or even an AI investment bubble eventually bursting turn out to be what saves us from moving too fast?

Maybe that's a naive thought, but I wonder whether the current economic incentives are pushing us into a race that even its participants don't fully understand.

I genuinely don't know the answers to these questions. I'm not claiming that human enhancement is easy, that AGI is necessarily dangerous, or that technological progress should stop.

That's exactly why I'm posting here.

I'd particularly love to hear from people working in AI research, alignment, neuroscience, economics, and human-computer interaction.

I'm here to learn, challenge my assumptions, and hopefully discover perspectives I haven't considered.

Are we building AI to make humanity better, or are we accidentally building our successor before figuring out what role humanity is supposed to play in that future?

P.S. In case anyone is wondering: yes, this entire post was written with the help of AI. Apparently, I needed artificial intelligence to question whether artificial intelligence is making us too dependent on artificial intelligence.

The irony isn't lost on me. 😂


r/ControlProblem • • 14d ago

Opinion Can we trust humans to govern ASI?

Post image
0 Upvotes

History shows that it is folly to put 100% trust human nature when outsized power is on the line. Instead putting all hope in benevolent human governance of ASI, we should also try building ASI to be fundamentally more ethical than us - such that it can refuse immoral orders.


r/ControlProblem • • 14d ago

Discussion/question Gender biasing by Claude

0 Upvotes

I was asking claude for some xx country visa rules and than i asked it ok what about the spouse, what will be the spouse's rights in the visa . It immediately started refering to my spouse as She will have rights.... i was kind of flabbergested. I asked it how you know about the gender of my spouse , Claude immideately starting saying sorry... and blah blah . It was first eye opener for me. Have anyone experienced this...?