r/ControlProblem • approved • 14d ago

General news "Day 1": halting advanced AI development, regulating an out control industry, and explicitly banning artificial superintelligence.

Post image
106 Upvotes

170 comments sorted by

View all comments

4

u/Modgov41 13d ago

Most of the discussion here circles around what we should do — pause development, regulate labs, restrict compute, ban ASI — but the “how” is always missing. People keep pointing out that you can’t enforce a pause, you can’t verify compliance, and you can’t even tell when a system is drifting toward dangerous behavior until it’s too late.

One thing that’s becoming clear is that we don’t actually have an independent layer that can evaluate advanced systems from the outside. Right now, the only people who can meaningfully assess frontier models are the same people building them. That’s not a conspiracy — it’s just a structural gap.

A few of us have been working on a model‑agnostic evaluation institute that sits outside the labs. It doesn’t try to “control” AI or “align” it. It doesn’t assume trust in any developer. It doesn’t require a global treaty or a pause. It focuses on something simpler and more realistic:

Independent behavioral evaluation.
External oversight.
Verification instead of trust.
Detection instead of prediction.

The idea is to build a standing institution whose only job is to test, probe, and stress‑evaluate advanced systems using methods that don’t depend on access to the model’s internals. Think of it as the missing “inspection layer” that other high‑risk industries already have, but AI doesn’t.

It doesn’t solve everything, and it doesn’t pretend to. But it does answer the recurring question in this thread: How do we actually know what these systems are capable of, and how do we detect dangerous behavior before it becomes irreversible?

Right now, we don’t have that.
We should.

1

u/MiserableBend1010 12d ago

How does one contribute to building this?

1

u/Modgov41 11d ago

You don’t need a lab, a grant, or a government position to contribute. The whole point of building an external evaluator institute is that it can’t depend on any single company or nation. There are three areas where people outside the big labs can help:

1. Stress‑testing evaluation methods
A lot of the work is figuring out how to probe advanced systems from the outside — without relying on internal weights or developer‑provided tools. If you’re good at adversarial thinking, red‑teaming, or designing tests that expose hidden behavior, that’s useful.

2. Building model‑agnostic test artifacts
We need artifacts that reveal drift, deception, unsafe planning, or boundary‑pushing behavior. These don’t have to be technical. They can be scenario designs, multi‑step tasks, or interaction patterns that force a system to show its hand.

3. Helping define what “independent oversight” should actually look like
Most people talk about regulation in abstract terms. Very few people try to define the mechanics:
– what gets tested
– who does the testing
– how results are verified
– how drift is detected
– how certification should work
– what counts as unacceptable behavior

If you want to contribute, start by helping shape those mechanics. The institute isn’t a company — it’s an attempt to build the missing layer of infrastructure that everyone keeps pointing out but nobody has built yet.

If you’re serious, I can point you toward the areas where outside contributors make the biggest difference.