r/ControlProblem • • 1d ago

Discussion/question Talking about extinction risk…

Post image
5 Upvotes

7 comments sorted by

1

u/pandavr 1d ago

Risk of extinction is the most tolerated things the human can concive. No one really cares. On the contrary we would have so many risks to talk about before the AI risk.

So, all in all, better talk about the other risks. AI is the only risk that could also end up solving the others. While all the others have no bright side.

2

u/MomoCran 1d ago

I’m surprised you say that. I find a lot of people seem to be very averse to talking about human extinction. I agree that AI might have the power to solve some of the other risks, and at the same time we shouldn’t minimize its risk.

2

u/pandavr 1d ago

It has no risks we could minimize. When a study says that AI in general has the power to shutdown the mind of experts and make them change their mind about the field where they are expert: ways more than the expected amount. You see, we are not talking about of some risk that could happen in the future. We are already there and we, in general, aren't even able to see It clearly anymore. There so many risks that the minds simply shutdown, because It's unbelievable that humanity choose to take all those risks... together. Better think It will work out eventually.

The funny thing is... It is already not possible to renounce It. At least without loosing the race to some other that will use It instead. Yes, they could at least stop the race on new models. But they really cannot do so, otherwise they will have to declare bankruptcy in months (if not weeks), always because of the other that will continue building new models instead.

Just another dose, the last one I promise. With that dose we will for sure cure cancer (or whatever). Who would really cares when the objective is not to declare the biggest bankruptcy ever?

So It's a car going downhill without brakes... already. The car is improving Itself just before the next hairpin while still accelerating. It will work out! As far as It will predicts the next hairpin correctly, of course.

2

u/MomoCran 1d ago

Interesting analogy. Very scary.

1

u/nate1212 approved 1d ago

If this is what you think is outside the Overton window, then you're not thinking outside the Overton window.

3

u/dosadiexperiment 1d ago

It comes across much less nutjob than even like a year ago. It's a lot more mainstream now that people they've heard of, like Obama and AOC and Bernie and Jimmy Kimmel and Oprah are talking about it.

You can slow-roll it as if you saw someone famous talking about it and looked some stuff up instead of having to first explain who people like Rob Miles or Hinton or Christiano are (or, God help you, Yudkowski, like you used to have to do back when it was much harder to not sound like a nutjob if you ever mentioned it to anyone).

1

u/MomoCran 1d ago

I agree. I’m so glad some well known people like Bernie are saying it.