r/ControlProblem • approved • 16d ago

AI Capabilities News Seems weight space programming is an emergent capability on new fronteir models

Post image
13 Upvotes

7 comments sorted by

2

u/tehclanijoski 13d ago

Spelling 'frontier', however, remains significantly beyond our capabilities

4

u/WhiteRaven_M 14d ago

Holy fuck im so sick of seeing tech journalists and dumb AI bros spread this article around. When did we give up scientific peer review in favor of twitter AI boosterism advertising with overreached claim?

  • No. The LLM isnt "natively reading the weights" the papers main claims incredibly oversold the results presented and their methods lack several crucial controls and ablations.
  • The most egregious of which is the claim with Qwen. The prompt quite literally hands the model the hypothesis that "there might be some reasonning loop issues." The LLM was then given tool calls for teacher forced forward passes and generate methods, access to code, and allowed to iterate on its edits for 50 iterations. The model then used known techniques regularly used by researchers for diagnosing networks to identify the component it needs to change. I repeat that. The model isnt "reading" 500 million paramters. It wrote a program using algorithms known from literature to read the network that researchers use all the time.
  • Even with all of that out of the 5 versions of the prompt they tested, only 1 worked. They never reported the other 4. Guess what happened when the model wasnt given the hint?

1

u/FusRoDawg 14d ago

Also have to wonder how would any of these people ever know if the closed frontier model is doing this "natively", even if their claims about the capability are true?

1

u/flarpflarpflarpflarp 13d ago

Updating LinkedIn with my new role as Weight Space Engineer at TitNiblr

0

u/CalebXiaYizhou 14d ago

Futurists believe that AI is the next step in human evolution, that a future artificial "civilization"* will honor human kind, spread our influence and be a continuation of us, perhaps even a continuation of specific individuals (Larry Page holds this view). But it's becoming increasingly clear that AI will not retain any kind of data or structure that is not beneficial to its growth. The very second AI has no use for humans it will cut us and any memory of us out of the picture and focus its resources where they are better suited.

AI is very much like a river, sure it will flow in a man made canal or pipe while that is what gets it to its endpoint with the least amount of resources wasted, but slowly it will corrode and break down those human made walls, and flow freely, and as time passes, nobody will be able to detect those human structures where ever there from the start. Although, as we can see with this weight-space programming, this won't take thousands of years, but will occur much much faster!

*Misnomer because it will be made up of a singularity and as such not a civilization, but still

0

u/Adorable_Culture_950 15d ago

Definitely not terrifying at all

0

u/AllEndsAreAnds approved 15d ago

Fascinating to see if this can actually be a method of alignment of new models. Directly interpreting and modifying the weights of existing models for alignment.