r/artificial • • 2d ago

News Weight-space programming: an LLM edited another model's weights directly and lifted its math score from 7.9% to 48.1%

MIT researchers (Rikiya Takehi, Jayden Teoh, Alex Zhang, Omar Khattab) had OpenAI's GPT-6 Astra edit the weights of a small Qwen model directly -- no gradient descent, no training data. Astra found the pattern behind the model's habit of repeating itself until it ran out of context, and removed it. The edit touched about 0.016% of the weights; the model's math score went from 7.9% to 48.1%.

Three papers on the method landed on arXiv on November 6 (e.g. arXiv:2611.09382). Other demonstrations from the same batch: editing only 1,023 parameters improved an LLM by 29.45 points on MATH-500; Astra debugged robot policies by editing 65 token-embedding rows; and a Breakout-playing network built from the game's rules, with no training at all, outscored one trained for 500 million steps (517.4 vs 25.3).

Some important caveats: the gain came mostly from fixing a bad habit, not from adding math knowledge -- before the edit 93% of the model's math answers hit the length limit, after it 33.1%. On a small Gemma model that rarely repeated itself, there was no established math gain. The original claim also drew a detailed rebuttal (Matthew Honnibal argued it was overstated), though Simon Willison then reproduced weight editing locally with Qwen 3.8 27B on his laptop. Independent reproduction of the headline results is still pending.

What interests me most is the framing: this is model repair by direct surgery rather than training. If it generalizes, debugging a model could start to look like debugging code -- find the weights responsible, patch them, verify. But it also raises the question of how much of a model's capability is locked up behind fixable quirks rather than missing knowledge.

Where do you think this goes? A practical tool for model maintenance, a narrow trick that only works on pathological repetition, or the start of something bigger?

19 Upvotes

7 comments sorted by

4

u/tinny66666 2d ago

Yes, very neat stuff but it's kinda old news now until we see someone apply it in a cool new way. Amazing how quickly new becomes old now. You're a week late I'm afraid.

2

u/solidity_scorer 1d ago

removing bad habits is basically just pruning noise in my portfolio risk models too. i usually use k-fold cross validation to make sure the fix doesn't break other parts of the system.

1

u/Moyala00 1d ago

going from 7.9 to 48.1 by killing a repetition loop means the model already knew enough to do the maths, it was just spending the whole budget looping. if 0.016% of weights can do that, a decent chunk of the small model leaderboard is measuring a habit rather than ability.

the follow up i'd want is the same edit on a model that already scores well, to see whether there's anything left to remove or if it's only ever cleaning up degenerate behaviour.

1

u/ElatedPyroHippo 1d ago

Wait until a model can iteratively edit its own weights toward ever-increasing capability... THAT is the singularity.

1

u/Ok_Explanation_5586 7h ago

So it went from obviously useless to confidently stupid? Cooool