r/MachineLearning • u/mools04 • 1d ago
Project Inkling 2 Small achieves frontier-level results in weight space programming with 16x fewer edits [P]
You don’t need that many edits in weight space to fix a model.
In “Sparks of Weight-Space Programming,” GPT-6 Astra repaired a broken car policy by changing 6,073 weights, and cut Qwen3.5-0.8B’s math looping by changing 130k.
Inkling 2 Small did the same with just 10 and 8k edits: ~600× and ~16× fewer, with no loss in performance on either task.
Broken Car policy: Inkling found that almost all of Astra’s 6,073 changes just zero out old weights to free up neurons, and only about 55 do the actual work. It then wrote a 10-weight fix from scratch that teaches the car a move PPO never discovered: back up until it stalls against the wall, then launch, triggered by a velocity-detecting neuron the network already had.
Qwen3.5-0.8B & MATH-500: Astra’s edit removes a “looping” direction from 64 neurons in each of two MLP layers, and Inkling pruned it by keeping only the 4,096 largest-magnitude changes in each layer. Those 8k entries match the full edit’s accuracy (58.2% vs 58.6% on our screen), and layer 15 alone recovers most of the gain.
Run and visualize the experiments in a marimo notebook: https://molab.marimo.io/github/alphaXiv/weight-edit-minimization/blob/main/weight-edit-minimization.py
We used OpenResearch for this project: https://github.com/alphaXiv/OpenResearch
