An AI agent opened up another neural network and changed its weights by hand, like a programmer patching code, and the small model got much better at math. That is the headline claim of “weight-space programming”, a research post from MIT CSAIL that has been one of the most-discussed AI stories on X this week. The experiment is real and interesting. It is also easy to read more into it than the authors and their critics say the data supports.<\/p>\n\n\n\n
The weight-space programming post<\/a> by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab describes a series of experiments in which Astra, a GPT-6-based agent run via the Codex CLI, was handed model weights and asked to change behavior by editing them. In the experiment that went viral, the agent targeted the MLP The authors report that, with a budget of 50 forward passes, MATH-500 accuracy went from 7.9 to 48.1, GSM8K from 13.8 to 67.1 and IFEval from 21.0 to 62.3. The share of answers that hit the token limit dropped from 93% to 33.1%. The same approach did not work on Gemma 3 270M, according to the post.<\/p>\n\n\n\n The story spread fast. Japanese outlet GIGAZINE covered it, and an X user pointed to a Hacker News thread with roughly 1,974 points. Lead author Rikiya Takehi wrote on X that he was “overwhelmed by the amount of attention this got.”<\/p>\n\n\n\n Language models are usually changed by training: fine-tuning, reinforcement learning or techniques like LoRA adjust millions of parameters through gradient descent. Interpretability research such as activation steering and model editing (for example ROME-style factual edits) has shown that targeted, small interventions can shift behavior. Weight-space programming pushes that idea toward automation: instead of a researcher designing the edit, a general coding agent explores the weights, forms hypotheses and writes the change.<\/p>\n\n\n\n Omar Khattab, the senior author, is known for DSPy and work on optimizing LLM programs through text and prompts. In posts on X he framed the work as a fresh angle on the long-running debate about optimizing models through text versus through weights.<\/p>\n\n\n\n According to the blog post, Astra was given access to the weights and a small evaluation loop. In the earlier experiments on tiny networks (17,000 to 172,000 weights), the agent read raw weight values to infer and modify what the network did. For Qwen, the agent wrote Python to analyze activations and weights, proposed a targeted edit to two MLP layers, and checked candidates with up to 50 forward passes before settling on the final change. No gradient descent was used for the final edit.<\/p>\n\n\n\n A Hugging Face reproduction, JonathanColetti\/Qwen3.5-0.8B-Astra-WeightEdit<\/a>, published the edited model and adds context: with the model’s thinking mode turned off, the base Qwen model already scores 45.8 on MATH-500, and the edit costs about 1.5 to 2 points in that setting. That suggests much of the jump comes from stopping the model from rambling past its token limit in thinking mode, not from new math ability.<\/p>\n\n\n\ndown_proj<\/code> matrices in layers 15 and 19 of Qwen3.5-0.8B and changed 131,072 values.<\/p>\n\n\n\nBackground: how models are normally changed<\/h2>\n\n\n\n
How it works<\/h2>\n\n\n\n