An AI Edited a Small Model’s Weights and Its Math Score Rose From 7.9% to 48.1%. The Released Logs Show the Gain Came From One Run in Seven, and Mostly From Answers That Stopped Looping

Asked to convert the point (0, 3) to polar coordinates, a small language model called Qwen3.5-0.8B starts sensibly, recalls the formulas, and then writes “Wait, tan(π/2) is ∞.” It writes that line more than 1,500 times, until it runs out of its 32,768 tokens without giving an answer. The transcript is one of the evaluation outputs released with “Sparks of Weight-Space Programming”, a post published on 2 November by four researchers at MIT CSAIL: Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. Their claim is that frontier language models “can natively read & write neural network weights”. The most quoted result is that one of them, OpenAI’s GPT-6 Astra, edited the weights of this small model and its score on MATH-500, a set of 500 competition-style problems, rose from 7.9% to 48.1%.

Astra did not train anything. According to the authors, it worked on a computer without internet, with a Python sandbox to read and change the weights, and with gradient descent forbidden. It could not run the small model itself: it had a budget of “passes”, each one a batch of answers to practice problems or a read-out of the model’s internal states, and it never saw the test sets. The authors ran it with budgets of 10, 25 and 50 passes. In the 50-pass run, which took 158 turns and about 20 hours, Astra found that the model looped on most problems (85 of its first 96 practice answers hit the token limit), traced the looping to one direction that the MLPs of two internal layers write into the model’s residual stream, and subtracted it from 64 columns in each: 131,072 weights, which the authors put at 0.016% of the model. The share of MATH-500 answers cut off at the limit fell from 93.0% to 33.1%. On GSM8K, school-level word problems, the score went from 13.8% to 67.1%; on IFEval, an instruction-following test, from 21.0% to 62.3%.

The authors do not present this as a model that got better at math. “Fixing loops, however, is not the same as making a model better at math,” they write: “it lets the model finish answers it could already work out.” To see how far that holds, we used the full answer texts in their release, which in each evaluation give the first of the four sampled answers to each of the first 200 of the 500 MATH-500 problems, and counted an answer as finished when its reasoning section closes. In the unedited model, 13 of the 200 finished, and all 13 were correct; of the 187 that did not finish, 2 were graded correct anyway. After the 50-pass edit, 152 finished and 92 of them were correct; none of the 48 unfinished ones were. On the 13 problems the original model had finished, the edited one was right 11 times. Astra’s own final report, written before any test was run, says much the same: on 1,152 practice problems, 576 of its 579 gains came from answers the original model had cut off, and accuracy on answers it had finished was “unchanged”.

The release also contains something the post does not show: the experimenters’ results table for this model, published as a verbatim excerpt of a private file. It lists seven completed runs. Three ended without any edit: one allowed no passes, one allowed 5, and one allowed 10 with a shorter version of the instructions. Three runs with 10 or 25 passes reached between 12.3% and 14.1% on MATH-500. The 48.1% comes from the only run with 50 passes, and it was not repeated; four more runs with a newer version of the instructions are listed without results. The post reports the 10-, 25- and 50-pass runs and does not mention the attempts that produced no edit; it is the logs’ README that says only the runs the post reports are included and “trial runs of earlier prompt versions” are left out.

The model is also a particular case. Qwen’s model card says Qwen3.5-0.8B runs without its thinking mode by default, and warns that in thinking mode it “is more prone to entering thinking loops compared to other Qwen3.5 models, which may prevent it from terminating generation properly.” The authors cite that warning and chose thinking mode. They sampled at temperature 0.6, while the card recommends 1.0 for text in thinking mode; whether the original model would loop as often at that setting is not in the release, and we did not test it. When the authors repeated the experiment on Gemma 3 270M, a model that rarely loops (8.5% of its answers hit the limit), none of Astra’s edits moved MATH-500 by more than about a point, a gain Astra itself reported as not established. The authors’ conclusion is that editing “could remove an obstacle that hid an ability, but it could not add an ability the model lacked.”

The other experiments in the post are smaller, and in most of them Astra worked without Python. By the authors’ account, it named the game each of four networks of about 17,000 weights had been trained to play, from the weights alone and without being given a list of games to choose from, and got all four right; it rewrote a small share of a failing driving network so that the car succeeded in all 1,024 test episodes; and it wrote small game-playing networks from scratch. It failed in four of four attempts to tell from weights alone which of two language models had been made sycophantic. Sean Goedecke replied on his blog that “a smart human ML researcher could perform all of these tasks”: the game networks’ inputs are labelled, and “three lines of Python” show which part of the screen each one watches. For the math edit, he writes, Astra followed a known recipe, comparing looping and non-looping texts and trimming the weights behind the difference, which “is exactly what a competent human researcher could have and would have done.” In that experiment Astra had Python and 50 batches of the model’s answers.

What we checked is narrow. We read the released results, transcripts and edit description; we did not run the model or apply the edit. The finished-or-not count uses only the first sampled answer to each of the first 200 problems in each evaluation, because the release does not include every answer’s length. The results table is an excerpt chosen by the authors, and the trial runs it mentions are not in the logs. The post is not peer-reviewed. The authors say they audited all 1,125 turns in the release to confirm that Astra used none of the built-in tools of Codex CLI, the OpenAI program they ran it through; we did not repeat that audit.