Xietao Wang-Lin, Anton Isopoussu, Louis Mahon
In iterative blind LLM bug-fixing, the rate at which correct code is damaged (beta) matches or exceeds the rate at which buggy code is repaired (alpha), and the loop settles into cycles where the same changes are added and removed indefinitely.
Iterative bug-fixing loops run on 20 random CodeContests+ problems with 40 C++ submissions each (~800 submissions, median 38-42 LOC), over up to 100 turns, with two models (Gemini 2.5 Flash-Lite, Qwen2.5-7B-Instruct), two edit styles (search/replace blocks vs whole-file), greedy and temperature 0.7 decoding; plus a difference-of-means steering vector probed in Qwen2.5-7B activations with 5-fold cross-validated AUC.
Damage rates are large: e.g. Gemini with SRBs, correct code, tau=0: beta=0.293 vs alpha=0.062 on incorrect code; at tau=0.7 beta=0.216 vs alpha=0.007. For whole-file edits the gap narrows (beta=0.165 vs alpha=0.040). Steering-vector AUC reaches ~0.7-0.8 mid-layers, near 1.0 at layer 19; negative steering (gamma=-0.5) halts editing entirely. Scope: tiny competitive-programming files (5-189 LOC, single file, no tests given to the model), only 2 models, so results do not directly transfer to large real-world codebases.
Only 20 problems and 2 models, including one small open-weight 7B; code is short competitive-programming submissions, not production code; the agent sees code but not test results or history, an artificial 'blind' setting the authors themselves motivate but which many real harnesses avoid; no baseline of human developers; mechanistic claims rest on one model and one steering method.
A skeptic would say the setup manufactures the result: telling a model to find a bug in code with no test feedback guarantees overeager pseudo-fixing, and real agentic repair tools run test suites as stopping conditions. Small files with single planted-or-submitted bugs are also the easy case for whole-file rewriting, inflating the SRB-vs-whole-file contrast. It is also a workshop-style report from a company lab with no external benchmarking.
Little or nothing yet for near-term AI economics, but it points at a concrete cost of unsupervised agentic coding loops: without grounded stopping conditions, agents spend tokens making code worse, which erodes the assumed ROI of autonomous repair agents. Relevant mainly as a caution against extrapolating agent reliability gains into unattended enterprise deployment.
Technical Capability: Shows that agentic loops without objective stopping conditions do not merely plateau but actively degrade correct outputs, and that an internal 'buggy code' direction causally controls editing propensity.
Labor & Knowledge Work: Weakly relevant: suggests that blindly trusting agentic repair pipelines could lower code quality, a consideration for teams deciding how much reviewer oversight to keep.