EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnesses

2608.28363 ↗ · 2026-08-28 · cs.AI

Tanmay Sah, Dolly Sah, Harshul Jain, Tanya Sah

The finding

Headline claim

Agent self-modifications that improve capability are frequently unrecoverable under conventional repair, but pairing witness capture with counterfactual round-trip verification and a sufficiently expressive recovery language makes most of them safely reversible.

The test

600 one-shot self-evolution tasks across six harness-state families using gpt-oss-120b, yielding 197 frozen capability-positive recovery failures, plus a 120-case controlled defect benchmark, a 2x2 grounding-by-expressivity factorial with 10 dev and 40 hidden counterfactual states per candidate, and a 300-task fresh holdout with 100 hidden states; partial replication with Qwen3.8-27B.

Effect size & scope

Conventional verifier-guided repair recovered 0/197 natural failures; a deterministic oracle showed 48/197 recoverable under base language L0 and 191/197 under extended L1. Exact state-address grounding lifted recovery from 0/48 to 38/48 (79.2%) on the L0-sufficient stratum; L1 enabled 142/143 (99.3%) on the S1 stratum; exact-address diagnostics on top of L1 reduced recovery to 133/143 (93.0%) on gpt-oss-120b but not on Qwen3.8-27B. Controlled benchmark: 114/120 (95.0%) repair at budget four vs 4/120 regeneration. Results hold only within the modeled harness-state surfaces and defined recovery languages; six mutation families, one-shot sampling without rerolls.

Limitations

Authors concede the recovery languages are implemented operational semantics, not completeness characterizations; joint optimization of forward mutation with recovery was not evaluated; the diagnostic-granularity interaction is model-dependent and unexplained; modeled state targets are an evaluation grouping, not exhaustive of real harness state. Unstated: all authors appear to be independent researchers (four sharing a surname), the benchmark and oracle are self-constructed by the same authors, no external baselines beyond snapshot-restore, single primary backbone, and tasks are synthetic harness mutations rather than real agent deployments.

Counterpoint

The paper's central failure bank is generated by the authors' own framework, and the oracle that defines 'recoverable' is also theirs, so the 0/197 headline may reflect a deliberately narrow recovery representation designed to be beaten by the authors' extended calculus. Six families of synthetic mutations on one lab harness say little about production agent failures; the 79.2% and 93.0% factorial cells rest on small strata (48 and 143), and there is no evidence any real deployed self-evolving agent produces these failure modes at meaningful rates.

Economy implication

Little or nothing yet for AI economics in the next 12 to 24 months. At most it is an early signal that safe agent self-modification requires verification infrastructure, which could eventually become a cost and moat component for agent platforms, but no production or market evidence is presented.

Who this matters for

Technical Capability: Agent builders get a concrete decomposition showing unrecoverable self-modifications split into a state-grounding problem and a recovery-language expressivity problem, both fixable but with model-specific interactions.

Bears on (advisory · papers never write evidence)

[1.3] Agentic reliability (multi-step completion without human correction) is improving materially, not just peak single-shot.

Abstract

LLM agents increasingly modify their own prompts, tools, middleware, resources, and execution harnesses at runtime. Such self-evolution can improve capability, but a successful mutation may leave persistent effects that cannot be safely reversed in states different from the one in which it was created. We introduce EvoUndo, a framework for representing, synthesizing, diagnosing, and independently verifying recoverability of model-generated self-modifications across counterfactual states. Across 600 unseen one-shot self-evolution tasks, we identify 197 capability-improving mutations that fail recoverability verification. Under the original recovery representation, conventional repair strategies recover 0/197 of these natural failures. Deterministic oracle analysis recovers 48/197 under the original recovery language L0, while the extended recovery calculus increases empirical oracle recovery to 191/197. A protocol-locked 2x2 grounding-by-expressivity intervention then separates two bottlenecks: exact state-address grounding increases successful recovery from 0/48 to 38/48 (79.2%) when the original language is sufficient, while extending the recovery language enables recovery on 142/143 (99.3%) failures in the oracle-defined S1 stratum. On the primary gpt-oss-120b backbone, adding exact-address diagnostics to the richer language reduces recovery to 133/143 (93.0%); a Qwen3.8-27B replication preserves the grounding and expressivity effects but not this negative interaction, indicating that the latter is model-dependent. These results indicate that reliable agent self-evolution requires co-designing verification, state grounding, witness semantics, and recovery-language expressivity rather than relying on iterative prompting alone.