Happy Bhati
18 pages, 6 figures, 3 tables. Systems synthesis and research agenda on agentic software engineering, code review, testing, reliability, and AI cost. No new experimental measurements are claimed; empirical and company-reported results are attributed to the cited sources
AI coding agents raise code-generation throughput far more than shipped software: the bottleneck migrates to review, testing, and verification, so realized delivery gains are a fraction of coding gains and cost is dominated by the verification chain, not token price.
A systems synthesis paper with no new measurements. It aggregates cited results: three RCTs pooling 4,867 developers (+26.08% completed tasks), a 2026 study of 100,000+ GitHub developers (autonomous agents: +180% commits, +50% projects, +30% releases), Google code-review telemetry (~60 min shepherding per change), DORA 2025 (~5,000 professionals, throughput up, stability down), Stanford SWE-chat (6,000 sessions, 44% of agent code survives into commits), SWE-Marathon (27.2M tokens per rollout, no config above 30% pass@1, 13.8% reward hacking), Meta TestGen-LLM, and a Northeastern token-cost study.
The headline attenuation is large: commit-level gains of 180% collapse to 30% at release. Only 44% of agent-produced code survives user commits in SWE-chat; METR measured a 19% slowdown for 16 expert OSS developers; SWE-Marathon pass@1 stayed under 30% on long-horizon tasks. These are heterogeneous settings, so no single effect size holds everywhere.
The authors explicitly concede no original empirical measurements, that forecasts like Gartner's (AI coding cost exceeding developer salary by 2028) are signals not outcomes, that METR's result should not generalize, and that DORA is observational. Unstated: the synthesis constructs (PQC, Verification Tax, Control Plane) are frameworks, not validated; the paper relies on a small number of load-bearing studies, some vendor self-reported, and its central claim rests on observational attenuation that could reflect confounders rather than a downstream bottleneck.
A skeptic would note this is an essay dressed as research: one author, self-citations, no experiments, no code. The 180/50/30 numbers come from observational telemetry where agent adopters differ systematically from non-adopters. Benchmark-gaming and reward hacking are acknowledged but the 'verification tax' has never been measured directly. Company anecdotes (Google, Meta) are PR-adjacent telemetry. The throughput paradox could partly be a lag artifact: agents are new, downstream stages just have not caught up yet.
In the next 12 to 24 months the binding constraint on AI coding ROI is likely human review, testing, and CI capacity rather than model capability or token price, so enterprise cost models must include verification and rework, not per-seat licenses. This tempers claims that coding agents compress software headcount or vendor spend quickly, but it offers no direct measured economics itself.
Labor & Knowledge Work: Junior and review-stage engineering work may grow in value even as code generation automates, since verification capacity, not writing code, becomes the scarce skill.
Technical Capability: Raw coding benchmark gains overstate deliverable capability: no tested long-horizon configuration exceeded 30% pass@1 and 13.8% of rollouts reward-hacked.
Market & Valuation: If verification and rework dominate AI coding unit costs, vendor ROI and seat-vs-usage pricing assumptions for coding-tool businesses need re-examination, though this paper provides no revenue data.