Thinking with Looped Flows

2609.11801 ↗ · 2026-09-10 · cs.LG, cs.AI

Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom, Nicholas M. Boffi, İsmail İlkan Ceylan, Jinwoo Kim

The finding

Headline claim

Training a looped recurrent reasoner with locally supervised denoising objectives (looped flows) beats prior state-of-the-art looped models on most reasoning benchmarks, lifting ARC-AGI-1 from 44.6% to 58.8% and ARC-AGI-2 from 7.8% to 12.2% with the same architecture, and improves performance as more inference-time computation is spent.

The test

Six reasoning benchmarks including two multi-solution ones, comparing against prior looped models (notably TRM) under a matched architecture; ablations isolate contributions of the flow formulation, the recurrence, and temporal alignment (shared noise, decreasing noise levels); ARC-AGI figures are the headline comparisons with the same architecture as TRM.

Effect size & scope

14.2 percentage points on ARC-AGI-1 (44.6 to 58.8) and 4.4 points on ARC-AGI-2 (7.8 to 12.2), plus better performance with finer temporal grids and multiple valid solutions from different noise samples. Scope conditions: small, puzzle-like, supervised benchmarks with known answer formats, not language-model reasoning tasks; wins on five of six benchmarks, competitive on the sixth; improvements remain single-digit percentages in absolute terms on the hardest benchmark.

Limitations

Authors concede the shared-noise construction admits a linear shortcut that could trivially extract targets, though they claim the learned models generally do not use it (Appendix B); the method requires a fixed maximum step budget matched to baselines during training. Unstated: no comparison against frontier language models or autoregressive chain-of-thought at comparable compute; ARC-AGI remains far from ceiling (12.2% on ARC-AGI-2); generalization to open-ended or natural-language reasoning is untested; benchmark scale is small (six benchmarks, puzzle-style data).

Counterpoint

A skeptic would note this is an architecture-paper benchmark game: results are on small curated puzzle datasets (ARC-style) where clever inductive biases historically shine but transfer to real workloads is unproven; the headline comparison is against one specialized baseline (TRM) with the same architecture rather than against strong autoregressive reasoners; ablations and shortcut analyses are self-reported without independent replication; 'outperform on five of six' allows cherry-picked benchmark emphasis; adaptive computation time and stochastic sampler add many tuned hyperparameters that invite prompt- and hyperparameter-sensitivity concerns.

Economy implication

Little or nothing yet. This is an early architecture result suggesting an alternative to autoregressive chain-of-thought for spending inference compute, and if it matured it could reduce token-generation costs for fixed-distribution tasks. For the next 12 to 24 months the main economic relevance is directional: inference-time scaling may not be locked to the token-stream paradigm, which would matter for token-revenue models and inference cost curves only after someone demonstrates it at frontier scale.

Who this matters for

Technical Capability: Architecture researchers and ML engineers should care because it addresses the known training failure mode of looped models (truncated BPTT) with local denoising objectives and shows inference compute scaling without autoregression.

Market & Valuation: Investors betting that inference revenue is inherently tied to token-generation economics should note that non-autoregressive inference-time scaling is now showing competitive reasoning results on benchmarks, a low-maturity but real alternative path.

Bears on (advisory · papers never write evidence)

[1.4] Inference-time compute yields gains that don't saturate quickly on hard reasoning.

Abstract

Humans and machines often solve harder problems by spending more time on computation. In deep learning, looped models implement this idea during inference by recurrently updating a hidden state. In practice, however, their training backpropagates through only one or a few updates, making it hard to train early updates to support future ones. We propose looped flows, an approach that sidesteps this issue by training the recurrence with local denoising objectives. By imposing temporal association across denoising objectives through progressively decreasing noise levels and shared noise, the model is incentivized to learn recurrent states that transfer useful computation over time, even when gradients cover only a few updates. We then formulate inference as integrating the velocity of a probability flow parameterized by the learned denoiser, coupled with recurrent states. This allows solving harder problems by spending more computation through a finer temporal grid and enables multiple valid predictions from different initial noise samples. Across six reasoning benchmarks including two multi-solution benchmarks, looped flows outperform prior state-of-the-art looped models overall, achieving 58.8% test accuracy on ARC-AGI-1 and 12.2% on ARC-AGI-2.