The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?

2609.30705 ↗ · 2026-09-25 · cs.AI · h-max 0 · 0 citations

Jiayi Chen, Guiling Wang

The finding

Headline claim

Increasing inference-time reasoning effort in LLM trading agents does not reliably improve net portfolio returns after transaction costs; effects are nonmonotonic and unstable across repeated generations.

The test

Controlled within-backbone experiment varying reasoning level while holding information, prompts, output schema, and portfolio rules fixed, across DeepSeek-Flash, GPT-5.6 Luna, and Gemini 3.1 Flash-Lite; 241 formation dates in 2024, 100 liquid US equities, three information conditions (numerical, identifiable news, masked news), 808,800 asset predictions, 10 bps transaction costs, equal-weight market-neutral long/short portfolios, and paired Newey-West inference on 241 daily return differences plus three-generation frozen audits on 48 dates.

Effect size & scope

None of nine primary annual low-vs-baseline tests shows material benefit (predeclared threshold 0.25 bps/day); one replicated DeepSeek audit finds low reasoning with masked news reduces returns by 9.016 bps/day (95% CI -16.248 to -1.784) versus no reasoning; GPT and Gemini sign-flip across generations in at least one news condition. Holds for these three models, one year, one portfolio mechanism.

Limitations

Authors concede results apply only to tested models, inputs, historical year, and portfolio mechanism; wide confidence intervals do not establish effects equal zero; inconclusive is not equivalence; masking cannot prove pretraining data absence; baseline portfolios not shown prospectively profitable. Unstated: single year 2024, single simple portfolio rule, opaque model naming, and possible contamination of 2024 history in pretraining data.

Counterpoint

Backtesting on one year of US equities with a single generic long/short rule may understate reasoning's value in other regimes, asset classes, or portfolio designs; the masked-news condition may strip signal the reasoning could exploit; only one backbone got the full four-level curve; and a null on one task does not refute test-time compute gains shown on verifiable tasks.

Economy implication

For inference-cost economics specifically: it weakens the assumption that paying for more test-time compute buys proportional decision value in noisy, delayed-feedback domains, suggesting vendors cannot price reasoning tiers on capability claims alone in applied settings. Otherwise little or nothing yet for broader AI-economy questions; this is one task, one year, three models.

Who this matters for

Technical Capability: Direct evidence that inference-time compute gains demonstrated on verifiable benchmarks do not transfer to noisy, delayed-feedback decision tasks, complicating the 'more reasoning is better' narrative.

Market & Valuation: Quant firms deploying LLM traders get a caution: reasoning-tier pricing may raise costs without reproducible alpha after costs.

Bears on (advisory · papers never write evidence)

[1.4] Inference-time compute yields gains that don't saturate quickly on hard reasoning.

[3.1] Inference cost per unit of capability is falling rapidly.

Abstract

While inference-time reasoning in large language models (LLMs) promises better decision making, its higher computational cost may not yield better economic outcomes. Yet reasoning controls are rarely evaluated as economic interventions, where changes in model outputs must translate into better portfolios after trading costs. We conduct a controlled study of representative LLMs from the DeepSeek, GPT, and Gemini families. We vary reasoning effort while holding information available at each formation date, prompts, output formats, and portfolio construction fixed. Our evaluation covers a full year of U.S. equities under three input conditions: numerical, identifiable news, and masked news. It includes more than 800,000 asset predictions and repeated model generations. Across all three model families, additional reasoning does not produce a reliable improvement in net portfolio returns. For DeepSeek, where we examine the full progression from no reasoning to maximum reasoning, performance is nonmonotonic. Repeated generations also produce unstable treatment effects and portfolio selections, even when overall scores remain similar. These findings show that additional reasoning can change financial decisions without reliably improving their economic value, motivating validation for each task before deployment.