Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators

2608.30110 ↗ · 2026-08-31 · cs.CL, cs.AI

Xinyue Zhao, Ruiyi Zhang, Liqin Ye, Rui Cao, Pengtao Xie, Sudheer Chava

The finding

Headline claim

In a strictly live, contamination-resistant six-month evaluation, web-search-enabled LLM agents produce macroeconomic nowcasts whose aggregate quality is broadly comparable to Federal Reserve and Bloomberg consensus nowcasts, with the top model (GPT-5) tying the Bloomberg ECOS professional consensus on a market-based aggregate score.

The test

LiveMacroEval: hourly nowcasts by four LLM agents (GPT-5, Claude-sonnet-4.5, Qwen3-235B, Qwen3-80B) over a six-month live window on sixteen US headline macro indicators (GDP, CPI, payrolls, housing, etc.), scored via a LiveMacro Score (predicted values' association with announcement-window S&P 500 futures returns) and a simulated LiveBetting Score (Polymarket-style trading), against five Fed regional-bank nowcasts, the Bloomberg ECOS consensus, and an auto-ARIMA baseline.

Effect size & scope

Aggregate performance is described as 'broadly comparable' to institutional benchmarks, with GPT-5 tying Bloomberg ECOS on the aggregate LiveMacro Score; performance varies widely across individual indicators, and several LLM agents are competitive with Fed nowcasts only for GDP and the unemployment rate on the LiveBetting Score. Six months and sixteen indicators is a small sample, so the tie at the aggregate level does not imply parity per indicator.

Limitations

Authors concede wide per-indicator heterogeneity, shared failure modes across models (an anchored-inflation floor and poor handling of sharp month-over-month reversals), and reliance on estimated surprise-to-return sensitivities for the headline metric. Unstated: six months is one macro regime; the LiveMacro Score measures market-usefulness rather than raw accuracy; the LiveBetting Score is a simulated return, not realized trading with liquidity and slippage; the possibility of implicit reliance on consensus information available on the open web is tested only via three tests rather than eliminated.

Counterpoint

'Ties the consensus' on a bespoke market-metric over six months may reflect metric design choices (weighted aggregation chosen partly because standard error metrics rank unstably, per Appendix K) rather than genuine nowcasting skill; the betting score is simulated, not realized; prompt sensitivity of agent pipelines is high; four models over sixteen indicators in a single regime is thin evidence; and copying tests can miss subtler forms of ingesting published consensus via web search, since the consensus itself is public information the agents can retrieve.

Economy implication

If this holds up, LLM agents become a credible low-cost, high-frequency substitute or complement for real-time macro data synthesis in trading desks and policy shops within 12 to 24 months, but the honest near-term read is modest: comparable-to-consensus accuracy on a handful of indicators over one regime does not yet establish that agents displace Fed or Bloomberg nowcast products, and Bloomberg's own terminal distribution of the consensus remains the bottleneck asset.

Who this matters for

Technical Capability: This is direct evidence that tool-using agents with web search can match expert benchmarks on a live, contamination-free, economically consequential task, not just static quizzes.

Market & Valuation: If agents tie the professional consensus on market-moving nowcasts, the pricing power of paid consensus products (Bloomberg ECOS) and Fed nowcast mindshare faces a real, cheap competitor.

Abstract

Nowcasting headline macroeconomic indicators, i.e., estimating an indicator's value for the current reference period before its official release, is critical for monetary policy and financial markets, and central banks devote dedicated teams of expert economists to producing such estimates. Large language model (LLM) agents are a promising candidate for this task, combining broad world knowledge with real-time web search and supporting queries at higher frequency than institutional nowcasts. Evaluating their nowcasting capability is, however, challenging: headline indicators such as GDP and CPI are widely reported and likely memorized during pretraining, so any evaluation on historical releases is vulnerable to data contamination. To address this, we introduce LiveMacroEval, a live, contamination-resistant benchmark in which LLM agents produce hourly nowcasts for sixteen major U.S. macroeconomic indicators over a pre-release window closing at each official release. Nowcast quality is assessed through a LiveMacro Score against announcement-window equity returns and a LiveBetting Score from simulated Polymarket-style trading, with Federal Reserve regional-bank nowcasts, the Bloomberg ECOS professional consensus, and an auto-ARIMA baseline as comparators. Over six months with four state-of-the-art LLM agents configured with web search, aggregate nowcast accuracy is broadly comparable to the institutional and professional benchmarks, with performance varying widely across individual indicators. This highlights LLM agents' potential as real-time estimators of macroeconomic conditions.