CORAL: An LLM-Native Harness for Production Recommender Systems

2609.02730 ↗ · 2026-09-02 · cs.CL

Muhammad Rafay Azhar, Yuhang Zhou, Gilbert Jiang, Yuchen Wang, Rahul Sharma, Matthew DeSousa, Jiayi Liu, Xin Guo, Lizhu Zhang, Xiangjun Fan

Accepted by RecSys '26 OARS Workshop

The finding

Headline claim

An LLM agent in a closed loop, using memory and a constrained numerical optimizer, can autonomously retune production recommender system parameters and match or beat human-driven online experimentation, improving engagement at no extra serving cost on one platform and cutting serving cost without hurting engagement on another.

The test

Two large-scale Meta social platform surfaces, each with a different control problem (retrieval budget allocation; serving capacity allocation across user segments), run as a 3-day-cycle agentic loop and evaluated with live A/B experiments; the paper text provided is truncated before the results tables, so exact effect sizes are not visible.

Effect size & scope

Qualitatively: engagement gains including for low-signal and new users at no additional serving cost on one surface, and substantial efficiency savings without engagement degradation on the other. No numbers are visible in the provided excerpt, so magnitude cannot be assessed.

Limitations

Authors concede k=3 day cadence and m=3 memory horizon were sensible defaults, not tuned, and that systematically selecting them is open; attribution of outcomes rests on comparing before/after plus available A/B effects; results are from two surfaces at one company with operator-chosen objectives and constraints. Unstated: the agent's decisions occupy a narrow, low-dimensional budget-allocation space with heavy scaffolding, so the result says little about open-ended engineering work; case studies are single-surface each with no published external replication; the provided text cuts off before the results, preventing verification of claims.

Counterpoint

This is industrial PR-adjacent science from Meta's own teams on Meta's own systems with no external baselines: the comparison is against the status quo human process, not against strong automated alternatives like Bayesian optimization or bandit allocation, which might match the LLM loop at lower cost. Prompt sensitivity, the deterministic optimizer doing the real feasibility work, and cherry-picked surfaces all remain possible; A/B metrics like engagement can drift for reasons unrelated to the agent's choices.

Economy implication

Little or nothing yet for broad AI economics: if replicated, the relevant signal is that LLM agents can substitute for scarce algorithm-engineer iteration on narrowly scoped, tool-guarded tuning tasks at large platforms, pointing toward automation of continual ML-ops work rather than headline model capability gains. The evidence base here is two internal case studies, not a market-moving result.

Who this matters for

Labor & Knowledge Work: Algorithm engineers whose day-to-day work is iterative experiment-driven tuning should note a production system claiming to absorb exactly that loop, though scope is confined to budget-allocation parameters.

Technical Capability: It is a concrete production demonstration that agentic reliability comes from harness scaffolding (deterministic tools, guardrails, memory) rather than raw model capability.

Bears on (advisory · papers never write evidence)

[3.7] AI coding tools reduce enterprise software headcount or external vendor spend faster than new AI-related roles or tooling costs offset the savings, producing net cost compression.

Abstract

Production recommender systems shape what billions of people see, and sustaining their performance requires continual optimization: as content, user behavior, and upstream models shift, the choices governing retrieval, ranking, and serving must be revisited. Traditionally, human engineers test such changes through online experiments--a slow, reactive process limited by engineering effort, leaving parts of the system unrevised as conditions change. Although large language models have been applied to ranking, user modeling, and offline model development, few systems place an agent in a continual closed loop that acts on a live recommender and learns from the measured effects of its decisions. We present CORAL (Constraint-Optimized Recommender via an Agentic Loop), an LLM-native harness that closes this loop: each cycle, the agent observes operating signals, reasons over a memory of past decisions and outcomes, and invokes tools--including a numerical optimizer that keeps changes within a fixed operating budget--to reconfigure the recommender, with measured outcomes informing the next cycle. We formulate this as a partially observed, non-stationary, constrained optimization problem in which the policy improves in context, without parameter updates, from its prior actions. Across two large-scale social platforms, evaluated with A/B experiments, the same harness improves engagement at no additional serving cost on one and reduces serving cost without degrading engagement on the other, spanning the engagement-efficiency frontier. Performance improves as the loop iterates, suggesting that a single agentic loop can automate continual optimization work traditionally performed by human algorithm engineers under explicit guardrails.