Tejas Srinivasan, Shikib Mehri, Nandita Shankar Naik, Anirban Das, William M. Campbell, Jesse Thomason
Frontier LLM agents handle underspecified user preferences but fail to satisfy preferences that are unachievable or triggered mid-interaction, and uncertainty-focused prompting does not close the gap; even the best model satisfies all user preferences less than 30% of the time.
AcCoRD, a benchmark of 100 collaboration scenarios in each of two simulated domains (online shopping in WebShop, travel booking in TravelGym), with a controlled taxonomy of four preference dynamics (hard underspecification, soft underspecification, unachievable, triggered). Five frontier LLMs were evaluated under two prompting strategies (ReAct and UncReAct) interacting with LLM-based user simulators, scored on task completion, outcome score, task score, and perfect outcome.
Claude-Sonnet-4.5, the strongest model, achieved PerfectOutcome below 30%; unachievable and triggered preferences were universally the hardest dynamics, and UncReAct prompting did not improve preference satisfaction or materially change agent behavior. Results are limited to two simulated domains with 100 scenarios each and LLM-judged preference scoring.
The authors rely on LLM-based user simulators and LLM preference scoring, both potential sources of noise and bias; scenarios are synthetic or LLM-generated modifications of UserBench tasks; the full text provided reports only headline qualitative findings without confidence intervals. Unstated concerns: simulator fidelity to real user behavior is unvalidated at scale, 100 scenarios per domain is small, and the negative result on prompting may reflect a single prompt design rather than a general limit of elicitation strategies.
A skeptic would note the benchmark's preference taxonomy is author-defined, scoring depends on an LLM judge that may misrate partial satisfaction, user simulators may behave unrealistically (e.g., rigidly withholding or backoff behavior), n=100 per domain with five models and two prompts is a thin grid, and 'prompting fails' claims are notoriously prompt-sensitive with no fine-tuned or RL-trained baselines tested.
Little or nothing yet for the economics of AI in the next 12 to 24 months. At most it suggests a near-term ceiling on fully autonomous customer-facing assistant deployments where negotiation and evolving preferences matter, implying human-in-the-loop costs persist until uncertainty modeling is trained rather than prompted.
Technical Capability: Agent builders get a concrete diagnostic: the bottleneck is not acting in environments but modeling user uncertainty, which prompting does not solve.
Labor & Knowledge Work: Teams deploying assistants for shopping, travel, or support should expect continued human escalation for preference negotiation rather than full automation.
Market & Valuation: Vendors selling agentic customer-experience products face a measurable quality ceiling on complex preference-heavy tasks, relevant to what enterprises will pay for.