Grounding Agent Memory: Environment-Probing Curation for Enterprise Agents

2609.11060 ↗ · 2026-09-10 · cs.AI, cs.SE

Susheel Suresh, Hazel Mak, Sahil Bhatnagar, Chhaya Methani, Alejandro Gutierrez Munoz

The finding

Headline claim

Giving a post-task memory curator read-only environment tools (propose-probe-commit) makes agent memory more accurate and actionable, raising pass rate on enterprise-like agent tasks while cutting task-agent tool calls and cost.

The test

Production-like GitHub Copilot SDK harness with fixed task-agent model (gpt-5.4): CLBench (40-question schedule with mid-stream SQLite schema drift, plus 30-question no-drift schedule on Sonnet 4.6 and Opus 4.7, five paired seeded runs per config) and 90 adapted APEX management-consulting tasks across six document worlds (five runs stateful, three stateless), comparing stateless, full ICL, memory-only, and memory-with-probing.

Effect size & scope

CLBench: pass rate 39% (no memory) to 73% with probing (61% with memory alone); pass-discounted reward 8.60 to 22.60 (20.00 memory alone); queries 8.8 to 4.7; task-agent cost $3.38 to $1.68. APEX: all 18 memory-vs-baseline mean reward comparisons positive, tool calls down 16-75%, probing best reward-per-dollar in five of six worlds; holds across two additional frontier models without schema drift. Scope: two benchmarks, Microsoft's own harness, single primary model, small question counts (40 and 90), curation costs excluded from reported task-agent cost.

Limitations

Authors note probes require a safe read surface (fallback to trajectory-only otherwise); curation-phase cost is tracked separately and excluded from headline cost figures; benchmarks are adapted by the authors themselves. Unstated: probe cost may erase the task-agent savings; benchmarks authored/curated by Microsoft and the vendored-memory ecosystem may favor their own design; results depend on environments amenable to cheap read-only verification; no comparison to external baselines like Mem0 or Zep in the reported numbers shown.

Counterpoint

This is a vendor paper (all Microsoft authors) evaluating on benchmarks adapted in-house inside its own product harness, with a memory design that presupposes the environment offers low-risk read-only tools; small task counts (40 CLBench, 90 APEX, ~6 worlds), benchmark-specific reward formulas (pass-discounted with hand-set B=15/B=100), and excluded curation costs all could inflate the story; prompt sensitivity of the curator is unexplored and no external memory systems are directly benchmarked in the reported section.

Economy implication

Modest and mostly 12-24 month operational: if agentic memory with verified curation generalizes, enterprise agent unit costs could fall meaningfully because agents stop re-discovering environment structure each session, improving the ROI case for agent deployments. But it is a single-vendor result on two self-adapted benchmarks; treat as an architectural signal (probe-before-write curation) rather than evidence on aggregate AI economics.

Who this matters for

Technical Capability: Shows agent memory reliability, not model capability, is now a controllable engineering lever: adding read-only environment verification to memory curation nearly doubles pass rate without retraining.

Labor & Knowledge Work: If agents complete database-analysis and consulting-analyst tasks at 73% pass with fewer interventions, the tasks these benchmarks proxy (junior data and consulting work) look more automatable than peak-single-shot results suggest.

Bears on (advisory · papers never write evidence)

[B1] Agentic reliability gains will translate into enterprise ROI.

Abstract

Persistent memory is entering production-oriented agent platforms to help long-horizon agents accumulate experience across sessions. Yet a post-task curator agent restricted to completed trajectories can preserve errors, overgeneralize partial evidence, or retain stale knowledge. We introduce environment-probing curation, a deployment-compatible extension that gives an existing asynchronous curator agent least-privilege, read-only world tools to check, scope, and refresh candidate memories. It requires no model retraining and leaves the task agent, retriever, memory representation, and production write authority unchanged. In a production-like GitHub Copilot (GHCP) harness built on its SDK, we compare stateless execution, full in-context learning, GHCP + Mem, and GHCP + Mem (w/ Env Probing) on CLBench database exploration and 90 adapted APEX management-consulting tasks. On CLBench, probing raises pass rate from 39% to 73% and pass-discounted reward from 8.60 to 22.60 while reducing queries from 8.8 to 4.7 per question and task-agent cost from \$3.38 to \$1.68. Across six APEX worlds, all 18 memory-versus-baseline mean reward comparisons are positive and task-agent tool calls fall by 16--75%; probing gives the best task-agent reward gain per dollar in five worlds. Probing also attains higher mean reward than GHCP + Mem on both Sonnet 4.6 and Opus 4.7 without schema drift. Environment probing therefore turns existing agent-memory curation into an environment-informed, auditable process while preserving a compact task-time interface.