What Should an Agent Forget? Separating What Is Stored from What Is Used

2609.10263 ↗ · 2026-09-09 · cs.AI

Yuhang Li, Yuchen Li

8 pages, 3 figures, 3 tables

The finding

Headline claim

A training-free, query-conditioned memory architecture (RD-Forget) that separates what an agent stores from what it uses at answer time substantially improves accuracy on memory benchmarks, with gains of 11 to 26 percentage points over stronger baselines on fact consolidation.

The test

Three QA suites (AMB-Text, 86 questions from LoCoMo; LME-KU, 78 LongMemEval knowledge-update questions; MAB-FC, 100 MemoryAgentBench fact-consolidation questions; 264 tasks per method) plus a query-intent evaluation on BEAM (150) and PersonaMem (579), across four model families (Qwen3.5-flash, GPT-5.6-Luna, MiniMax-M2.5, Kimi-K2.5), comparing RD-Forget against ACE and ReasoningBank memory policies with a 2,048-token memory budget and matched ablations (forgetting off, query conditioning off, slot grouping off, rescue off).

Effect size & scope

On fact consolidation (MAB-FC), RD-Forget beats the better baseline by 11 to 26 percentage points across backbones (e.g., GPT-5.6-Luna: 77 vs 51 for ACE); mean scores reach 66.59 (Qwen3.5-flash), 86.71 (GPT-5.6-Luna) versus ACE's 48.86 and 67.26. Ablations show no-forgetting and no-query-conditioning configurations score lowest. Gains hold across all four model families on the tested suites; the paper text is truncated before full ablation and intent-suite numbers, so robustness beyond the three main suites is not verifiable from what is shown.

Limitations

The authors concede model-judged scoring is within-protocol (same model family curates, answers, and judges) rather than independent evaluation; task counts are small (86 to 100 questions per suite); MAB-FC uses substring-matching rule accuracy; some auxiliary inputs (query dates, task-type labels feeding intent detection) are implicit; the budget B=2,048 tokens is fixed. Unstated: curation and selection are LLM-based, so results depend on curator quality and prompt phrasing; Figure 2 is a constructed example, not a measurement; baselines (ACE, ReasoningBank) were designed for different memory problems, so the comparison may partly reflect fit to the tested task types.

Counterpoint

Small, partially benchmark-derived suites (AMB-Text comes from one LoCoMo conversation) invite overfitting of a bespoke pipeline to the eval; self-judging by the same model family can inflate RD-Forget's advantage; substring-matching accuracy is brittle; ablations are only 'matched Luna'; and a training-free framework with many tuned switches (slot syntax, 0.4 bonus, budget, rescue logic) has many degrees of freedom for benchmark-tuning without pre-registration or code availability shown.

Economy implication

Little or nothing yet for the macro AI-economics debate; at most, it signals that agent-memory engineering is an active, cheap (training-free) differentiator layer, implying agent reliability gains may come from scaffolding and context management rather than model scale alone, which mildly favors app-layer differentiation over model-layer rent.

Who this matters for

Technical Capability: Agent builders get concrete evidence that separating storage from answer-time use, with slot-scoped supersession and historical rescue, is a tractable path to reliable memory updates without retraining.

Labor & Knowledge Work: If persistent agent memory becomes reliable at cheap inference cost, agents can hold long-running user and project context, expanding the range of knowledge-work tasks automatable end to end.

Bears on (advisory · papers never write evidence)

[1.3] Agentic reliability (multi-step completion without human correction) is improving materially, not just peak single-shot.

[B1] Agentic reliability gains will translate into enterprise ROI.

Abstract

Persistent language agents need stored experience to remain available across time, while each answer requires evidence suited to a particular question. A superseded fact can mislead a current-state answer and still be essential for a historical query. We present RD-Forget, a training-free framework that separates what an agent stores from what it uses. A retained source archive preserves observations, and a query-conditioned memory view controls their influence on the current answer. A frozen language-model curator extracts relevant evidence, groups facts into semantic slots, and preserves the relations needed for multi-hop reasoning. Same-slot replacement links suppress superseded values in current-state contexts, while intent-aware retrieval makes earlier evidence eligible again. A rate-distortion formulation guides construction of the answer-time view within a memory budget. Experiments span conversational memory, knowledge updating, fact consolidation, long-context reasoning, and personalization under a shared answering pipeline. The results associate accurate answers with both query-relevant evidence construction and control over obsolete alternatives. Configurations without forgetting or query conditioning have the largest score deficits, while slot grouping, historical access, and relation preservation contribute complementary functions. Retaining history while selectively controlling its use offers a practical way to accommodate changing facts and future questions.