Boyu Yang, Jiazheng Sun, Zilong Lu, Zhi Qiu, Xin Peng, Jun Zheng
15 pages, 12 figures
A post-retrieval layer that combines LOO screening with sampled Shapley attribution can identify and suppress memory records that harm an LLM agent's task performance, recovering task accuracy that single-record counterfactual screening misses.
Evaluation on ten long-dialogue memory pools with injected harmful memories, comparing MeClear's Shapley-based attribution and verified query-scoped clearance against Leave-One-Out baselines, using injected-target-recall and task-recovery metrics.
Injected target recall of 85.9% and overall task recovery rate of 82.3%, a 25.5 percentage-point improvement over LOO baselines. The theoretical result (Theorem 1) shows LOO misses redundant harmful memories entirely (zero effect), while Shapley assigns them -Delta/2 each.
Authors concede values and task gains must be approximated by finite counterfactual evaluations (sampling cost), that attribution depends on the evaluator and tolerance threshold tau, and that clearance is query-conditioned, not a permanent fix. Unstated: the required number of agent executions per query (L permutations times K memories) may be prohibitively expensive in production; the 'harmful' memories appear to be synthetically injected rather than naturally occurring; baseline comparison is limited to LOO.
The experimental scale is small (ten memory pools), harmful memories are injected by the authors rather than emerging organically, and the method requires many counterfactual agent executions per query, which at inference cost could exceed the value of the fix. No comparison to simpler baselines like retrieval-score thresholds or admission filters beyond LOO. The 25.5pp gain is against a weak strawman baseline known to fail by construction.
Little or nothing yet for the economics of AI. If mature, this line of work could reduce the operational cost of maintaining reliable long-horizon agents, but the compute overhead of Shapley estimation makes the net cost-benefit unclear.
Technical Capability: Agent builders working on long-horizon memory reliability get a concrete technique for detecting redundant harmful memories that single-record interventions structurally cannot find.