ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL

2608.28476 ↗ · 2026-08-28 · cs.CL

Zhuoshi Pan, Qizhi Pei, Junru Lu, Honglin Lin, H. Vicky Zhao, Di Yin, Xing Sun

10 pages, 6 figures, 5 tables, accepted to EMNLP 2026 (Main Track)

The finding

Headline claim

Training agents with an expanded context-management toolset plus RL with snapshot-level fine-grained credit assignment yields better long-horizon task accuracy while keeping the working context more compact than prior proactive context-management baselines.

The test

Multi-turn long-context QA (NovelQA, NarrativeQA for SFT, LongBench-v2 for RL) and deep search tasks, across base models Qwen3-8B, Qwen3-14B, Gemma4-E4B-it, and search-tuned WebSailor-7B and WebExplorer-8B, compared against prior proactive context-management baselines (StateLM, Sculptor, AgentFold, ReSum, SUPO); the paper text provided cuts off before reporting headline result numbers.

Effect size & scope

Not quantifiable from the provided text: the main results section (5.2) is truncated, so no accuracy deltas, token-efficiency numbers, or significance values are visible; the qualitative claims are consistent gains across models and benchmarks plus a more compact context, and a pilot study showing branching variance across context-editing tools (Qwen3-8B on NovelQA, 10 continuation rollouts per tool).

Limitations

The full text provided omits all quantitative main results, ablation magnitudes, and statistical testing; authors rely on rule-based teacher scaffolding for SFT synthesis that is excluded from training data but shapes behavior; the sensitivity score mixes context-length and entropy variation with hand-set weights alpha and beta; reward design includes format and penalty terms that could be gamed; evaluation is limited to QA and deep search benchmarks, not real enterprise or multi-day tasks; the pilot-study variance analysis uses only Qwen3-8B on a single benchmark.

Counterpoint

Gains may reflect the larger toolset rather than the RL credit-assignment contribution, and the ablations needed to separate these are not visible in the provided text; snapshot-level rewards still derive from sparse outcome correctness, so 'fine-grained credit assignment' remains reward-shaped by final answer accuracy, which the authors' own Figure 3 shows can mismatch context-management quality; benchmarks like LongBench-v2 and NovelQA may be contaminated or saturating; no held-out real-world agent tasks; results hinge on specific base models and a bespoke harness that others may not reproduce despite released code.

Economy implication

Little or nothing yet at the macro level; the narrow relevance is to inference-cost economics: if agents can hold accuracy with materially smaller working contexts, per-task token spend for long-horizon agent deployments falls, which bears on inference cost curves and agent unit economics, but the paper does not report dollar costs and the provided text lacks the effect sizes to size that saving.

Who this matters for

Technical Capability: This is a concrete recipe for making multi-step agents reliable on long tasks, addressing the context-window bottleneck that currently caps unattended agent runs, so labs and agent builders should track whether the gains replicate.

Market & Valuation: If context compression lets agents finish long tasks with fewer tokens, it pressures the inference-revenue line of API vendors who currently benefit from verbose agent contexts, though the paper itself never prices this.

Bears on (advisory · papers never write evidence)

[1.3] Agentic reliability (multi-step completion without human correction) is improving materially, not just peak single-shot.

Abstract

Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical editing decisions for branch sampling and estimates action-level advantages from all branched trajectories that pass through the corresponding context editing action. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, consistently outperforming existing baselines across various base models and benchmarks. Code is available at https://github.com/Tencent/ContextPilot.