Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
WHY IT MATTERS
This paper proposes an evidence-aware reinforcement learning approach to reduce repetitive copying behavior in long-context LLM reasoning tasks, demonstrating improvements in factual grounding.
A new reinforcement learning method directly targets the tendency of LLMs to copy verbatim from long input contexts rather than reasoning over evidence. The approach reduces repetitive copying and improves factual grounding in extended-context tasks.
For operators, this failure mode undermines reliability in document-grounded agents, legal analysis, and research assistants that must synthesize rather than regurgitate. Reducing copying without sacrificing context utilization lowers the risk of hallucination and spurious verbatim output. This signal suggests that future long-context deployments may require explicit reinforcement signals for evidence-aware behavior rather than relying on scale or attention mechanisms alone.
Operationally, builders can integrate evidence-aware reward functions into their fine-tuning pipelines, replacing or augmenting existing supervised data for long-context tasks. The primary workflow shift: evaluation metrics must now track copying ratios alongside accuracy, making automated detection of plagiarism-like behavior cheaper and more systematic. Second-order effect: agents processing lengthy proprietary documents become safer to deploy, potentially accelerating adoption in regulated industries.
SOURCE
ArXiv
SHARE
MORE FROM STUFFINSIDER
CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents
Jul 22RESEARCHAppearance Pointers – Multimodal Region Control of Diffusion Transformers
Jul 22RESEARCHSkewAdam – tiered optimizer cuts MoE state memory by 97% (fits 6.7B MoE on 40GB GPU)
Jul 22RESEARCHSWE-Pruner Pro: LLM pruning technique revealed on HuggingFace
Jul 21