A new arXiv preprint titled Recursive Self-Evolving Agents via Held-Out Selection identifies a critical failure mode for unguarded recursive self-evolution of LLM agents: severe cross-benchmark performance regressions. The work, posted June 17, 2026 by researchers Michael Nguyen, Quoc Nguyen, and Paul Vuong, argues that held-out selection gates are a non-negotiable safeguard for these systems.
The paper challenges the widespread practice of unvalidated natural-language context evolution for agent improvement, an approach often reported as successful in prior work only on the single benchmark where it is evaluated, with cross-distribution failure modes left unmeasured. The paper is available on arXiv under the identifier 2606.28374.
The research introduces RSEA (Recursive Self-Evolving Agent), a system that maintains a compact three-layer natural-language state to improve frozen LLM agents without weight updates. The three layers are a top-level imperative strategy, reusable task-specific skills, and a procedural playbook that logs execution reflections and error corrections.
RSEA evolves these natural-language artifacts (including reflections, workflows, and playbooks) from its own execution history across generations, rather than adjusting underlying model parameters. Full details of the RSEA system architecture and keep-better selection gate design are available in the preprint.
Recursive Self-Evolving Agents via Held-Out Selection Eliminate Cross-Task Regressions
The core innovation distinguishing RSEA from prior unguarded self-evolution methods is its strict keep-better selection gates. These gates evaluate every candidate update to the three-layer state against a disjoint held-out test split before committing the change to active operation. Only updates that do not regress performance on the held-out split are deployed, preventing harmful changes from being adopted. This mechanism enables what the authors term “monotone-safe evolution,” where the agent’s performance never falls below the base ReAct baseline on any evaluated task.
RSEA Outperforms Six Faithful Baselines on ALFWorld, With No Cross-Benchmark Underperformance
The RSEA evaluation tested the system against six faithful baselines (ReAct, Reflexion, GEPA, AWM, ACE, Dynamic Cheatsheet) across four diverse benchmarks, including ALFWorld and WebShop. All methods were run on a single shared local backbone to ensure apples-to-apples comparison, eliminating variance from different base model weights. Full cross-benchmark evaluation results and the Dynamic Cheatsheet failure case are documented in the preprint.
On the ALFWorld benchmark, RSEA achieved 69.3% single-pass accuracy, outperforming the base ReAct agent’s 64.6% single-pass score with a statistically significant McNemar p-value of 0.015. With retry enabled, RSEA reached 79.4% accuracy, the highest overall result reported on ALFWorld to date.
Critically, RSEA never significantly underperformed the base ReAct agent on any evaluated benchmark, falling back to vanilla ReAct behavior when evolved context would hurt performance. The study also surfaces a key limitation of current evolution artifact research: no single artifact type universally wins across task types.
While RSEA leads on the ALFWorld benchmark, the concrete-workflow induction method AWM outperforms all other approaches on the other three evaluated benchmarks. This underscores the need for task-specific evolution strategies rather than one-size-fits-all solutions for agent self-improvement.
Unguarded Context Evolution Carries High Cross-Benchmark Failure Risk
The paper uses the Dynamic Cheatsheet baseline to demonstrate the danger of unvalidated context evolution. Dynamic Cheatsheet curates agent context online without a held-out validation gate, updating its state based on execution feedback alone. On ALFWorld, it reached 70.7% single-pass accuracy, nearly matching RSEA’s single-pass score. When evaluated on the WebShop benchmark, however, Dynamic Cheatsheet collapsed to a score of 0.14, less than a third of the base ReAct agent’s 0.43 score on the same benchmark. This 67% performance drop, the authors note, makes unguarded evolution unsuitable for production agent deployments where consistent performance across diverse task distributions is a hard requirement.
The finding also highlights a widespread gap in current agent evaluation practices, where most self-evolution methods are only tested on a single benchmark. This single-benchmark evaluation masks cross-distribution failure modes that would emerge in real-world use cases, where agents encounter task types not seen during development.
