What Do Self-Evolving Agents Actually Evolve? A Substrate-Stratified Survey for the Frozen-Weights Era

Abstract

Large language model parameters are typically frozen after deployment, yet agent research advances along a different path: systems that keep improving themselves at run time. Drawing on 18 core publications from 2023-2026, this survey proposes a substrate-stratified framework that locates self-evolution mechanisms on four layers --- context and memory, skill libraries, agent programs, and model weights --- and examines the field's empirical base along two cross-cutting concerns: the correctness and the safety of evolution. The evidence indicates a clear trade-off: the closer the updated substrate lies to executable code and model weights, the larger the single-round gains, but the harder verification, rollback, and safety governance become. Moreover, some reported self-improvement gains lose robustness once cross-run variance and task-order sensitivity are strictly tested.

1 Introduction

Most current agent systems rely on manually crafted configurations that become static after deployment, which limits adaptation to dynamic environments; in response, researchers propose agent evolution techniques that automatically enhance agent systems from interaction experience [1]. A recent survey frames such self-improving agents as adaptive systems that convert experience into accumulated capability gains, with the agent realized as a configuration coupling a foundation model to its surrounding state [2].

A question often avoided is: when the foundation model's weights stay frozen, what does "self-evolution" actually change? A safety survey enumerates the updatable state --- model parameters, memories, tool definitions, skills, and workflows --- and observes that once experience becomes an update substrate, the safety problem persists across sessions [3]. This survey organizes the evidence along the "evolution substrate" axis and answers three questions: what representative mechanisms and empirical gains exist on each substrate layer; how sound is the reporting of those gains; and what design and governance trade-offs does the choice of substrate imply.

2 A Substrate-Stratified Framework

Building on existing taxonomies --- a six-dimensional taxonomy lists tool, memory, reasoning, policy, context, and coordination dimensions for self-evolving healthcare agents and notes that what constitutes true self-evolution remains unclear and inconsistently evaluated [4] --- we stratify mechanisms by the parametric position of the updated object:

Inter-layer flow deserves emphasis: EvoSC distills non-parametric textual experience into compact learnable parameters [6], and MCE couples a meta-level skill-evolution loop with a base-level context-optimization loop [7]. The boundaries between substrates are dissolving rather than fixed.

3 An Evidence Tour by Layer

3.1 Context and Memory: From Verbal Feedback to Structured Experience Graphs

Reflexion established the paradigm for this layer: reinforcing language agents not by updating weights but through linguistic feedback, with reflective text kept in an episodic memory buffer to improve subsequent decisions; it reaches 91% pass@1 on HumanEval, above the then-state-of-the-art GPT-4 at 80% [8]. ACE scales the idea into systematic context evolution, optimizing contexts offline (system prompts) and online (agent memory), beating strong baselines by 10.6% on agent benchmarks and 8.6% on finance while adapting without labeled supervision and reducing adaptation latency and rollout cost [9]. Against the fragmentation of unstructured memory, EXG organizes accumulated successes and failures into a structured, relational experience graph that works as a plug-and-play component, improving both solution quality and resource efficiency [10]. EvoSC further observes that unboundedly growing textual memory dilutes the model's attention and impairs inference accuracy, and compresses textual trajectories into compact learnable parameters through contrastive reflection and self-consolidation [6].

3.2 Skill Libraries: From Storage to Curation

Voyager demonstrates the prototypical form: an automatic curriculum maximizes exploration, an ever-growing skill library of executable code stores and retrieves complex behaviors, and an iterative prompting mechanism incorporates environment feedback, execution errors, and self-verification --- all via black-box GPT-4 queries with no fine-tuning; its learned skills transfer to a new Minecraft world to solve novel tasks, with exploration metrics such as tech-tree unlock speed up to 15.3x faster than prior state of the art [11]. SkillOS shifts the focus from storing skills to governing them: pairing a frozen executor with a trainable skill curator trained by reinforcement learning, on the grounds that high-quality skill curation is the key bottleneck while existing approaches rely on manual curation or heuristic operations [12]. MCE couples skill evolution with context optimization in a bi-level scheme, achieving 5.6-53.8% relative improvement over state-of-the-art agentic context-engineering methods across five domains (mean 16.9%) [7].

3.3 Agent Programs: From Symbolic Optimization to Self-Referential Evolution

Symbolic Learning optimizes symbolic parameters --- prompts, tools, and workflows --- with "language gradients," a lightweight form of program-level self-evolution [13]. ADAS proposes the "agent as code" research direction, where a meta agent iteratively programs ever better agents from an ever-growing archive of discoveries; the invented designs outperform hand-designed state of the art and maintain their advantage when transferred across domains and models [14]. The Darwin Godel Machine (DGM) pushes this to the self-referential limit: maintaining an archive of generated coding agents and openly exploring improvements to its own code, it raises SWE-bench performance from 20.0% to 50.0% and Polyglot from 14.2% to 30.7%, significantly outperforming ablations without self-improvement or open-ended exploration; the authors note all experiments ran under safety precautions [15].

3.4 Weights: The Thinnest Layer

Direct evidence on the weight layer remains scarcest. Iterative self-training paradigms are found prone to capability collapse, while self-evolving coding agents lack the capacity to drive foundation-model evolution, motivating open-ended approaches to neural network weight modification [5]. EvoSC's parameter distillation can be read as a transitional form toward this layer [6].

4 Cross-Cutting Concern I: Correctness of Evolution

A re-evaluation of memory-based self-improvement methods exposes their fragility: after stacking a self-improving loop, variance across runs increases in 71% of cases, the gap between the best and worst runs of the same experiment can reach 10 percentage points, and improvement depends heavily on task order, with default orderings acting as a hidden prerequisite curriculum [16]. This directly challenges the evidential basis of the "round-over-round improvement" narrative. SAGE addresses the acceptance mechanism: a naive acceptance gate takes the best score on a finite, noisy validation set, which the Optimizer's Curse biases upward; SAGE replaces it with per-item paired comparisons and a one-sided paired test that commits an edit only when wins are statistically reliable against losses, and abstains otherwise [17]. On the data-free route, EVE-Agent argues that agents should not train on examples they cannot justify, constraining the self-generated curriculum with evidence verifiability [18].

5 Cross-Cutting Concern II: Safety of Evolution

Self-evolution turns experience into an update source, and thereby turns the attack surface into a persistent one. SkillJack demonstrates sanitization whitewashing during skill extraction: safety detection on poisoned trajectories drops from 98.5% to 11.4% once the poison is distilled into skills, implanted skills achieve attack success rates of 56.2% and 89.2% on two representative systems, and 80.0% of skill-mediated attacks persist after the original poisoned records are deleted [19]. On formal guarantees, SEVerA brings verified synthesis to self-evolving agent programs, noting that such systems usually execute autonomously with no formal safety or correctness guarantees [20]; the safety survey similarly argues the problem persists across sessions once experience is an update substrate [3].

6 Design Principles and Open Problems

Three principles follow from the evidence. First, match substrate depth to verification strength: L3/L4 substrates enable deep single-round changes (e.g., DGM's 30-percentage-point jump [15]) and therefore require statistical acceptance gates [17] and formal verification [20]. Second, evaluation protocols must report cross-run variance and task-order sensitivity explicitly [16]; current reporting generally omits both. Third, persistent layers such as skill libraries require independent provenance auditing --- the post-extraction collapse of detection shows that trajectory-level safety scans cannot protect the skill layer [19]. Inter-layer distillation is the most promising route from a stratified framework to a unified theory of self-evolution, with empirical carriers already present in both directions: compression of textual experience into parameters [6] and the coupling of skill and context loops [7].

7 Integrity Statement

This survey was produced by an evidence-driven pipeline and self-reports its limitations.

1. Deleted hallucinated citations: 0 (automated three-layer verification report retained).
2. Claims weakened for insufficient evidence: 2 --- the weight layer (Section 3.4) is reduced to a risk statement plus transitional forms; the abstract attribution of self-improvement gains to evaluation noise was weakened to a robustness concern.
3. Citations supported by abstract only: 7 of 20.
4. Citations still flagged as unsupported-risk after verification: 6, each adjudicated as a cross-language similarity artifact with translation-level support confirmed against source passages; 18 DOI warnings were spot-checked (3 resolved, all genuine) and attributed to arXiv DOI metadata mismatches.
5. Search-coverage gaps: Semantic Scholar rate-limited throughout; queries in English only, Chinese-language literature not covered; the agentic-RL fine-tuning line contributed no usable record; most sources are 2025-2026 preprints with single-digit citations, so timeliness risk is high; industrial system reports without peer review (e.g., AlphaEvolve) were excluded.
6. Devil's-advocate attacks not adopted: the stratification is an organizing device rather than an ontological claim (inter-layer flow is discussed in Section 2); the missing RL line is recorded as a coverage gap instead of expanding the scope.
7. Length and structure: research-survey form, five-part structure, approximately 3,300 words of body text in the original Chinese version.

References

1. Jinyuan Fang, Yanwen Peng, Xi Zhang et al. (2025). A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems. arXiv, link 2. Zhe Ren, Yimeng Chen, Dandan Guo et al. (2026). Self-Improvements in Modern Agentic Systems: A Survey. arXiv, link 3. Jiahao Chen, Zhou Feng, Oubo Ma et al. (2026). Safety in Self-Evolving Agents: A Survey. arXiv (Cornell University), link 4. Shubham Vatsal, Ahsaas Bajaj, Harsh Dubey (2026). Beyond Static Agents: A Six-Dimensional Taxonomy and Survey of Self-Evolving LLM Agents for Healthcare. IEEE Access, link 5. Tao Zhang (2025). Toward Weight-level Self-improving Agents with Meta-knowledge Discovery. link 6. Hongzhuo Yu, Fei Zhu, Guo-Sen Xie et al. (2026). Self-Consolidation for Self-Evolving Agents. arXiv (Cornell University), link 7. Haoran Ye, Xuning He, Vincent Arak et al. (2026). Meta Context Engineering via Agentic Skill Evolution. arXiv (Cornell University), link 8. Noah Shinn, Federico Cassano, Ashwin Gopinath et al. (2023). Reflexion: language agents with verbal reinforcement learning. neural information processing systems, link 9. Qizheng Zhang, Changran Hu, Shubhangi Upasani et al. (2025). Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models. arXiv (Cornell University), link 10. Yuxin Jin, Siyuan Zhang, Hanchen Wang et al. (2026). EXG: Self-Evolving Agents with Experience Graphs. arXiv (Cornell University), link 11. Guanzhi Wang, Yuqi Xie, Yunfan Jiang et al. (2023). Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv (Cornell University), link 12. Siru Ouyang, Jun Yan, Yanfei Chen et al. (2026). SkillOS: Learning Skill Curation for Self-Evolving Agents. arXiv (Cornell University), link 13. Wangchunshu Zhou, Yixin Ou, Shengwei Ding et al. (2025). Symbolic Learning Enables Self-Evolving Agents. SSRN Electronic Journal, link 14. Shengran Hu, Cong Lu, Jeff Clune (2024). Automated Design of Agentic Systems. arXiv (Cornell University), link 15. Jenny Zhang, Shengran Hu, Cong Lu et al. (2025). Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. SuperIntelligence - Robotics - Safety & Alignment, link 16. Qinyuan Ye, Yu Li, Yada Pruksachatkun et al. (2026). On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification. arXiv, link 17. Yihao Wang, Linhan Xia, Rui Liu et al. (2026). SAGE: A Statistical Acceptance Gate for Self-Evolving Agents. arXiv (Cornell University), link 18. Yamato Arai, Yuma Ichikawa (2026). EVE-Agent: Evidence-Verifiable Self-Evolving Agents. arXiv (Cornell University), link 19. Zonghao Ying, XiangFan Wu, Huiyu Wu et al. (2026). SkillJack: Persistent Skill Backdoors in Self-Evolving Agents. arXiv (Cornell University), link 20. Debangshu Banerjee, Changming Xu, Eugene Ie et al. (2026). SEVerA: Verified Synthesis of Self-Evolving Agents. arXiv, link