SkillOpt: Training Agent Skills Like Neural Network Weights

Paper: 2605.23904 Authors: Yifan Yang, Ziyang Gong, Weiquan Huang, Qihao Yang, Ziwei Zhou, Zisu Huang, Yan Li, Xuemei Gao, Qi Dai, Bei Liu Categories: cs.AI, cs.CL

The Gap

Agent skills today come from three sources: hand-crafted by humans, generated one-shot by LLMs, or evolved through self-revision methods like TextGrad or EvoSkill. None of these behave like a proper optimizer. Hand-crafted skills don’t improve automatically. One-shot generation has no feedback loop. Self-revision methods make uncontrolled edits without validation gates, so they drift rather than converge. The field lacks a text-space equivalent of SGD for skills.

Problem: Agent skills don't improve reliably under feedback
   |
   v
Assumption: Skills are external state that can be optimized
   |
   v
Method: Separate optimizer model + validation gate + edit budget
   |
   v
Evidence: +23.5 points on GPT-5.5, beats all baselines on 52 cells
   |
   v
Conclusion: Text-space optimization works when disciplined

The Increment

One sentence: Before SkillOpt, agent skills were static artifacts or drifted under loose self-revision; after SkillOpt, skills are trained through bounded edits with strict validation, converging like neural network weights.

Core Mechanism

SkillOpt separates the agent (frozen) from the skill (trainable). The skill is a text document containing instructions, examples, or heuristics. An optimizer model observes scored rollouts from the agent executing tasks with the current skill. It proposes edits: add a new example, delete a confusing instruction, replace a heuristic. Each edit is bounded by a learning-rate budget (max tokens changed per step). The edit is applied to a candidate skill, tested on a held-out validation set, and accepted only if validation score strictly improves. Rejected edits go into a buffer to prevent cycling. After multiple edits (an epoch), a slow update merges accepted changes into the skill, and a meta update adjusts the learning rate.

Rollouts (train) --> Optimizer Model --> Proposed Edit
                                              |
                                              v
                                    Apply to Skill Copy
                                              |
                                              v
                                    Test on Validation Set
                                              |
                         +--------------------+--------------------+
                         |                                         |
                    Score Up                                  Score Down
                         |                                         |
                    Accept Edit                              Reject --> Buffer
                         |
                         v
                    Update Skill

Think of SkillOpt as a personal trainer for the agent. The agent is the athlete (frozen, not changing its muscles). The skill is the training regimen (diet, exercises, rest schedule). The optimizer is the trainer who watches performance videos (rollouts), suggests tweaks to the regimen (edits), and only keeps changes that improve race times on a test track (validation set). The learning-rate budget is how much the trainer can change per week. The rejected-edit buffer is a logbook of failed experiments so the trainer doesn’t repeat mistakes. The slow update is the monthly review where successful tweaks become permanent habits. The meta update is adjusting how aggressive the trainer should be based on whether the athlete is plateauing or improving.

Key Concepts

• Text-space optimization: Neural networks optimize in weight space (continuous numbers). SkillOpt optimizes in text space (discrete tokens). The skill is a document, not a parameter vector. Edits are add/delete/replace operations on text spans, not gradient steps on floats. The validation gate replaces the loss function: an edit is “better” if it raises validation accuracy, not if it lowers a differentiable loss. This makes optimization discrete and non-differentiable, but interpretable and transferable across models.

• Validation gate: Most self-revision methods apply edits if they seem reasonable or improve training score. SkillOpt requires strict improvement on a held-out validation set. This prevents overfitting to training examples and stops the skill from drifting into local minima. It’s the equivalent of early stopping in neural network training, but applied per edit instead of per epoch. Without this gate, skills accumulate noise and degrade over iterations.

• Learning-rate budget: In SGD, learning rate controls step size in weight space. In SkillOpt, the learning-rate budget controls how many tokens can be added, deleted, or replaced per edit. A small budget (e.g., 50 tokens) forces incremental changes. A large budget allows rewriting entire sections. The budget is adaptive: if validation scores plateau, the meta update shrinks the budget to make finer adjustments. This prevents the optimizer from making wild swings that break the skill.

Framework Shift

Before (one-shot or loose self-revision):

Human/LLM --> Skill (static) --> Agent --> Task
                                   |
                              (no feedback loop
                               or uncontrolled edits)


After (SkillOpt):

Rollouts --> Optimizer --> Edit Proposal --> Validation Gate
                                                    |
                                               Accept/Reject
                                                    |
                                                    v
                                            Skill (evolving) --> Agent --> Task
                                                    ^
                                                    |
                                            (disciplined loop)

From static artifact to trainable state, the core shift is treating the skill as the optimization target with the same rigor as neural network weights.

Expert Assessment

Problem choice: Real gap. Agent skills are a bottleneck in practice. Hand-crafting doesn’t scale, one-shot generation is hit-or-miss, and existing self-revision methods (TextGrad, EvoSkill) lack the discipline to converge reliably. This paper identifies that the missing piece is a proper optimizer with validation gates and learning-rate control. The problem sits at the intersection of prompt engineering and meta-learning, both active areas.

Method maturity: Clever insight with solid engineering. The core idea—treat skills as external state and optimize them like weights—is elegant. The validation gate and learning-rate budget are borrowed from neural network training, which is smart reuse. The rejected-edit buffer and slow/meta updates add stability without overcomplicating. However, the method requires a separate optimizer model, which adds cost during training (though zero cost at deployment). A simpler approach might be to use the agent itself as the optimizer, but that risks instability. The choice to freeze the agent and optimize the skill externally is defensible.

Experimental integrity: Strong. The paper tests on six benchmarks, seven models, and three execution harnesses (52 cells total). SkillOpt wins or ties on all cells. Baselines include human-written skills, one-shot LLM, and four self-revision methods (Trace2Skill, TextGrad, GEPA, EvoSkill). The +23.5 point gain on GPT-5.5 is substantial. Transfer experiments show skills retain value across model scales and execution environments, which is a good robustness check. One potential concern: the validation set is held-out from training, but it’s unclear if it’s from the same distribution as the test set. If validation and test distributions differ, the validation gate might not generalize. The paper doesn’t discuss this explicitly.

Writing quality: Clear structure, but the related work section is dense and could be trimmed. The method section is well-organized, though the slow/meta update mechanism is introduced late and feels tacked on. The ablation study (if present) would benefit from being highlighted earlier to show which components matter most. The transfer experiments are buried in the appendix but deserve more prominence in the main text since they demonstrate the skill artifacts’ value.

Verdict: strong accept — First systematic text-space optimizer for agent skills with rigorous validation and strong empirical results across diverse settings.

Takeaways

Practitioners can steal the validation gate pattern: don’t accept changes just because they improve training metrics—require strict improvement on a held-out set. This applies beyond agent skills to any iterative refinement process (prompt tuning, config optimization, hyperparameter search). The learning-rate budget is also transferable: when optimizing discrete objects (text, code, configs), bound how much you change per step to prevent instability. Finally, the rejected-edit buffer is a simple but effective way to avoid cycling in discrete optimization—keep a memory of what didn’t work and don’t retry it.