MLEvolve: Self-Evolving LLM Agents for Automated Machine Learning Discovery

Paper: 2606.06473 Authors: Shangheng Du, Xiangchao Yan, Jinxin Shi, Zongsheng Cao, Shiyang Feng, Zichen Liang, Boyuan Sun, Tianshuo Peng, Yifan Zhou, Xin Li Categories: cs.AI, cs.CL

The Gap

Existing LLM agents for machine learning engineering (like OpenHands, AIDE) treat each search branch independently—when one exploration path discovers something useful, that knowledge stays siloed. They also lack memory across tasks, starting from scratch every time. For long-horizon problems like discovering novel ML algorithms or optimizing pipelines over 12+ hours, this memoryless, isolated search becomes wasteful. Prior methods either explore too broadly without convergence (pure Monte Carlo) or exploit too narrowly without discovering fundamentally new approaches (greedy search).

Problem: Long-horizon ML algorithm discovery
         |
         v
Existing agents' limitations:
  - Tree search branches don't share info --> wasted exploration
  - No memory across tasks --> repeated failures
  - Flat search control --> can't shift explore/exploit balance
         |
         v
Assumption: Cross-branch knowledge + accumulated experience
            + progressive search schedule = better discovery
         |
         v
Method: MLEvolve
  [Graph-based tree search] + [Retrospective Memory] + [Hierarchical control]
         |
         v
Evidence: MLE-Bench (medal rate), AlphaEvolve comparison (math optimization)
         |
         v
Conclusion: SOTA on ML engineering benchmarks + cross-domain generalization

The Increment

One sentence: Before MLEvolve, LLM agents searched for ML solutions in isolated trees with no memory; after, they search in knowledge-sharing graphs with evolving experience, achieving competitive performance in half the standard time.

Core Mechanism

MLEvolve has three interlocking components. First, Progressive MCGS (Monte Carlo Graph Search) extends traditional tree search by adding reference edges between branches—when one branch discovers a useful technique, other branches can access it through these cross-links. The search starts with high-entropy exploration (try many diverse approaches) and gradually shifts to low-entropy exploitation (refine promising ones) following an annealing schedule.

Second, Retrospective Memory stores two kinds of knowledge: a cold-start knowledge base (general ML patterns) and a dynamic global memory (task-specific learnings from previous runs). When starting a new search branch, the agent retrieves relevant past experiences—both successes and failures—to guide decisions.

Third, Hierarchical Control decouples strategy from execution. A planner agent decides what to try next (which algorithm family, which hyperparameters) while separate coder agents implement those decisions using adaptive coding modes (direct generation vs. retrieval-augmented generation vs. chain-of-thought). This separation prevents low-level coding failures from derailing high-level strategy.

Input Task
    |
    v
[Planner Agent] <----- [Retrospective Memory]
    |                   /              \
    v                  /                \
Strategy Tree     Cold-start KB    Dynamic Memory
    |                                (past runs)
    +----+----+----+
    |    |    |    |
    v    v    v    v
 Branch1 Branch2 Branch3 ...
    |    |    |    |
    *====*====*====*  <-- Reference edges (graph structure)
    |
    v
[Coder Agents] (adaptive modes)
    |
    v
Generated Code --> Execution --> Feedback
                                    |
                                    +---> Update Global Memory

Think of MLEvolve like a research lab with multiple PhD students working on related problems. Each student (branch) pursues their own approach, but they regularly share findings in group meetings (reference edges). The lab keeps a library of textbooks (cold-start KB) and a wiki of past project notes (dynamic memory). A senior advisor (planner) assigns research directions but doesn’t micromanage implementation details. Early on, the advisor encourages diverse explorations—try neural nets, try ensemble methods, try different feature engineering. As deadlines approach, the advisor narrows focus to the most promising directions and pushes for refinement. When a student gets stuck on coding, they can choose their own method: write from scratch, adapt code from the wiki, or think through the logic step-by-step before coding.

Key Concepts

• Progressive MCGS (Monte Carlo Graph Search): Imagine searching for a hiking trail in a mountain range. Traditional tree search is like sending explorers down different valleys, but they never talk to each other—if one finds a waterfall, others might waste time searching for water elsewhere. Graph search adds walkie-talkies and shared maps. The “progressive” part means you start by sending explorers in all directions (high entropy) to survey the terrain, then gradually focus resources on the most promising paths (low entropy) as you learn more. The entropy schedule controls this shift: early on, 80% of compute goes to exploration; by the end, 80% goes to refining the best route found so far.

• Retrospective Memory: This is a two-tier filing system. The cold-start KB is like a cookbook—general recipes that work across many dishes. The dynamic global memory is like your personal cooking journal—notes on what worked last Tuesday when you made risotto, what burned, which substitutions succeeded. Before starting a new task, the agent queries both: “Show me standard approaches for tabular classification” (from KB) and “What did I learn when I worked on the credit-default dataset last week?” (from memory). This prevents rediscovering known failures and accelerates convergence by starting from informed priors rather than random guesses.

• Adaptive Coding Modes: Not all code generation requires the same approach. When implementing a standard algorithm (logistic regression), direct generation from the LLM is fast and reliable. When tackling something unusual (custom loss function for imbalanced data), retrieval-augmented generation helps—fetch similar past implementations and adapt them. When debugging subtle failures (why does validation accuracy plateau?), chain-of-thought reasoning works better—explicitly reason through the logic before writing fixes. The coder agents switch between these modes based on task complexity and available examples, like a programmer choosing between copying from Stack Overflow, reading documentation, or whiteboarding the problem.

Framework Shift

Before (isolated tree search):          After (graph search + memory):

Task --> Agent                          Task --> [Retrospective Memory]
           |                                            |
           v                                            v
      Tree Search                              [Planner Agent]
      /    |    \                                      |
     /     |     \                                     v
  Branch Branch Branch                         Progressive MCGS
    |       |       |                          (graph structure)
    v       v       v                          /    |    \    \
  Code    Code    Code                        /     |     \    \
    |       |       |                     Branch--Branch--Branch--Branch
    v       v       v                       |       |       |       |
  Result  Result  Result                    *=======*=======*=======*
                                             |   (shared knowledge)  |
  (no cross-talk,                            v                       v
   no memory,                             Code (adaptive)         Code
   flat explore/exploit)                     |                       |
                                             v                       v
                                          Result --> Memory Update

One sentence: From isolated tree branches searching blindly, to knowledge-sharing graph nodes that learn from accumulated experience and progressively focus effort.

Expert Assessment

Problem choice: This addresses a real bottleneck. Long-horizon agent tasks genuinely suffer from the isolation and amnesia the authors identify. The problem sits squarely in the current trajectory of applying LLMs to complex, multi-step engineering tasks—relevant and timely.

Method maturity: The core ideas (graph search over trees, experience replay, hierarchical control) are well-established in classical AI and reinforcement learning. The contribution here is thoughtful integration rather than algorithmic novelty. That said, the progressive entropy schedule is a clever detail, and the separation of planning from coding is practical. Nothing groundbreaking, but solid engineering that solves the stated problems.

Experimental integrity: The MLE-Bench results are encouraging—achieving comparable performance in half the time is meaningful. However, the baselines are mostly prior LLM agent systems, not specialized AutoML tools like AutoGluon or FLAML. The AlphaEvolve comparison is good for demonstrating generalization, but it’s a single domain (mathematical optimization). More diverse baselines and ablation studies would strengthen confidence. The paper doesn’t deeply analyze failure modes or edge cases.

Writing quality: The abstract and introduction are clear. The method section becomes dense—too many moving parts introduced simultaneously without sufficient visual aids. The results section focuses on aggregate metrics (medal rate, submission rate) but doesn’t dig into qualitative analysis: which types of tasks benefited most from cross-branch sharing? Where did memory retrieval fail? A deeper case study would elevate the work from “here’s our system and numbers” to “here’s what we learned about agent design.”

Verdict: Weak accept — Solid engineering contribution with practical improvements on a relevant problem, but lacks conceptual depth and thorough experimental analysis that would make it a strong accept.

Takeaways

• Progressive entropy scheduling: If you’re building any iterative search system (hyperparameter tuning, architecture search, prompt optimization), consider starting with high-variance exploration and gradually annealing to low-variance exploitation. The paper’s schedule (entropy decreases linearly from 0.8 to 0.2 over the search budget) is a concrete starting point you can adapt.

• Hierarchical agent decomposition: Separate strategic decision-making from tactical execution. If your agent keeps getting derailed by implementation details, split it: one planner that decides “what to try next” and separate executors that handle “how to implement it.” This prevents low-level failures from corrupting high-level reasoning.

• Cross-branch knowledge sharing: If you’re running multiple search branches (A/B testing, ensemble methods, parallel experiments), don’t treat them as independent. Add explicit mechanisms for branches to share intermediate findings—even a simple shared cache of “what worked” and “what failed” can reduce redundant exploration.

• Dual memory system: Combine static domain knowledge (cold-start) with dynamic task-specific memory (accumulated experience). When starting a new task, query both to build informed priors. This pattern applies beyond ML agents—useful for any system that faces recurring but slightly varied tasks (customer support bots, code review assistants, diagnostic systems).