ArchAgent v2: When an Evolutionary Agent Out-Prefetches the Champion

Paper: 2608.09874 Authors: Abraham Gonzalez, Raghav Gupta, Akanksha Jain, Hanna Alam, Alexander Novikov, Po-Sen Huang, Matej Balog, Marvin Eisenberger, Sergey Shirobokov, Ngân Vũ Categories: cs.AI, cs.AR

The Gap

The FunSearch/AlphaEvolve line of work established a recipe: let an LLM propose code mutations, score them with a cheap deterministic evaluator, keep a population of winners, repeat. It works beautifully when the artifact is one self-contained function and the evaluator returns a number in milliseconds.

ArchAgent v1 pushed that recipe into computer architecture and won on cache replacement policies — a good fit, because a replacement policy is a single decision function at a single cache level, and the storage budget is easy to keep in your head. Prefetching breaks all three of those conveniences at once:

• The artifact is not one function. DPC4 asks for prefetchers at L1D, L2, and LLC. They interact: an aggressive L1 prefetcher changes the access stream the L2 prefetcher sees, which changes what the LLC prefetcher should do. The joint space is a product, not a sum.
• The budget is a hard constraint the LLM cannot see. A candidate that writes a 64K-entry table is a fine C++ program and a fantasy chip. Prior agentic loops treat this as a prompt instruction, which LLMs violate cheerfully.
• The evaluator is slow. ChampSim over a full trace suite is minutes to hours, and multi-core configurations are worse. Evolution speed is bounded by simulation latency, not by model quality.

The human baseline to beat is BertiGO, the hand-designed DPC4 winner — a strong, tuned, published artifact, not a strawman.

[Problem] 3 interacting prefetchers + hard storage budget + slow sim
     |
     v
[Assumption] cross-level coupling is weak enough that
             greedy per-level optimization is near-optimal
     |
     +---> [Method A] cascaded evolution:
     |                evolve L1 -> freeze -> evolve L2
     |                -> freeze -> evolve LLC
     |
     +---> [Method B] realizability loop:
     |                candidate code -> size estimator
     |                -> byte count fed back as signal
     |
     v
[Evidence] DPC4 rules, same traces, same budget:
           +3.8% geomean IPC over baseline
           +0.3% over BertiGO (the champion)
           +4.6% vs +2.6% on low-bandwidth single core
           multi-core: no win, sim latency kills it
     |
     v
[Conclusion] agentic search can beat a human champion on a
             real budgeted uarch task -- *if* you decompose
             the space and make the constraint checkable

The Increment

One sentence: Before this paper, agentic microarchitecture search could design one component at one cache level; after it, the same machinery designs a coordinated three-level prefetcher that fits a real hardware budget and narrowly beats the human champion.

Core Mechanism

The outer loop is standard evolutionary code search: a database of scored candidate programs, an LLM that reads a few of them plus their feedback and emits a mutated version, an evaluator that produces a fitness score. What v2 adds are two modifications to what the loop is searching over and what the evaluator says back.

Cascaded search turns one hard problem into three easier ones arranged in a sequence. The agent evolves only the L1D prefetcher first, with L2 and LLC held at a fixed reference (baseline or empty). Once that population converges, the best L1D design is frozen — its code becomes part of the immutable environment — and the search moves to L2, then repeats for LLC. Each stage sees a smaller mutation surface, so the LLM’s edits are more likely to be meaningful, and each stage’s fitness signal is less noisy because only one component is moving. The price is explicit: this is greedy coordinate descent over cache levels, so any design whose value only appears when two levels change together is unreachable.

The hardware-realizability feedback loop puts a size estimator inside the evaluator, next to the simulator. Every candidate is parsed for its declared state — tables, counters, history buffers — and a byte count is computed and returned to the agent in the *same feedback channel as the performance number. So the agent is not told “stay under the budget” once in a prompt; it is told “your design is 41 KB, the limit is 32 KB” after every attempt, which is a signal it can actually optimize against. Oversized candidates can be rejected or penalized rather than silently accumulating in the population as unimplementable champions.

                    +--------------------------+
                    |   candidate database     |
                    |  (code + score + size)   |
                    +--------------------------+
                        ^                  |
                sample  |                  | k parents +
                        |                  v feedback text
                    +--------------------------+
                    |    LLM mutation step     |
                    |  edits ONE level's code  |
                    +--------------------------+
                                 |
                                 v new candidate
              +------------------------------------+
              |             EVALUATOR              |
              |                                    |
              |  [size estimator] --> bytes, pass? |
              |         |  (fast, cents)           |
              |         v                          |
              |  if over budget: reject / penalize  |
              |         |                          |
              |         v                          |
              |  [ChampSim] --> IPC over traces    |
              |         (slow: the bottleneck)     |
              +------------------------------------+
                                 |
                                 v  score + size + notes
                    (back into database)

  STAGE CONTROLLER (the cascade):

  stage 1:  [ L1D: evolving ] [ L2: fixed ] [ LLC: fixed ]
                    |  converge, take best, freeze
                    v
  stage 2:  [ L1D: FROZEN   ] [ L2: evolving ] [ LLC: fixed ]
                    |
                    v
  stage 3:  [ L1D: FROZEN   ] [ L2: FROZEN ] [ LLC: evolving ]
                    |
                    v
             final three-level prefetcher

The metaphor: renovating a three-story building with the inspector standing next to you.

The building is the memory hierarchy; each floor is a cache level. The architect (the LLM) redraws blueprints; the occupancy trial (ChampSim) tells you how well tenants actually live in the result, but it takes days and you can only run so many.

v1 renovated a single-floor cottage, so none of this mattered. In a three-story building, letting the architect redraw all three floors at once produces incoherent drafts — every revision changes too much to learn anything from the trial. So you renovate floor by floor and seal each one: finish the ground floor, lock the door, move up. You lose the designs that require knocking out a shared wall between floors, and you accept that, because the alternative is never converging.

The second change is the inspector with a tape measure. Previously the architect was handed the building code at the start of the project and then drew whatever they liked; violations were discovered at the very end, wasting the whole trial. Now the inspector measures every draft immediately and says “this is 41 square meters over” before the trial ever runs. Cheap check first, expensive check second — and because the number comes back every time, the architect learns to design within the envelope rather than being nagged about it.

The admitted failure fits the metaphor too: multi-core is renovating a whole city block. The trial takes so long that the architect gets almost no feedback, and the search stalls.

Key Concepts

• Data prefetching, from zero: Main memory is slow — hundreds of cycles. If the CPU waits until it *needs a piece of data before asking for it, it stalls. A prefetcher watches the stream of addresses the program touches, guesses what it will touch next, and fetches it early so it’s already sitting in cache. Concretely: if a program reads array elements at addresses 100, 164, 228, a prefetcher spots the stride of 64 and pre-fetches 292. Real programs are messier — pointer chasing, hash lookups — which is why prefetcher design is a decades-old research sport. Guess wrong and you waste memory bandwidth and evict useful data, which is exactly why the low-bandwidth result (4.6% vs BertiGO’s 2.6%) is the most interesting number in the paper: it says the discovered design is unusually well-behaved about not wasting bandwidth.

• Why multiple levels make it a product, not a sum: The L1 prefetcher’s requests become part of the traffic the L2 prefetcher observes. If L1 already caught all the simple strides, the patterns remaining for L2 look completely different than they would in isolation. So you cannot design each level independently and staple them together — the *input distribution to level N+1 depends on your choice at level N. This is precisely why cascading (rather than parallel independent search) is the right decomposition: freezing L1 fixes the input distribution that L2 is being optimized against.

• Realizability as a feedback signal, not a prompt rule: There’s a general lesson here. Hard constraints given as instructions (“use at most 32 KB”) are weak — the model has no way to check itself. Hard constraints given as a *fast automated checker inside the evaluation loop become part of the optimization landscape. The size estimator costs essentially nothing compared to a simulation, so it doubles as a filter that saves the expensive evaluator from wasting cycles on fantasy hardware.

Framework Shift

Before (agentic uarch search, v1 style):     After (ArchAgent v2):

  prompt: "design a prefetcher,               stage-wise search:
   budget 32KB, here is the API"
           |                                   [L1] evolve --> freeze
           v                                        |
  +------------------------+                        v
  |  LLM edits everything  |                   [L2] evolve --> freeze
  |  L1 + L2 + LLC at once |                        |
  +------------------------+                        v
           |                                   [LLC] evolve --> done
           v
     [ ChampSim ] --> IPC                     each candidate first hits
           |                                  a cheap gatekeeper:
           v
   population of candidates,                    code -> [size est.]
   many silently oversized,                        |         |
   signal diluted across 3                     over? drop  under? simulate
   simultaneously-changing parts                             |
                                                             v
                                                        IPC + bytes
                                                     both fed back as text

From “search the whole space and hope the model respects the rules” to “search one axis at a time, and make every rule a number the model gets told after every attempt.”

Expert Assessment

Problem choice: A real gap, and a well-chosen one. Prefetching is genuinely the harder sibling of replacement policy — multi-component, budget-constrained, benchmark-standardized — so it is the natural next rung after v1. Using DPC4 rules with a published champion as the baseline is the right move: it makes the claim falsifiable in a way that most “LLM designs hardware” papers are not. Where it sits in the trajectory: this is the paper that tests whether agentic design scales past toy component counts, and the honest answer it gives is “to three components, yes; past single-core simulation cost, not yet.”

Method maturity: Clever framing, unremarkable machinery. Cascaded freezing is greedy coordinate descent, which architects have been doing by hand forever; the contribution is recognizing that it’s also the right *curriculum for an LLM evolutionary loop, because it shrinks the mutation surface and de-noises the fitness signal simultaneously. The realizability loop is even simpler — a static analyzer wired into the evaluator — and it is probably the more transferable idea precisely because it is so cheap. Two things a reviewer should press on: the cascade order is a hyperparameter (why L1 first? does LLC-first do better? an ablation here is mandatory, and the abstract doesn’t advertise one), and the greedy decomposition provably forfeits jointly-designed solutions, so some accounting of how much is lost — even a small two-level joint search as a reference point — would strengthen the claim considerably.

Experimental integrity: The headline is honest in a way I appreciate: +0.3% over BertiGO is stated plainly rather than dressed up, and the multi-core failure is admitted in the abstract. But 0.3% geomean on a benchmark suite that was *also the fitness function is exactly where I get nervous. Evolutionary search with the simulator score as fitness is, structurally, training on the test set. ChampSim is deterministic, so this isn’t statistical noise — it’s overfitting, which is worse, because it looks clean. The question that decides how much this paper is worth is: was there a held-out set of traces the search never saw? If yes, and the margin holds, this is a solid result. If the reported number is on the same traces that drove 12,000 evaluations, then +0.3% over a human design should be read as roughly a tie. The low-bandwidth single-core number (4.6% vs 2.6%) is more convincing to me than the headline, because a bandwidth-constrained configuration punishes exactly the overfitting failure mode — sloppy, speculative prefetching — so a large gap there suggests real behavioral quality, not curve-fitting. Two other gaps: “size estimation” of C++ state is a proxy for realizability that says nothing about timing, ports, or power; and there’s no compute-cost comparison — 12,000 simulated candidates versus one human team is a resource asymmetry that belongs in the paper, not just in the reader’s head.

Writing quality: The abstract signals the usual corner-cut: the *interesting section is the profiling of 12,000 candidates — how the agent actually explores, which prefetcher motifs it rediscovers (does it reinvent Berti-like delta learning? IP-stride? something genuinely unfamiliar?), where it plateaus, what fraction of candidates die on the size check — and that section is almost certainly the one given the least space. Rewriting it as the paper’s centerpiece, with concrete examples of the synthesized logic and a diff against known human designs, would move this from “we won a competition” to “here is what automated search teaches us about the prefetcher design space.” That’s the difference between a case study and a contribution. Secondary ask: describe the frozen-stage handoff precisely enough to reproduce, including what the non-evolving levels are set to during each stage.

Verdict: weak accept — a real, falsifiable result against a strong human baseline with honestly reported limits, held back by a margin thin enough that generalization beyond the fitness traces, not the method, decides whether it holds up.

Takeaways

Things worth stealing, in rough order of transferability:

1. Turn hard constraints into fast checkers inside the fitness function, never prompt text. If your agent must respect a budget (memory, latency, token count, API cost, code size), write a cheap static estimator and return its number in the feedback string every iteration. It does double duty: optimization signal *and a filter that stops the expensive evaluator from burning cycles on invalid candidates. This applies far outside architecture — any agentic search with a resource envelope.

2. Cascade-and-freeze for multi-component search. When the artifact has N interacting parts and your evaluator is expensive, evolving all N at once dilutes credit assignment into uselessness. Evolve one, freeze it, move on. Crucially, order the cascade so that each frozen stage *fixes the input distribution of the next — that’s why L1-then-L2-then-LLC is coherent rather than arbitrary. The same shape works for multi-stage data pipelines, compiler pass sequences, or chained model prompts.

3. Evaluator latency is the real ceiling of agentic search, not model capability. The multi-core failure here is the paper’s most useful negative result. If you’re building an agentic optimization loop, budget your engineering effort toward cheap proxy evaluators (shortened traces, sampled workloads, staged evaluation where only survivors get the full run) before you spend it on better prompts or bigger models.

4. Test discovered designs in the regime that punishes overfitting. The low-bandwidth result is the trustworthy one. Whatever your domain, find the configuration where sloppy-but-lucky solutions get penalized, and report that separately — it’s the cheapest available evidence that your search found structure rather than memorized the benchmark.

5. Treat the evolution log as a research artifact. 12,000 scored candidates with lineage is a dataset about how design spaces are shaped. Mining it for rediscovered motifs and plateau points is often more durable than the winning candidate itself.