Strategy Before Steps: Can an LLM Agent Plan Total Syntheses Like a Chemist?

Paper: 2608.07454 Authors: Daniel Armstrong, Xuan-Vu Nguyen, Octavian Susanu, Gabriel Gibberd, Théo A. Neukomm, Taddäus Strunden, Dan Forster, Morgane Delattre, Shawn Teh, Clément Rols Categories: cs.MA, cs.AI

The Gap

Computer-aided synthesis planning has been dominated for a decade by one architecture: a single-step retrosynthesis model (template-based like Synthia/Chematica and AiZynthFinder’s expansion policies, or template-free seq2seq/graph-edit models) wrapped in a tree search (MCTS, Retro*, A*-style best-first). You give it a target, it proposes one-step disconnections, recurses, and stops when every leaf is a purchasable compound. The reported numbers are excellent — solve rates in the high 90s on USPTO-190, Retro*-190, PaRoutes.

The problem is where those benchmarks come from. They are carved out of the same reaction databases (USPTO, Reaxys) that trained the models. So the evaluation asks: can you rediscover chemistry you were fitted to? Near-complete success on that question tells you almost nothing about the frontier. Complex natural products — densely functionalized, polycyclic, stereochemically loaded — are precisely the targets whose enabling steps are rare, bespoke, sometimes invented for that one molecule. Template libraries contain that chemistry least.

There is a second, subtler failure: these systems plan in the wrong unit. They reason one bond at a time, greedily, with no representation of strategy — no notion that this whole route hinges on a single transannular Diels-Alder, that a protecting group choice three steps back exists only to enable it, or that you want a fallback if the key step fails. Human chemists plan top-down: pick a strategic skeleton-forming disconnection first, then figure out how to get there. Nobody had shown a machine doing that convincingly.

PROBLEM
[ NP total synthesis needs inventive chemistry ]
[ template search covers only catalogued chemistry ]
                |
                v
DIAGNOSIS
[ benchmarks drawn from training corpus  ==>  circular evaluation ]
[ planning unit == single bond           ==>  no strategy layer   ]
                |
                v
ASSUMPTION
[ LLMs carry tacit strategic chemistry from text        ]
[ strategy is a better planning unit than the step      ]
                |
                v
METHOD
[ SynthEx : propose competing strategies > place key steps
             > fill routine steps > self-critique > revise ]
                |
                v
EVIDENCE
[ more convergent routes ] + [ wider reaction space ]
[ blinded experts: key steps ~ published human syntheses ]
[ SynthAtlas : 1000+ NP routes released ]
                |
                v
CONCLUSION
[ strategy-first agentic planning reaches targets
  that catalogue-based search cannot ]

The Increment

One sentence: Before, automated retrosynthesis produced route trees that experts skimmed and dismissed; after, an LLM agent produces plans whose *key steps experts mistake for and argue with as human work — and a thousand of them are now public for targets with no literature route.

Core Mechanism

SynthEx is an agentic loop, not a single model. The first stage is divergent: given a target structure, it proposes several competing strategies — high-level accounts of how the skeleton gets built, each committing to a different strategic disconnection and a different set of building blocks. This is the part existing tools have no vocabulary for. A tree search implicitly chooses a strategy by whichever branch scores best at depth one; SynthEx makes the strategy an explicit, nameable, comparable object before any step is committed.

The second stage is stratified execution. Within a chosen strategy, steps are split into key steps (the skeleton-forming, stereochemistry-setting, hard-to-find transformations that the route lives or dies by) and routine steps (functional group interconversions, protections, oxidation-state adjustments — chemistry that is well precedented and cheap to reason about). Reasoning effort is allocated asymmetrically: key steps get deep, targeted deliberation; routine steps get filled in as connective tissue. The route assembler then has to make these cohere — the routine steps must actually deliver the substrate the key step needs, with the right protecting groups and the right oxidation states.

The third stage is critique. The system reads back its own route and attacks it: chemoselectivity conflicts, stereochemical ambiguity, functional group incompatibility, missing protecting groups, implausible yields. Then it revises. This closes the loop that pure search architectures lack — search can only re-score branches, it cannot notice that a route is internally incoherent as a narrative.

                     [ TARGET STRUCTURE ]
                              |
              +---------------+---------------+
              |               |               |
        STRATEGY A      STRATEGY B      STRATEGY C
      (disconnect X)  (disconnect Y)  (disconnect Z)
              |               |               |
              +-------> [ COMPARE / SELECT ] <+
                              |
                              v
              [ DECOMPOSE INTO STEP TYPES ]
                     /                \
            KEY STEPS               ROUTINE STEPS
      (skeleton + stereo)      (FGI / protect / redox)
        deep reasoning            cheap precedent
                     \                /
                      v              v
                [ ROUTE ASSEMBLER ]
             checks substrate handoff
             checks PG + oxidation logic
                        |
                        v
                  [ CRITIC ]
        chemoselectivity . stereo . compatibility
                        |
              fails ----+---- passes
                |               |
                v               v
          [ REVISE ] ----> [ ROUTE + BACKUPS ]
             ^  |                   |
             +--+                   v
                              [ SynthAtlas ]

Think of it as a screenwriters’ room building a heist movie, and the mapping is tight. The target molecule is the vault. The competing strategies are competing loglines pitched in the room: go through the roof, bribe the guard, swap the truck — each a different theory of how the whole film works, argued *before anyone writes dialogue. The key steps are the set pieces: the two or three scenes the audience came for, the ones the director will spend half the budget on. The routine steps are the connective tissue — the car ride, the briefing, the walk down a corridor — necessary, but nobody agonizes over them, and they exist only to deliver the characters to the set piece in the right emotional and physical state. That handoff is the route assembler’s job: if your hero needs a keycard in the vault scene, an earlier scene has to have given it to him (that is what a protecting group is). The critic is the table read, where someone says “wait, he’s handcuffed in scene 4, how is he climbing in scene 6” — a continuity error, which is exactly what chemoselectivity and oxidation-state mismatches are. And the blinded expert panel is the test screening: not “did the plot obey the rules of screenwriting” but “did people sit forward and argue about it.”

A template-based tree search, in this metaphor, is a writer who only ever asks “what is the most common next scene given this scene?” — one shot at a time, from a corpus of produced films. It can assemble something shootable. It cannot pitch a heist.

Key Concepts

• Benchmark circularity: Imagine grading a student by giving them back the exact textbook exercises they studied, then reporting 97% and calling them a mathematician. Retrosynthesis benchmarks are extracted from USPTO/Reaxys; the models were trained on USPTO/Reaxys. Any route the literature already contains is, by construction, reachable by recombining catalogued templates. So high solve rates measure *coverage of the record, not synthetic ability. The tell is that performance collapses on natural products, where the record is thinnest — and that is exactly where a chemist would say the interesting problems live. This paper’s response is to change the measurement instrument (blinded human experts, novel targets without literature routes), not just the model.

• Key steps vs routine steps: In a 20-step total synthesis, maybe three steps decide whether the route works. Those are the ones that build the ring system, set the difficult quaternary stereocenter, or stitch two large fragments together. The other seventeen are bookkeeping: swap an alcohol for a bromide, protect a ketone, oxidize a primary alcohol to an aldehyde. Any competent chemist can do the seventeen; the three are where creativity and risk live. Existing planners treat all twenty as the same kind of object and spend the same search effort on each. SynthEx separates them and spends unequal effort — which is both a compute-allocation trick and a better model of how the problem is actually shaped.

• Convergent vs linear routes: A linear route is a relay race — you carry one growing molecule through every step, so a failure at step 18 costs you everything upstream, and total yield is a product of twenty numbers. A convergent route builds two or three fragments in parallel and joins them late, like assembling a car from a chassis and a body made on separate lines. Convergence is a hallmark of good human route design: shorter longest-linear-sequence, better yield economics, easier troubleshooting. The paper reports SynthEx’s chemistry is *more convergent than existing tools produce — a meaningful signal, because fragment coupling requires reasoning about two subgoals at once, which single-step greedy expansion structurally discourages.

Framework Shift

Before (mainstream CASP):            After (SynthEx):

     [ target ]                          [ target ]
         |                                   |
   one-step model                  +---------+---------+
   proposes edits                  |         |         |
    /    |    \                  strat A   strat B   strat C
   v     v     v                    |
  p1    p2    p3                    v  (explicit compare)
   |     |     |               [ chosen strategy ]
  ...   ...   ...                    |
   |                        +--------+--------+
   v                        |                 |
 [ buyable leaves ]     key steps        routine steps
                        (deep think)     (cheap fill)
   scored by                    \           /
   template priors               v         v
   + reachability            [ assemble + check handoff ]
                                      |
                                      v
                                 [ critic ]
                                      |
                              revise <+> emit route
                                          + backups

  unit of planning: A BOND       unit of planning: A STRATEGY
  evaluated vs: THE CORPUS       evaluated vs: HUMAN CHEMISTS
  failure mode: no route         failure mode: plausible-looking
                found            but wrong chemistry

From bottom-up recombination of catalogued reactions to top-down commitment to a strategy that is then filled in and criticized, the core shift is moving the planning unit up one level of abstraction, from the step to the plan — which also moves the failure mode from “cannot find anything” to “found something that reads well,” and that is why the evaluation had to change too.

Expert Assessment

Caveat up front: I am working from the abstract and framing, not the full methods and SI. Treat the numbers-level judgments below as priors about this class of paper, not verified readings of their tables.

Problem choice: Real gap, and a well-chosen one. The circularity critique of retrosynthesis benchmarks has been muttered at conferences for years — everyone knows 97% on USPTO-190 does not mean the problem is solved — but few groups were willing to give up the leaderboard and go argue with chemists instead. Natural products are the right stress test: they are where the record is thin, where strategy dominates, and where a human expert can still tell the difference in thirty seconds. Sitting this in the trajectory of the field: this is the paper that says the last decade optimized the wrong objective.

Method maturity: The insight is architectural and genuinely good — plan in the space of strategies, stratify effort between key and routine steps, close a critique loop. That is a real modeling decision about the shape of the problem, not a prompt trick. But the implementation is orchestration: LLM calls, roles, a critic, retries. There is little here that could not be rebuilt by a competent team in a few months, and the paper’s performance is likely bounded by the underlying model’s chemistry knowledge more than by the scaffold. Two things I would want to see and suspect are thin: an ablation isolating how much of the gain comes from the strategy-first stage versus simply having a strong base model with a critic, and whether a cheaper baseline — an LLM proposing strategic disconnections that then *seed a conventional template search — recovers most of the benefit. That hybrid is the obvious simpler alternative and it deserves to be the headline baseline.

Experimental integrity: The blinded expert assessment is the right instrument and the strongest part of the contribution. It is also the softest evidence in the paper, and both things are true at once. “Judged comparable to published human syntheses” depends entirely on panel size, chemist seniority, target selection, and how the comparison was framed; those details decide whether this is a finding or an anecdote. The phrase “engaged with them as genuine synthesis plans” is doing rhetorical work — engagement is a real and interesting signal, but it is not feasibility. The reaction-space coverage claim needs the most scrutiny: an LLM will happily propose transformations outside the template corpus, and “outside the corpus” is indistinguishable from “wrong” without expert or experimental adjudication. Novelty metrics reward exactly the behavior that hallucination produces. And the elephant: zero wet-lab validation. Nobody ran a step. Until someone executes even one SynthEx key step at the bench, the ceiling on what this paper can claim is “expert-plausible,” not “correct.” Releasing SynthAtlas is the honest move here — it hands the community the material to falsify the claim.

Writing quality: The abstract is polished to a Nature-style shine, and that is itself a warning sign — “a response algorithmic route prediction has not previously accomplished” is a sentence optimized for a press release, not a claim you can check. The corner-cutting is almost certainly in the evaluation protocol: I would bet the blinding procedure, panel composition, target sampling, and scoring rubric are compressed into a paragraph and a supplementary table when they carry the entire load of the central claim. Rewrite that section as the paper’s spine — with per-target expert scores, inter-rater agreement, and the raw comparison prompts — and this goes from persuasive to hard to dismiss. Second suggestion: report the failure cases in the main text. A paper claiming expert-level plans that does not show its worst outputs is asking for trust it has not earned.

Verdict: weak accept — the reframing (strategy as the planning unit, human experts as the measuring instrument) is correct and overdue, and the released atlas is a genuine community asset; but the evidence is expert opinion on an unstated sample with no bench validation, and the ablation that would show the strategy-first stage is load-bearing appears to be missing.

Takeaways

Concrete things worth stealing:

• Diagnose your benchmark’s provenance before believing your score. If the eval set and the training corpus come from the same generative process, a high number measures memorization coverage. The generalizable move: identify the sub-population where the corpus is thinnest (here, natural products) and make that your test set. This applies to code generation, theorem proving, and every “we get 95% on X” claim.
• Choose your planning unit deliberately. Most agent frameworks plan at the granularity of the primitive action because that is what the environment exposes. This paper’s win is refusing that default: propose several complete *strategies, compare them explicitly, then commit. Transfers directly to code agents (propose competing architectures before writing functions), to research agents (propose competing experimental designs before running one), to any long-horizon task where early commitments dominate the outcome.
• Stratify reasoning effort by step criticality. Split your plan into the few steps that decide success and the many that are bookkeeping. Spend deep inference on the former, cheap templates on the latter. This is a better compute-allocation policy than uniform search depth, and it requires only that you can classify steps — which is often easy even when solving them is hard.
• Make the critic check narrative coherence, not just local validity. Their critic looks for cross-step incompatibilities: does step 12 actually deliver what step 13 needs. Most self-critique loops in agent systems validate each step in isolation. The interesting bugs live in the handoffs.
• When your outputs exceed your benchmark, change the instrument. Blinded expert comparison against human artifacts, with “did they engage with it seriously” as an outcome measure, is a reusable evaluation design for any generative system that has outgrown automated metrics. Just budget for the rigor — panel size, blinding, inter-rater agreement — or it becomes the weakest link, as it arguably is here.
• Ship the artifacts. Releasing a thousand routes for targets with no literature route converts an unfalsifiable claim into a standing invitation to falsify it. That is the strongest credibility move available to a paper whose main evidence is subjective.