Graph-as-Policy: Using Computation Graphs to Make Robots More Reliable for Variable Tasks

Paper: 2607.05369 Authors: Kaiyuan Chen, Shuangyu Xie, Letian Fu, Justin Yu, William Pacini, Sandeep Bajamahal, Hudson Kim, Jaimyn Drake, Daehwa Kim, Haoru Xue Categories: cs.RO, cs.AI, cs.CL, cs.LG

The Gap

Modern robotics has two main camps for building reliable systems. One camp, exemplified by Task and Motion Planning (TAMP) and the Robot Operating System (ROS), builds interpretable, modular programs from hand-coded skills. It’s great for “fixed automation” where everything is predictable, but it struggles with “Variational Automation” (VA) tasks—like picking differently shaped objects from a bin or opening varied handles—because its rigid graphs can’t adapt to the infinite variety of the real world. The other camp, model-free reinforcement learning (RL), learns end-to-end policies that *can adapt to variation, but they are black boxes, notoriously data-hungry, and often fail to meet the persistent, high-reliability standards of commercial and industrial settings. The gap is this chasm: how do you get the interpretability and structure of TAMP with the adaptability of model-free policies?

Problem: Rigid programs break under variation; flexible policies lack reliability and structure.
     |
     v
Assumption: Combining a structured skill library with a learning-to-plan mechanism can bridge this gap.
     |
     v
Method: GaP builds directed computation graphs from a skill library, then uses self-play in a generated simulation to optimize the graph's structure and parameters.
     |
     v
Evidence: On 8 new VA benchmarks (4 sim, 4 real), GaP significantly outperforms baselines in success rate and throughput.
     |
     v
Conclusion: Graph-as-Policy is a viable and effective framework for creating reliable, adaptable robot programs.

The Increment

One sentence: Before this paper, you had to choose between an interpretable but brittle program and an adaptable but opaque policy; after this paper, you can have a system that automatically generates, tests, and refines interpretable graph-structured policies that are both adaptable and reliable.

Core Mechanism

GaP operates as a multi-agent coding system, like a small team of AI programmers. Its foundation is MORSL (Modular Open Robot Skill Library), a curated library of reusable, high-quality robotic skill modules for perception, planning, and control. When given a task (e.g., “pick up the cup”), a “graph-generation agent” doesn’t write a monolithic script. Instead, it composes a directed computation graph, where each node calls a specific skill from MORSL (e.g., detect_object, plan_trajectory, execute_grip) and edges define the data and control flow between them. This is the initial, interpretable “program.”

This is where the magic happens. To handle variation, GaP doesn’t just run this program. A second “simulation-generation agent” analyzes the task and the initial graph, then procedurally generates a lightweight internal simulation environment that can instantiate many task variants (e.g., cups of different sizes, positions, orientations). A third “evaluation agent” then runs the current graph-policy across hundreds of these simulated variants in parallel, collecting success/failure data. Finally, a “refinement agent” analyzes this data and iteratively modifies the graph—adding nodes, changing parameters, or rewiring connections—to improve its overall success rate. This cycle of generate-simulate-evaluate-refine continues until the graph stabilizes.

[Task Description] --> [Graph-Gen Agent]
                             |
                             v
                      [Initial Graph-Policy]
                             |
         +-------------------+-------------------+
         |                                       |
         v                                       |
[Simulation-Gen Agent] --+                      |
         |                |                      |
         v                v                      |
   [Internal Sims]    [Eval Agent] --+           |
   (parallel, varied)      |         |           |
                           v         v           |
                     [Performance Data]          |
                           |                     |
                           v                     |
                   [Refinement Agent] ------------+
                           |
                           v
                   [Refined Graph-Policy] --> (loop back to sim gen)

Think of GaP as a high-stakes cooking competition for robot recipes. The MORSL library is your pantry, stocked with perfectly reliable basic techniques (chop, sauté, bake, plate). The graph is the recipe—a specific sequence of steps using those techniques to make a dish (the task). The problem is, you can’t just practice with one tomato; you need to make sure the recipe works with all possible tomatoes (different ripeness, size, shape). So, the simulation-generator is the competition producer, rapidly sourcing a huge variety of tomatoes (task variants) for you to practice with. You, the chef (evaluation agent), execute your recipe (graph) on each tomato variant, noting successes and failures. Finally, the refinement agent is your coach, who watches the tapes, spots that your knife angle is wrong for large tomatoes, and tells you to adjust that step in the recipe (modify the graph). You then practice again with a new batch of varied tomatoes. This iterative, simulated practice is how you end up with a single, robust recipe that works perfectly with any tomato.

Key Concepts

• Variational Automation (VA): This is the paper’s name for a crucial category of tasks that existing systems handle poorly. Imagine a factory where a robot must pack items into boxes. In *fixed automation, every item is identical and in the same spot. In variational automation, items are similar (all cups, all boxes) but differ in exact shape, size, orientation, or position on the conveyor belt. The “variation” is the key challenge. VA tasks require a system that generalizes, not one that memorizes a single perfect execution. GaP is designed specifically for this middle ground between rigid repetition and completely unstructured environments.

• Graph-as-Policy (GaP): This is the core architectural idea. Instead of thinking of a robot’s behavior as a single neural network (a policy) or a linear script, GaP represents it as a directed graph of modular operations. Each node is a reusable skill (a “verb” like “detect” or “plan”), and the edges define the flow of information and control. The beauty is that this representation is inherently interpretable—you can “read” the robot’s intended strategy—and it’s malleable. The learning algorithm doesn’t tweak a million weights in a black box; it restructures and re-parameterizes this explicit, human-readable graph.

Framework Shift

Before (mainstream approach):            After (this paper):
                                        
[Task] --> [Monolithic Policy (NN)]      [Task] --> [Graph Composer]
               |                                  |
               v                                  v
         [Opaque Weights]                [Directed Graph of Skills]
               |                                  |
               v                                  |
     [Direct Execution]                   [Sim Generator]
               |                                  |
               v                                  v
     (Fails on variation)                [Parallel Sim Trials]
                                                  |
                                                  v
                                          [Graph Refiner]
                                                  |
                                                  v
                                           (Robust Policy)

From learning a monolithic policy to composing and refining an interpretable computation graph, the core shift is in the unit of learning: it moves from optimizing inscrutable parameters within a fixed architecture to optimizing the explicit structure and parameters of a human-understandable program.

Expert Assessment

Problem choice: Excellent. The gap between brittle symbolic plans and unreliable black-box policies for real-world, variable tasks is a genuine and critical bottleneck for deploying robots in commercial settings. This paper squarely targets that chasm, which puts it on a very relevant trajectory in robotics.

Method maturity: Clever insight with a pragmatic, multi-agent twist. It’s essentially a form of graph-structure search guided by simulation performance. The reliance on a pre-existing, high-quality skill library (MORSL) is a strength for reliability but also a potential limitation for tasks requiring fundamentally new low-level skills. The multi-agent framing feels a bit like over-engineering the description of what is effectively a single optimization loop, but it doesn’t undermine the core idea.

Experimental integrity: The creation of 8 new VA benchmarks is a valuable contribution itself, providing a concrete testbed for the community. Comparisons against TAMP baselines (which are weak on VA) and model-free RL baselines (which are weak on reliability) seem fair. The real-world validation on 4 tasks adds significant credibility. The numbers (“significantly outperform”) are presented, but the absolute performance levels (e.g., 85% vs. 60%) and failure modes would need close inspection in the full paper.

Writing quality: The abstract is clear and sets the scene well. The description of the multi-agent harness is where the writing likely gets dense. The “simulation-generation agent” is the most novel part, and that section would benefit most from a rewrite with a clearer, more intuitive example walking through the process step-by-step. Explaining *how it generates diverse, useful simulations is key.

Verdict: strong accept — This is a smart, well-motivated synthesis of ideas from program synthesis, TAMP, and simulation-based learning that addresses a pressing practical problem with a novel and promising framework.

Takeaways

1. The “Graph-as-Policy” paradigm is highly transferable. For anyone building complex, multi-step AI systems (not just robotics), representing the system’s logic as a directed graph of modular components is powerful. It enables interpretability, easier debugging, and targeted optimization. This idea could improve workflow engines, data pipelines, or even complex agent architectures.
2. Self-play in procedurally generated environments is a potent recipe for robustness. The technique of automatically creating a distribution of challenge cases (the simulation generator) to test and stress-test a policy is a lesson in building reliable AI. Practitioners in any domain with variable inputs (e.g., web automation, logistics) should consider simulating their edge cases rather than just testing on real data.
3. Don’t throw away your structured priors; learn over them. The paper’s success comes from starting with high-quality, human-designed skills (MORSL) and using learning to *compose and adapt them, not replace them entirely. This hybrid approach—learning the structure atop reliable components—is a powerful pattern for building trustworthy AI systems.