Agent JIT Compilation: Compiling Web Automation Tasks into Parallel Code

Paper: 2605.21470 Authors: Caleb Winston, Ron Yifeng Wang, Azalia Mirhoseini, Christos Kozyrakis Categories: cs.LG, cs.AI

The Gap

Computer-use agents (CUA) like Browser-Use and OpenAI’s agent follow a fetch-screenshot-execute loop: take screenshot → call LLM → execute one action → repeat. This sequential pattern has two problems: (1) every action requires a full LLM roundtrip, causing high latency, and (2) the LLM generates raw tool calls without validation, leading to frequent errors from incorrect API usage (e.g., clicking before a page loads).

The core limitation is treating the agent as an interpreter that executes one instruction at a time, rather than a compiler that can reason about the entire task structure upfront.

Problem: Sequential execution + no validation
   |
   v
Assumption: Task structure can be analyzed before execution
   |
   v
Method: Compile task → validated code plan → parallel schedule
   |
   v
Evidence: 10.4x speedup (JIT-Planner), 2.4x speedup (JIT-Scheduler)
   |
   v
Conclusion: Compilation beats interpretation for web automation

The Increment

One sentence: Before this paper, web agents executed one action per LLM call in sequence; after, they compile tasks into validated code that parallelizes independent operations.

Core Mechanism

The system has three components working in sequence. JIT-Planner takes a natural language task and generates multiple candidate code plans (Python functions containing LLM calls, tool calls, and control flow). Each plan is validated against tool specifications—preconditions and postconditions that define valid usage patterns (e.g., “click requires element to be visible”). The planner selects the minimum-cost candidate based on estimated LLM and tool latencies.

JIT-Scheduler then analyzes the selected plan to find parallelization opportunities. It identifies independent operations (e.g., filling two unrelated form fields) and uses Monte Carlo simulation over learned latency distributions to estimate the cost of different execution orders. The scheduler outputs an optimized execution graph.

Finally, the tool protocol enforces invariants at runtime. Each tool (click, type, scroll) declares its preconditions and postconditions as executable checks. Before executing a tool call, the runtime verifies preconditions; after execution, it verifies postconditions. This catches errors early and provides structured feedback for plan regeneration.

Task description
   |
   v
JIT-Planner: Generate N plans → Validate → Select min-cost
   |
   v
JIT-Scheduler: Find independent ops → MC simulation → Optimize order
   |
   v
Runtime: Check preconditions → Execute → Check postconditions

Think of this like a restaurant kitchen. The old approach (sequential agents) is a single cook who reads one recipe step, performs it, then reads the next step—no prep work, no parallelization. JIT compilation is a head chef who reads the entire order, breaks it into prep tasks (chop vegetables, boil water, marinate meat), assigns tasks to different stations that can work in parallel, and validates each station’s output before the next step (e.g., “water must be boiling before adding pasta”). The head chef (planner) creates the execution plan, the sous chef (scheduler) optimizes station assignments, and each station (tool protocol) has a checklist to verify their work meets requirements.

Key Concepts

• Tool Protocol with Invariants: Instead of treating tools as black-box functions, each tool declares what must be true before it runs (preconditions) and what will be true after (postconditions). For example, click(element) has precondition “element must be visible on screen” and postcondition “element’s click handler has fired.” These are executable checks, not documentation. When a plan violates a precondition, the system catches it before execution and can regenerate the plan with feedback. This is like type checking for agent actions—it prevents a whole class of runtime errors by validating the plan structure upfront.

• Monte Carlo Cost Estimation: The scheduler needs to decide execution order for parallel operations, but tool latencies are stochastic (network delays, page load times vary). Instead of using average latencies, the scheduler samples from learned latency distributions and simulates thousands of execution traces. For each candidate schedule, it computes the distribution of total completion times and selects the schedule with the lowest expected cost. This accounts for variance—a schedule that’s fast on average but occasionally very slow might be worse than a slightly slower but more consistent schedule.

• Code Plan Validation: Generated code plans are validated against tool specifications before execution. The validator parses the Python AST, extracts all tool calls, and checks that each call’s arguments match the tool’s type signature and that the call appears in a context where its preconditions can be satisfied. For example, if the plan calls click(element) but never calls scroll_to(element) first, and element is off-screen, the validator rejects the plan. This is static analysis for agent code—catching bugs at compile time rather than runtime.

Framework Shift

Before (sequential interpretation):        After (JIT compilation):

Task --> LLM --> Action --> Screenshot     Task --> Planner --> Code Plan
         ^                      |                      |
         |                      |                      v
         +----------------------+                  Validator
                                                       |
    (repeat for each action)                          v
                                                  Scheduler
                                                       |
                                                       v
                                                  Parallel Execution
                                                  (with invariant checks)

From interpreting one action at a time to compiling the entire task into validated, parallelizable code, the core shift is moving reasoning from runtime to compile time.

Expert Assessment

Problem choice: Real gap. Sequential execution is a genuine bottleneck in current CUA systems—Browser-Use’s median task takes 60+ seconds. The problem sits at the intersection of LLM agents and systems optimization, which is timely given the recent push toward agentic workflows.

Method maturity: The planner and validator are straightforward applications of existing techniques (multi-candidate generation, AST parsing). The scheduler’s Monte Carlo approach is clever but not novel—similar methods exist in compiler optimization. The real contribution is recognizing that web automation is a compilation problem, not an interpretation problem. However, the paper doesn’t explore simpler baselines like caching plans for similar tasks or using a faster LLM for action selection.

Experimental integrity: Baselines are fair—Browser-Use and OpenAI CUA are state-of-the-art. The 5 web applications cover diverse interaction patterns (forms, search, navigation). However, the paper doesn’t report failure modes in detail. What happens when the planner generates invalid code that passes validation? How often does the scheduler’s cost model mispredict? The accuracy gains (+28%, +9%) are substantial but not broken down by error type (tool misuse vs. task misunderstanding).

Writing quality: The abstract and introduction are crisp. The method section is dense—Figure 2 (system architecture) does heavy lifting but isn’t explained incrementally. The related work section is thin and doesn’t position this against compiler optimization literature, which would strengthen the framing. The evaluation section could use a failure analysis subsection.

Verdict: weak accept — Solid systems contribution with clear practical impact, but the novelty is more in problem framing than algorithmic innovation.

Takeaways

Steal the invariant protocol pattern: If you’re building any agent system with tool use, declare preconditions and postconditions for each tool. Validate plans against these specs before execution. This catches a huge class of errors (wrong argument types, missing dependencies, invalid state transitions) at “compile time” instead of discovering them mid-execution. The implementation is straightforward—just Python decorators and AST parsing.

Compilation beats interpretation for structured tasks: If your agent’s task has analyzable structure (dependencies between actions, parallelizable subtasks), generate a full execution plan upfront rather than deciding each action on-the-fly. You get two wins: (1) validate the entire plan before committing resources, and (2) optimize the execution schedule globally rather than locally. This applies beyond web automation—think data pipelines, infrastructure provisioning, multi-step API workflows.

Monte Carlo scheduling for stochastic operations: When optimizing execution order under uncertainty, don’t use average latencies—sample from distributions and simulate. This is especially valuable when variance is high (network operations, LLM calls, user-facing systems). The overhead of simulation is negligible compared to the cost of a bad schedule.