BrowserForge: 203,238 Trajectories, One Per Website

Paper: 2608.24848 Authors: Fei Tang, Huawen Shen, Zhiqiong Lu, Zhengxi Lu, Pengyuan Lyu, Chengquan Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen Categories: cs.CL

The Gap

Web agents that act from rendered pixels — screenshots in, clicks out — avoid two real problems: the brittleness of parsing a page’s HTML, and the token cost of reading an accessibility tree. But they pay for it in data. Learning to act from pixels requires large numbers of high-quality interaction trajectories, and how to produce that data at scale is genuinely open.

The reason is a coverage ceiling rather than a volume one, and it is worth stating precisely. Public datasets contain only a few thousand trajectories, drawn from a fixed, narrow set of websites. Recent automated synthesis pipelines improved the throughput but stayed bound to predefined site lists or tutorial sources. So the pipeline can generate more episodes, but the number of distinct websites the agent ever sees barely grows.

That is the binding constraint. A web agent’s difficulty is largely a function of how many different sites it has encountered — layouts, widget conventions, login patterns, navigation idioms. A thousand more trajectories from the same fifty websites mostly re-teach what the model already knows. The field was scaling episodes while holding diversity constant.

   PIXEL-BASED WEB AGENTS: THE DATA PROBLEM

   why pixels? avoids:
     - brittleness of HTML parsing
     - token cost of accessibility trees
        |
        v
   but requires: large volumes of high-quality
                 interaction trajectories
        |
        v
   [THE REAL CEILING IS COVERAGE, NOT VOLUME]
     public datasets: a few thousand trajectories
                      from a FIXED, NARROW site set
     automated pipelines: more throughput, but bound to
                      PREDEFINED site lists / tutorials
        |
        v
   more episodes from the same ~50 websites
      -> mostly re-teach known patterns
      -> distinct websites seen by the agent barely grows
        |
        v
   [GAP] scale in episodes, constant in diversity

The Increment

One sentence: Before this paper, web-agent data pipelines scaled the number of episodes while the number of distinct websites stayed flat; after it, parallel browser sandboxes over the open web produce 203,238 trajectories each from a distinct website, and a compact model fine-tuned on them improves on the live web.

Core Mechanism

Three components, and the first one is what changes the diversity ceiling.

Open-web sourcing. Rather than expanding a curated list, the framework exposes the agent to hundreds of thousands of real, openly reachable websites. This is the structural break from prior work: diversity stops being something a human maintains as a list and becomes something the open web supplies. It is also why the constraint moves from coverage to engineering — the sites now exist in abundance, and the problem becomes driving browsers at them reliably.

A sandbox cluster manager. Scheduling hundreds of concurrent browsers at high utilisation. This is the throughput component, and the emphasis on utilisation is the right one: browser automation is slow and failure-prone, so the practical yield per unit of wall-clock time depends on how well the cluster keeps working rather than on how many sandboxes are nominally available.

A Proposer-Solver dual-agent loop. One agent turns a raw page into an executable task; the other collects a verified trajectory for it. The two roles are separated because they are different jobs — inventing a task that is feasible on this particular page, and actually completing it — and the verification step is what keeps spurious trajectories out. A rule-plus-model cleaning pipeline then removes failed runs and rewrites the surviving reasoning into a single unified chain-of-thought style, which matters for training consistency: trajectories produced by different agents would otherwise carry different reasoning formats.

One design decision deserves attention because it inverts the usual framing. Page structure such as the accessibility tree is used only as a synthesis-time signal; the released agent acts purely from the screenshot. The accessibility tree helps the pipeline understand a page well enough to construct tasks and verify completions, but it is deliberately excluded from the trained agent’s input. That keeps the agent’s advantage — no HTML parsing, no tree reading at inference — without giving up the structural information where it is cheap.

The results follow the diagnosis. The corpus is 203,238 trajectories, each collected from a distinct website — larger and more diverse than prior trajectory datasets. Fine-tuning a compact multimodal model on it raises success rate on live Online-Mind2Web from 25.66% to 33.33%, and consistently improves step accuracy on static Multimodal-Mind2Web, with the gain growing as the corpus scales. Controlled analyses confirm that open-web sourcing and broad website coverage are the key contributors — which is the check that matters, because it verifies that diversity, not just volume, is what moved the number.

   BROWSERFORGE

   [1] OPEN-WEB SOURCING
       hundreds of thousands of real, openly
       reachable websites
       -> diversity comes from the web, not a list
                    |
   [2] SANDBOX CLUSTER MANAGER
       hundreds of concurrent browsers,
       scheduled for HIGH UTILISATION
                    |
   [3] PROPOSER-SOLVER DUAL-AGENT LOOP
       Proposer: raw page -> executable task
       Solver:   task -> VERIFIED trajectory
                    |
                    v
   rule + model cleaning: drop failed runs,
   rewrite surviving reasoning into ONE unified CoT style
                    |
                    v
   CORPUS: 203,238 trajectories, each from a DISTINCT site
                    |
                    v
   fine-tune compact multimodal model
     Online-Mind2Web (live):  25.66% -> 33.33%
     Multimodal-Mind2Web:     step accuracy improves,
                              gain grows with corpus scale
   CONTROLLED: open-web sourcing + broad site coverage
               are the key contributors

   NOTE: accessibility tree used ONLY at synthesis time;
         the released agent acts from the SCREENSHOT alone

Think of it as teaching someone to navigate cities by only ever driving them around their own neighbourhood. They can get very good at their neighbourhood — every shortcut, every one-way street — and if you measure them there, they look competent. Send them to a city with different conventions and the competence evaporates, because what they learned was a specific place, not the skill of navigating. Generating ten thousand more trips through the same neighbourhood does not help; the only fix is to drive somewhere new, which is why the paper’s contribution is the sourcing stage rather than the throughput. The sandbox cluster is what makes “somewhere new” affordable at hundreds of cities in parallel, and the Proposer-Solver split is how each new city yields a usable lesson rather than a failed outing.

Key Concepts

• Coverage ceiling versus volume ceiling: more trajectories from the same sites re-teach known patterns, so diversity — not episode count — is the constraint that binds. Distinguishing these is what redirects effort from generation throughput to sourcing.
• Synthesis-time structure, inference-time pixels: using the accessibility tree to build and verify tasks while excluding it from the trained agent’s input. It gets the structural signal where it is cheap and keeps the deployment advantage of acting from screenshots.
• Proposer-Solver separation with verification: dividing task invention from task completion, and requiring the trajectory to be verified. The two are different problems, and the verification step is what prevents a fluent but incorrect trajectory from entering the corpus.
• Unified reasoning format for training: rewriting surviving trajectories into one chain-of-thought style. Mixed formats across agents would otherwise inject a formatting confound into fine-tuning, obscuring what the model actually learned.

Framework Shift

Before (scale episodes, fixed site list):
  curated list of sites / tutorials
  -> generate more trajectories per site
  -> distinct websites seen: barely grows
  -> measured gains: bounded by coverage

After (open-web sourcing, parallel sandboxes):
  hundreds of thousands of openly reachable sites
  + cluster manager (hundreds of concurrent browsers)
  + Proposer-Solver with verification
  -> 203,238 trajectories, each from a DISTINCT site
  -> live Online-Mind2Web 25.66% -> 33.33%
  -> gain GROWS with corpus scale

From generating more experience in known places, to generating experience in places the agent has never been, the core shift is that for pixel-based agents the scarce input is diversity of environment, not quantity of episodes.

Expert Assessment

Problem choice: Very good, and correctly diagnosed. Recognising that the field’s data problem was coverage rather than volume is the insight that makes the rest work, and the observation that automated pipelines inherited the curated-list assumption is a fair reading of why the ceiling persisted.

Method maturity: The engineering is the contribution and it is substantial — sourcing over the open web, scheduling hundreds of concurrent browsers at high utilisation, and generating verified tasks per page is a real system, not a prompt. The Proposer-Solver split with verification is the right structure, and the decision to keep the accessibility tree out of the trained agent’s input while using it during synthesis is a well-judged asymmetry. The main caveat is that open-web sourcing raises questions the paper treats lightly: site availability, robots directives, and the composition of what “openly reachable” yields are all part of the data distribution a deployed agent will face.

Experimental integrity: The controlled analyses are the strongest element, because showing that open-web sourcing and broad coverage are the key contributors is what separates this from “we generated more data and the number went up.” Improvement on live Online-Mind2Web — as opposed to a static split — is the harder test and the more convincing one, and reporting that the gain grows with corpus scale is a useful scaling signal rather than a single point.

Writing quality: The diagnosis is laid out clearly and the three components each map onto a stated constraint, which makes the system legible. What would elevate the paper is a diversity analysis of the corpus itself: a distribution over site categories, and a comparison against prior datasets on distinct-site count, would let a reader see the coverage claim directly rather than infer it from performance.

Verdict: strong accept — it identifies the binding constraint as diversity of environment, builds the system that removes it, and verifies with controlled analyses that diversity is what produced the gain.

Takeaways

• For agent training data, count distinct environments, not episodes. A pipeline that generates more of the same is scaling the wrong axis.
• Get structural information where it is cheap and exclude it where it is expensive. Using the accessibility tree at synthesis time while training purely from screenshots preserves the deployment advantage.
• Separate inventing a task from completing it, and verify the result. Unverified trajectories enter the corpus silently, and a fluent wrong answer is harder to detect later than a missing one.
• Normalise the reasoning format before fine-tuning. Trajectories from different generators carry different formats, and that difference will compete with the signal you are trying to learn.