Synthetic Computers at Scale: Training AI Agents in Simulated User Worlds

Paper: 2604.28181 Authors: Tao Ge, Baolin Peng, Hao Cheng, Jianfeng Gao Categories: cs.AI, cs.CL, cs.LG

The Gap

Existing agent training relies on short, context-free tasks. You ask an agent to “book a flight” or “summarize this document,” but real productivity work spans weeks, depends on accumulated context in your filesystem, and requires coordinating across multiple artifacts. Prior synthetic data methods generate isolated task episodes without the grounding environment where real work happens — your messy Downloads folder, that half-finished presentation from last week, the email thread buried in your inbox.

The boundary: we can train agents on toy tasks, but we can’t simulate the month-long project where success depends on remembering what’s in /projects/Q2_launch/drafts/ and coordinating with three colleagues across a dozen files.

Problem: Agents fail at long-horizon productivity work
   |
   v
Gap: No training data grounded in realistic user environments
   |
   v
Assumption: Synthetic computers + long simulations = rich training signal
   |
   v
Method: Generate 1000 computers -> Run month-long agent simulations
   |
   v
Evidence: 8+ hours runtime, 2000+ turns, performance gains in/out-of-domain
   |
   v
Conclusion: Scalable substrate for agent self-improvement

The Increment

One sentence: Before this paper, synthetic agent training happened in sterile task bubbles; after, we can generate entire user worlds with realistic filesystems and run month-long productivity simulations at scale.

Core Mechanism

The system has two phases. First, synthetic computer generation: create a realistic user persona, then populate a filesystem with hierarchical folders and content-rich artifacts (Word docs, Excel sheets, PowerPoints) that reflect that user’s work context. Think marketing manager with /campaigns/, /budgets/, /competitor_analysis/ folders, each containing plausible documents.

Second, long-horizon simulation: one agent (the “objective creator”) examines the synthetic computer and generates a productivity goal requiring multiple deliverables and roughly a month of human work. Another agent (the “worker”) then acts as that user, navigating the filesystem for context, coordinating with simulated collaborators, editing documents, creating new artifacts, until the objective is complete. This runs for 8+ hours of agent time, spanning 2000+ turns.

Synthetic Computer Generation:
   Persona -> Folder Structure -> Content Artifacts
      |            |                    |
      v            v                    v
   [Marketing] [/campaigns/]  [budget_Q2.xlsx]
                [/reports/]   [competitor_brief.docx]
                              [deck_draft_v3.pptx]

Long-Horizon Simulation:
   Objective Creator                Worker Agent
         |                               |
         v                               v
   "Launch Q2 campaign"  ->  [Navigate filesystem]
   "Need: deck, budget,       [Read context docs]
    email draft"              [Edit artifacts]
                              [Coordinate w/ team]
                              [Produce deliverables]
                                      |
                                      v
                              [2000+ turns, 8+ hrs]

Think of it like a flight simulator for productivity work. Real pilots train in simulators that replicate cockpit layouts, instrument panels, and flight conditions. Here, the “cockpit” is a user’s computer environment — the filesystem is the instrument panel, documents are the controls, and the “flight conditions” are the messy, context-dependent nature of real work. You don’t train a pilot by having them press random buttons; you simulate realistic scenarios where they must coordinate multiple systems over hours. Similarly, this method doesn’t train agents on isolated tasks — it drops them into a realistic user world and makes them complete a month-long project by navigating accumulated context, just like a real user would.

Key Concepts

• Synthetic Computer: Not just a folder tree, but a coherent user world. If the persona is a product manager at a SaaS company, the filesystem should contain /roadmaps/, /user_research/, /sprint_planning/ with realistic documents inside — not random Lorem Ipsum, but plausible content like “Q3 Feature Prioritization” with actual feature names, user quotes, and priority scores. The computer is “synthetic” because it’s generated, but “realistic” because it reflects how real professionals organize their work. The key insight: productivity work is strongly conditioned on environment. An agent can’t write a quarterly report without knowing what’s in /reports/2024_Q1/ or what the team discussed in /meeting_notes/.

• Long-Horizon Simulation: Most agent benchmarks are short: 10 turns, 5 minutes. This paper runs simulations for 2000+ turns over 8+ hours of agent runtime, equivalent to weeks of human work compressed. Why does this matter? Because real productivity objectives require sustained context. You can’t “launch a marketing campaign” in 10 turns — you need to draft the deck, get feedback, revise, coordinate with design, update the budget, write the email announcement. Each step depends on prior steps. Long-horizon simulation forces the agent to maintain context, recover from mistakes, and coordinate across artifacts, generating training signal that short tasks can’t provide.

• Experiential Learning Signal: The simulation produces trajectories showing how an agent navigates a complex environment to achieve a goal. These trajectories become training data. Unlike supervised learning (here’s input, here’s correct output), experiential learning captures the process: the agent tried reading /old_campaigns/2023_launch/ for inspiration, realized the budget format changed, navigated to /templates/, found the new format, then applied it. This process-level signal teaches not just what to do, but how to explore, recover, and adapt — skills that matter for long-horizon work.

Framework Shift

Before (mainstream approach):        After (this paper):

Task: "Book a flight"                Synthetic Computer:
  |                                    /home/user/
  v                                      /work/
Agent -> [Isolated action]  ->          /personal/
         [No context]                   /projects/Q2_launch/
         [Short episode]                  /drafts/
                                          /final/
Result: Works on toy tasks,              /meeting_notes/
        fails on real work
                                  Objective: "Complete Q2 launch"
                                    |
                                    v
                                  Agent -> [Navigate filesystem]
                                           [Read context]
                                           [Coordinate artifacts]
                                           [2000+ turns]
                                    
                                  Result: Learns grounded,
                                          long-horizon behavior

From isolated task episodes to grounded environment simulations, the core shift is training agents in the context where real work happens, not in a vacuum.

Expert Assessment

Problem choice: This is a real gap. The field has been stuck on short-horizon benchmarks (WebShop, ALFWorld) that don’t capture how professionals actually work. The insight that productivity is environment-dependent is obvious in hindsight but underexplored. The timing is right — we have capable enough base models to run these long simulations without collapsing immediately.

Method maturity: The approach is more engineering than algorithmic novelty, which is fine. The clever part is recognizing that persona abundance (billions of possible users) makes this scalable in principle. The brute force part is running 8-hour simulations per computer. I’d want to see ablations: does the realism of the synthetic computer matter, or would random folder structures work? How much does simulation length contribute vs. environment complexity?

Experimental integrity: The paper shows in-domain and out-of-domain improvements, which is good. But the baselines are unclear — what are they comparing against? Agents trained on short tasks? No training? The 1,000 computers is a proof-of-concept scale, not production scale. I’d want to see learning curves: does performance keep improving at 10K computers? 100K? The claim about “billions of synthetic user worlds” is aspirational, not demonstrated.

Writing quality: The abstract oversells (“foundational substrate”) before showing the evidence. The method section is clear, but the results section is thin — we get “significant improvements” without detailed breakdowns. The paper would benefit from a failure analysis: what kinds of objectives do agents still fail at after this training? Where does the method hit diminishing returns?

Verdict: weak accept — the core idea (synthetic computers + long simulations) is sound and addresses a real gap, but the execution feels like an early-stage proof-of-concept rather than a mature system. The experimental validation is promising but not thorough enough to justify the strong claims about scalability and foundational impact.

Takeaways

Steal the environment-first framing: If you’re building agents for any domain with accumulated context (customer support with ticket history, software engineering with codebases, research with paper libraries), don’t train on isolated tasks. Generate realistic environments first, then simulate work within them. The environment is not just context — it’s the substrate that makes long-horizon behavior possible.

Simulation length matters: The 2000+ turn, 8+ hour simulations aren’t just longer — they force qualitatively different behavior. Short tasks let agents get away with pattern matching; long tasks require memory, error recovery, and planning. If your domain has multi-step workflows, push simulation length until you see agents developing these capabilities.

Persona-conditioned generation scales: The insight that you can generate billions of diverse user worlds by varying personas is transferable. If you need diverse training data in any domain with user-specific context (personalized tutoring, custom workflows, individual preferences), condition generation on persona attributes rather than trying to manually create diversity.