StagedWorkspace: The Agent Is Editing a File It Never Actually Read

Paper: 2608.18050 Authors: Yining Hua, Hongbin Na, Yifan Zhou, Akshay Kalose, Cyrus Ayubcha, Levi Lian Categories: cs.AI

The Gap

Software-engineering agents solved a problem they never had to name. SWE-bench hands the agent a repository checkout; SWE-agent gives it patch-oriented tools over that same checkout. Search, edit, diff and test all resolve against one state, and everybody — agent, grader, human reviewer — agrees on what “the current version” means. That agreement came free with git and with the fact that source code is line-addressable text.

Move to PDFs, spreadsheets, slide decks, notebooks and mixed-format project folders, and the free lunch ends. Office agents typically run a parser that flattens files into searchable text, and that parsed cache is a copy. The agent searches the copy, opens the original, edits the original, and submits the original. Nothing in that loop forces those four things to refer to the same version. The paper’s framing of the failure is blunt and correct: an agent can retrieve a column definition from one workbook version, edit a different version in its sandbox, and hand in a deliverable that contradicts the evidence it cited.

The authors trace this to three interface trade-offs the field has been picking between rather than resolving. Artifact-only harnesses keep the true files but make search miserable — the agent pages through 700 PDFs burning context. Parsed-only harnesses give you search but lose layout hierarchy, formulas and anything visual, and you cannot execute a parsed spreadsheet. And an unversioned mutable workspace lets an agent overwrite or delete files with no durable diff for anyone to review. StagedWorkspace’s move is to name the missing thing — a workspace-state contract — and then argue it is not a comfort feature but a measurable experimental variable that current benchmarks leave uncontrolled.

[PROBLEM] agent searches a parsed copy, edits the native file,
          submits a third thing. no rule ties them together.
     |
     v
[WHY IT PERSISTS] coding agents inherited the contract from
                  git + line-addressable text. binary office
                  files have no equivalent, so harnesses pick
                  ONE view and eat the other's weakness
     |
     v
[CLAIM] every view (search / edit / review / submit) should
        name the workspace version it exposes or modifies
     |
     +----------------+------------------+
     v                v                  v
[W_t native]    [C_t parsed]        [D_t = diff(W_0, W_t)]
 authoritative   hash-keyed          journaled review
 execute+submit  stale-markable      surface
     |                |                  |
     +----------------+------------------+
                      v
[EVIDENCE] fixed-harness paired ablations on OfficeQA Pro
           (~697 Treasury PDFs) and APEX-Agents (480 rubric
           tasks, ~166 files each). dual view beats the worse
           single view by 8.3-12.1 Pass@1 / 4.7-9.2 rubric pts
                      |
                      v
[CONCLUSION] workspace state is an experimental variable, and
             benchmarks that ignore it confound interface with
             model capability

The Increment

One sentence: Before, “give the agent office files” was an unexamined plumbing decision made differently by every harness; after, it is a controlled variable with paired ablations attached, plus a concrete mechanism — content-hash-keyed parsed records — that keeps the searchable view honest about which version it is showing.

Core Mechanism

The state model is three lines and genuinely is the whole idea. At a synchronization point t, taken after each batch of tool calls, the workspace is W_t (the current native files, authoritative for execution and submission), C_t (parsed records, each tagged with the source path and content hash of the file version it was parsed from), and D_t = delta(W_0, W_t) (the review diff against the starting state). Parsed tools read C_t. Native tools read and edit W_t. Review tools expose D_t. Crucially, C_t and D_t are derived views, not independent document copies — there is exactly one authoritative artifact tree.

The synchronization rule is the part worth stealing. After every mutating tool batch, the harness hash-scans the sandbox. Files whose hashes match reuse their cached parsed records untouched. Files whose hashes changed have their parsed records marked stale immediately and queued for asynchronous re-parsing. The freshness label is agent-visible: a parsed read during the re-parse window still returns the old value, but it returns it stamped stale rather than silently pretending to be current. That distinction — wrong-and-labeled versus wrong-and-confident — is the entire safety story, and it is why the design does not need re-parsing to be fast. Underneath, the workspace also keeps an accepted-artifact tree and a staged-change journal, which is what lets it detect stale edits, hold pending changes for review, and promote or roll back.

The diff surface is deliberately format-specific rather than universal: text files get line diffs, spreadsheets expose row- and cell-level changes, slide decks expose slide-level changes, and unsupported binaries fall back to before/after previews. This is the pragmatic concession to the fact that git diff has nothing to say about a .pptx.

  turn loop, one synchronization point:

  [ sandbox hydrated from W_t ]
            |
            v
  ReAct: reason -> tool batch -> observe    (budget: 250 calls)
            |
            v
  +--------------------------------------------------+
  |  HASH SCAN over sandbox                          |
  |                                                  |
  |  report.pdf    hash match     -> reuse record    |
  |  budget.xlsx   hash MISMATCH  -> mark stale,     |
  |                                  queue re-parse  |
  |  memo.docx     hash match     -> reuse record    |
  +--------------------------------------------------+
            |
            v
        W_t     -> W_(t+1)
        C_(t+1)  = sync(C_t, W_(t+1))
        D_(t+1)  = delta(W_0, W_(t+1))
            |
            v
  three agent-facing views over ONE state:

     parsed C        native W          review D
   search, grep,   open, render,    workspace_diff,
   parsed read     execute, edit    file_diff
        |               |                 |
        +-------+-------+--------+--------+
                v                v
          evidence used   ==  artifact submitted


  the stale window, concretely (cell B12: 4.2 -> 4.6):

    before edit      re-parse pending      after re-parse
    -----------      ----------------      --------------
    source  4.2      source  4.6           source  4.6
    cached  h_before cached  h_before      cached  h_after
    read -> 4.2      read -> 4.2           read -> 4.6
    [current]        [STALE]               [current]
                        ^
                        +-- wrong value, but *labeled* wrong

The metaphor: a construction site with as-built drawings. The building under construction is W_t — the only thing that actually exists and the only thing that gets handed over. The drawing set pinned up in the site office is C_t: a flat, searchable, indexable representation you consult precisely because walking the whole building to find one dimension is absurd. The punch list is D_t: what changed between the day work started and today.

The failure mode StagedWorkspace attacks is the one that ruins real projects. Somebody moves a wall on Tuesday. The drawing in the office still shows the wall where it was, and it looks perfectly authoritative — clean lines, title block, revision number. On Thursday a subcontractor reads that drawing, orders ductwork to the old dimension, and the mistake is not discovered until handover. Nobody lied; the drawing simply belonged to a version of the building that no longer exists.

The content hash is the revision stamp. Every drawing carries the revision of the building it was traced from, and a foreman comparing stamps can tell in one glance whether it is live. When the wall moves, the drawing does not get quietly corrected later — it gets a big red SUPERSEDED stamp immediately, while the draftsman is still redrawing it. A subcontractor reading it during that window still sees the old wall, but sees the stamp too, and knows to go look at the building. The punch-list walkthrough before handover is the review-diff arm: you walk the client through every change against the original scope before you hand over the keys. And “artifact-only” versus “parsed-only” are the two bad site policies this replaces: burn the drawings and make everyone walk the building, or lock the building and let everyone work off drawings they cannot verify.

Key Concepts

• Workspace-state contract: A contract here means an obligation the interface takes on, not a legal document. The obligation is: every observation and every mutation must be able to name which version of the workspace it refers to. Coding agents have this without thinking about it — grep, your editor, and pytest all see the same working tree, so “the code” is unambiguous. The moment your search index is a separately-built artifact, “the file” splits into two nouns that can disagree, and the contract is what forbids that disagreement from being silent. The paper’s contribution is less the mechanism (content hashing is 1970s technology) than the insistence that this is a thing benchmarks must control rather than a thing each harness improvises.

• Hash-keyed cache invalidation, and why the stale label is the load-bearing part: The obvious engineering answer to “the cache is out of date” is “re-parse faster.” That is the wrong axis. Re-parsing a 200-page PDF is slow and always will be, so any system that relies on freshness-by-speed has a window where it is confidently wrong. StagedWorkspace instead makes staleness a first-class, agent-visible property. The value returned during the window is identical either way; what changes is whether the model knows to distrust it. A model that reads B12 = 4.2 [stale] can decide to open the native workbook; a model that reads B12 = 4.2 cannot. Invalidation is also per-record, keyed on path plus hash, so editing one workbook in a 166-file project folder does not nuke the index for the other 165.

• Dual access, and why “which single view is worse” flips between benchmarks: The two benchmarks stress opposite halves. OfficeQA Pro is find-and-read over ~697 Treasury Bulletin PDFs, so parsed search is the load-bearing tool and the weak arm is artifact-only (the agent can open PDFs but has no indexed way to locate the right table). APEX-Agents is deliverable-heavy professional work across cross-format project folders, so native execution is load-bearing and the weak arm is parsed-only (the agent finds the instructions but cannot run pandas on a parsed spreadsheet). The paper’s own trace example makes this crisp: on a HarFeast task, parsed search identifies what the survey columns mean, and native execution applies that reading to the same 3,000-row workbook that will be graded. Neither view alone completes the task, and that is the actual argument for dual access — not that more tools are better.

Framework Shift

Before (mainstream office-agent harness):   After (StagedWorkspace):

  pick ONE view, eat its weakness             ONE authoritative state W_t
                                                       |
  [ parser ] --build--> [ index ]              +-------+-------+
       ^                    ^                  v       v       v
       |                    |               parsed  native  review
    (once)              agent searches       C_t     W_t     D_t
       |                    |                  |       |       |
  [ native files ] <--edits-+                  +---+---+---+---+
       |                                           |       |
       | (index never told)                    every record carries
       v                                       the source HASH it
  [ submitted file ]                           came from
                                                   |
  search saw v1                            hash mismatch ==> STALE
  edit hit v2                              (visible to the agent)
  submit was v3                                    |
  ...and nothing noticed                           v
                                           search, edit, review and
  interface = plumbing detail              submit all name a version

                                           interface = measured variable

From “which view do we give the agent” to “which version does each view name,” the core shift is treating the workspace itself as state under contract rather than as a pile of files the tools happen to point at.

Expert Assessment

Problem choice: Real, and the diagnosis is better than the solution. The observation that the coding-agent world got a state contract for free from git — and that everyone else has been improvising it — is one of those things that is obvious the second someone says it and was apparently not said clearly before. The broader claim is the sharper one: if harnesses differ in an uncontrolled way on workspace access, then cross-harness leaderboard comparisons for office and workplace agents are partly measuring plumbing. The paper’s own Discussion makes this point about small-versus-large model gaps, and it deserves more attention than the system does.

Method maturity: The mechanism is deliberately boring, which I count in its favor, but let us be honest that the novelty is in the framing and the measurement, not the engineering. Content-hash-keyed cache invalidation with a stale flag is standard build-system practice; the derived-view discipline is standard database practice. The one genuinely thoughtful design choice is exposing staleness to the model instead of chasing freshness — that is the right call and the paper does not oversell it. What is missing is a comparison against the cheap alternatives: a naive mtime-based invalidation, or the dumbest baseline of all, re-parse-on-every-read for the file you just touched. On task folders of ~166 files where you edit two, I would like to know how much of the gain survives against that strawman. Also unaddressed: what happens when the parser itself is nondeterministic, so a re-parse of an unchanged-in-substance file produces a different record.

Experimental integrity: Mixed, and readers should know exactly which numbers to trust. The paired three-arm ablations are the good part — model, prompt, parser, retriever, grader, file tracker and 250-call tool budget all held fixed, 10,000-resample paired bootstrap over matched items, three attempts per task for the ablation models. That is careful work, and the authors are commendably explicit that only the ablations support causal reads. But the abstract leads with “63.9% versus published 29.3%,” and that is a non-paired cross-harness comparison against a row that used a different agent, retrieval backend and parser. The paper says so in the Limitations; the abstract does not. The honest headline is the paired one, and it is smaller: 8.3-12.1 Pass@1 on OfficeQA against artifact-only, 4.7-9.2 rubric points on APEX against parsed-only.

Two more things a careful reader should catch. First, the abstract’s phrase “the more limiting single view” is doing quiet work — the identity of that arm flips between benchmarks, so the range 8.3-12.1 and the range 4.7-9.2 are measured against different comparators. The paper is transparent about this in Section 4.2 and the reason is well-argued, but the abstract’s phrasing lets it read as a single consistent effect. Second, dual versus artifact-only on APEX has positive paired lifts for every model but bootstrap p values above 0.05, and several OfficeQA lifts have half-widths close to the effect (Gemini 3.1 Pro: +8.3 with a 7.9 half-width). The strongest claims are the ones with the smallest margins. The review-axis result is weaker still: 57 file-editing tasks, roughly 12% of APEX, reported without intervals or p values, with per-model lifts from +2.5 to +8.5 that do not obviously separate from noise at that n. The authors themselves call it a supporting mechanism rather than the whole system, which is the right posture.

Credit where due on two counts: the efficiency diagnostics genuinely head off the “you just spent more compute” objection — GPT-5.4 dual is essentially cost-matched with artifact-only ($0.84 vs$0.85 per question) while scoring higher — and the failure analysis is unusually unflattering to the paper’s own thesis. Reporting that 188 of 452 evaluated APEX tasks are all-arm zeros, meaning they do not move under any read-axis toggle, is exactly the number a promotional paper would bury.

Writing quality: The Methods section is clean and the state model is stated with admirable economy. The abstract is the weak link and is where the corners got cut: it foregrounds the least-controlled comparison, compresses two different ablation contrasts into one phrase, and omits that the review-axis evidence is a small unlabelled subset. Rewriting the abstract to lead with the paired ablations — which are the paper’s real achievement — would raise the whole thing, because the careful work is already done and is being undersold by a headline that invites the skepticism it does not deserve. Section 2 is also long for what it establishes; the entire related-work argument is “everyone has half of this,” and it takes four dense paragraphs to say it.

Verdict: weak accept — a genuinely useful reframing with careful paired ablations underneath, held back by an abstract that oversells with cross-harness numbers the paper itself declines to defend.

Takeaways

Things that transfer whether or not you care about office agents:

• Make staleness visible instead of racing to eliminate it. The generalizable move is admitting that any derived view — a search index, an embedding store, a summary cache, a materialized dashboard — has a window where it is wrong, and that labeling the window is cheaper and more robust than shrinking it. If your RAG index lags your source of truth, returning [possibly stale, source changed 4s ago] gives a model something to act on. Returning the stale value bare does not.

• Key cache records on content hash plus path, and invalidate per record. Sounds trivial until you notice how many agent stacks rebuild or trust the whole index. On a 166-file project folder where the agent touched two files, per-record invalidation is the difference between a 2-file re-parse and either a full rebuild or silent corruption.

• Make the diff a tool the model can call before it submits. Even the paper’s weakest result points somewhere useful: letting the agent see what it actually changed, right before handoff, is cheap to implement and the observed lift was largest for the weaker model. A diff proves nothing about semantic correctness, but it is a compact last chance to compare intended edits against actual ones.

• Format-specific diffs beat no diffs. Row/cell diffs for spreadsheets, slide-level diffs for decks, before/after previews for opaque binaries. Waiting for a universal answer to “what is a diff for a .pptx” means shipping nothing.

• For evaluation people, the real payload: if you are benchmarking agents on file-based work, publish which view of the workspace each arm had. Otherwise your leaderboard gap between a small model and a large one may partly be a gap between two parsers. The paper’s finding that non-frontier models gain most from a synchronized workspace, while frontier models barely move, is precisely the interaction that makes uncontrolled harnesses misleading.