Code as Agent Harness: When Programs Become the Infrastructure for AI Agents

Paper: 2605.18747 Authors: Xuying Ning, Katherine Tieu, Dongqi Fu, Tianxin Wei, Zihao Li, Yuanchen Bei, Jiaru Zou, Mengting Ai, Zhining Liu, Ting-Wei Li Categories: cs.CL, cs.AI

The Gap

Existing LLM agent research treats code as a target output—something to generate and evaluate for correctness. But as agents move from toy benchmarks to real-world deployment (GUI automation, scientific discovery, DevOps), code is increasingly the medium through which agents reason, act, maintain state, and verify outcomes. Prior work fragments this reality: some papers focus on code generation, others on tool use, still others on multi-agent coordination. No unified framework captures code’s role as the operational substrate—the harness—that makes agents executable and verifiable.

Problem: Agents need executable infrastructure
   |
   v
Observation: Code already serves this role (implicitly)
   |
   v
Gap: No unified view of "code as harness"
   |
   v
Method: Survey organizing around harness layers
   |
   v
Evidence: 100+ papers across 7 application domains
   |
   v
Conclusion: Code-as-harness is a design principle

The Increment

One sentence: Before this paper, code was an agent’s output; after, code is the agent’s skeleton—the harness that structures reasoning, action, memory, and multi-agent coordination into something you can run and verify.

Core Mechanism

The paper doesn’t propose a new algorithm. It’s a conceptual framework organizing existing work into three layers. Layer 1 (Interface): Code connects agents to three primitives—reasoning (code as structured thought), action (code as executable commands), and environment modeling (code as state representation). Layer 2 (Mechanisms): Planning, memory, and tool use enable long-horizon tasks; feedback loops and optimization make the harness adaptive. Layer 3 (Scaling): Multi-agent systems use shared code artifacts for coordination, review, and verification.

Layer 3: Multi-Agent Scaling
   |
   +---> Shared code artifacts
   +---> Coordination protocols
   +---> Verification workflows
   |
Layer 2: Harness Mechanisms
   |
   +---> Planning (decompose tasks into code)
   +---> Memory (code as persistent state)
   +---> Tool use (code as API glue)
   +---> Feedback (execution results -> refinement)
   |
Layer 1: Harness Interface
   |
   +---> Reasoning: code as structured thought
   +---> Action: code as executable commands
   +---> Environment: code as state model

Think of it like electrical wiring in a building. Before, you had appliances (agents) that someone evaluated by plugging them into a test socket (benchmarks). Now you’re designing the entire electrical system. Layer 1 is the outlets and switches—the interface where agents connect to power (reasoning), control devices (action), and sense the environment (state). Layer 2 is the circuit breakers, timers, and smart home logic—mechanisms that route power, remember schedules, and adapt to load. Layer 3 is the grid connecting multiple buildings—shared infrastructure where agents coordinate without stepping on each other’s toes. The harness isn’t the appliance; it’s the wiring that makes appliances useful in a real building.

Key Concepts

• Agent Harness: The infrastructure that connects an agent’s high-level intent to executable operations. In traditional robotics, a harness is the physical wiring and control logic between a brain (planner) and actuators (motors). Here, code plays that role: it translates abstract goals (“book a flight”) into concrete API calls, maintains state across steps, and provides feedback from execution. Without a harness, an agent is just a language model generating text. With a harness, it’s a system that *does things.

• Code as Reasoning Substrate: Instead of generating natural language chains-of-thought, agents write code to think. Why? Code has syntax constraints (catches errors early), is executable (you can run it to verify), and composes (functions call functions). Example: An agent solving a math problem doesn’t write “first I’ll compute X, then Y”—it writes x = compute_x(data); y = compute_y(x). The code *is the reasoning trace, and running it proves correctness.

• Execution-Based Verification: Traditional agents rely on human eval or model-based scoring. Code-as-harness enables a tighter loop: write code, run it, check if it crashes or produces the right output, refine. This shifts verification from subjective judgment to objective execution. Example: A coding assistant doesn’t just generate a function—it runs tests, sees failures, and iterates. The harness makes this loop fast and automatic.

Framework Shift

Before (code as output):              After (code as harness):

  User Query                            User Query
      |                                     |
      v                                     v
  LLM generates code                   LLM writes code
      |                                     |
      v                                     v
  Human evaluates                      Code executes
      |                                     |
      v                                     v
  Task done (or fails)                 Feedback loop
                                            |
                                            v
                                       Refine & retry
                                            |
                                            v
                                       Task done (verified)

From “code is the deliverable” to “code is the delivery mechanism,” the core shift is making agents executable systems rather than text generators.

Expert Assessment

Problem choice: Real and timely. The gap between “LLMs can code” and “LLMs can build reliable agents” is wide. Framing code as infrastructure rather than output is a natural next step, though not entirely novel—robotics and formal methods have long used code as operational substrate. The contribution is recognizing this pattern in LLM agents and organizing scattered work under one lens.

Method maturity: This is a survey, not a method paper, so “maturity” means conceptual clarity. The three-layer framework is clean but not groundbreaking—it’s a taxonomy, not a theory. The real value is synthesis: pulling together 100+ papers and showing they’re all variations on “code as harness.” The risk is that the framework is too broad to be falsifiable. What *wouldn’t fit under “code as harness”?

Experimental integrity: No experiments—this is a literature survey. The authors cite extensively and fairly represent prior work. One weakness: the paper doesn’t critically assess which harness designs actually work in practice. It catalogs approaches but doesn’t rank them by reliability, scalability, or failure modes. A practitioner reading this still doesn’t know *which harness to build.

Writing quality: Solid structure, but the paper tries to cover too much. The application section (coding assistants, GUI automation, embodied agents, etc.) reads like a laundry list. Cutting two domains and going deeper on the rest would strengthen the narrative. The “open challenges” section is the most valuable—it names hard problems (regression-free improvement, multi-agent state consistency) that the field hasn’t solved.

Verdict: weak accept — Useful synthesis and a clear conceptual frame, but lacks depth on what makes a harness succeed or fail in practice. Good for orienting newcomers, less useful for experts building systems.

Takeaways

Steal the three-layer mental model: When designing an agent, ask: (1) What’s my interface? (code for reasoning, action, or environment modeling?), (2) What mechanisms do I need? (planning, memory, feedback loops?), (3) Does this scale to multiple agents? (shared state, coordination protocols?). This checklist prevents ad-hoc designs.

Execution-based verification is underused: If your agent generates code, run it. Don’t just check syntax—execute it in a sandbox, capture errors, and feed them back. This loop is cheap and catches bugs that static analysis misses. The paper shows this pattern across domains (coding assistants, scientific discovery), but many production systems still rely on human review.

Multi-agent harnesses need shared state primitives: The paper flags this as an open problem, but the implication is clear: if you’re building multi-agent systems, design for state consistency from day one. Use version control for code artifacts, message queues for coordination, and formal verification for critical paths. Don’t bolt these on later.

Code as reasoning is a forcing function: Writing code to think forces precision. Natural language lets agents handwave; code doesn’t compile if the logic is broken. If your agent’s reasoning is fuzzy, try making it write executable pseudocode instead of prose. The syntax errors will tell you where the thinking is sloppy.