Do AI Agents Know When a Task Is Simple? — Letting LLM Agents Right-Size Their Effort

Paper: 2607.13034 Authors: Junjie Yin, Xinyu Feng Categories: cs.AI, cs.CL, cs.SE, eess.SY

The Gap

LLM coding agents today are like interns who read the entire codebase before fixing a typo in a comment. They follow a maximum-context-first strategy: every task triggers a full dependency crawl, re-reading files already inspected, expanding scope indiscriminately. Prior work on retrieval-augmented agents (AutoCodeRover, Agentless, SWE-Agent) focused on finding the right file but never asked how much finding is actually needed. Adaptive retrieval approaches (e.g., Self-RAG, FLARE) adjust query strategies but still assume the agent should explore broadly before acting. Nobody formalized the question: “How little context does this task actually require?”

The logical path from gap to conclusion:

LLM agents always read maximally
         |
         v
No prior work estimates task complexity upfront
         |
         v
Propose: estimate difficulty -> execute minimally -> expand if stuck
         |
         v
E3 method: Estimate, Execute, Expand
         |
         v
MSE-Bench + LLM-Case experiments
         |
         v
Same accuracy, 85-92% less cost

The Increment

One sentence: Before this paper, agents treated every task like it needed a full audit; after, they scope their effort to the actual complexity—cutting tokens by 91% with no accuracy loss.

Core Mechanism

E3 (Estimate, Execute, Expand) works in three phases. Phase 1 — Estimate: The agent receives a task instruction and, before reading any files, classifies the task’s difficulty and determines a *minimum-sufficient execution scope—an initial guess at how many files and how much context are truly needed. This is a lightweight inference step, not a full search.

Phase 2 — Execute: Armed with this estimate, the agent executes a *minimum viable path—reading only the files in scope, making the edit, and attempting verification. Crucially, it does not expand scope preemptively. It trusts the estimate and acts.

Phase 3 — Expand: If verification fails (the edit is wrong, tests fail, the agent detects inconsistency), the agent expands scope incrementally—reading more files, checking more dependencies—until the task is resolved or a budget is exhausted.

The authors formalize this with the Agent Cognitive Redundancy Ratio (ACRR): the ratio of resources actually consumed to the minimum needed. An agent with ACRR = 1.0 is perfectly efficient; today’s agents run at ACRR = 5-10x on simple tasks.

Task arrives
    |
    v
+-----------+
| ESTIMATE  |  classify difficulty, set initial scope
+-----------+
    |
    v
+-----------+
| EXECUTE   |  act within minimum viable path
+-----------+
    |
    v
  verified?
   /    \
  yes    no
   |      |
   v      v
 done   +--------+
        | EXPAND |  broaden scope, retry
        +--------+
            |
            v
         verified? --> loop or budget-exhausted

A load-bearing metaphor: the doctor’s visit.

Imagine going to a doctor with a headache. The maximum-context-first agent is a doctor who orders an MRI, blood panel, spinal tap, and genetic screening before asking where it hurts. E3 is a doctor who asks three questions (“Where does it hurt? How long? Any other symptoms?”), makes a preliminary diagnosis, and prescribes aspirin. If the aspirin doesn’t work, then they order the blood test. If that’s inconclusive, then the MRI. The “Estimate” phase is the intake questionnaire. The “Execute” phase is the first-line treatment. The “Expand” phase is the escalation ladder of diagnostics. The ACRR measures how many unnecessary tests were ordered compared to the minimum needed to reach the right diagnosis. A good doctor’s ACRR is close to 1.0; today’s LLM agents are running at 5-10x because they skip the questionnaire entirely.

Key Concepts

• Minimum-Sufficient Execution: Think of packing for a trip. You *could pack every item you own “just in case.” Minimum-sufficient execution is checking the weather forecast and packing exactly what you need. The paper formalizes this for code agents: the smallest set of file reads, context tokens, and edit attempts that can plausibly complete the task. The key insight is that you can estimate this before acting, and it’s usually much smaller than what agents currently do.

• Agent Cognitive Redundancy Ratio (ACRR): This is a single number that answers: “How many times more work did the agent do than necessary?” If a one-line edit required reading 3 files but the agent read 30, the ACRR is 10x. It’s like measuring fuel efficiency for agents—how many tokens per unit of useful work. The paper finds current agents run at ACRR 5-10x on simple tasks, meaning 80-90% of their computation is waste.

• E3 (Estimate, Execute, Expand): The core loop. Think of it as a “try the simple thing first” discipline. The novelty isn’t any single step—it’s the sequencing: *never expand before you’ve tried the estimate. This is counterintuitive because LLM agents are trained to be thorough. E3 says: thoroughness is a fallback, not a default.

Framework Shift

Before (mainstream approach):           After (this paper):
                                        
Task --> Read everything --> Act        Task --> Estimate scope --> Try minimal
           |                                |
           v                                v
    Always thorough,                 Efficient first,
    always expensive                 expand only if needed
           |                                |
           v                                v
    ACRR = 5-10x                     ACRR approaches 1.0
    (massive redundancy)             (right-sized effort)

From “always be thorough” to “be thorough only when the easy path fails,” the core shift is making task complexity estimation a first-class operation rather than an afterthought.

Expert Assessment

Problem choice: This is a real and underexplored gap. Every practitioner building LLM agents has noticed the “reads the whole repo before editing a comment” behavior. The field has been so focused on retrieval quality (did we find the right file?) that it ignored retrieval quantity (do we even need to search?). This sits at the right intersection of software engineering and agent design—it’s not a manufactured problem.

Method maturity: The three-phase E3 loop is elegant in its simplicity—essentially “try cheap first, escalate if needed.” But the Estimate phase is where the magic happens, and the paper somewhat glosses over how difficulty classification actually works in practice. Is it a prompt engineering trick? A lightweight classifier? The MSE-Bench results suggest it’s deterministic (capability-controlled simulator), which means the estimation may be easier than in the wild. The real-model LLM-Case experiment partially addresses this, but with only one model (gpt-4o) and one project. There’s a risk that the estimation heuristic is tuned for the benchmark.

Experimental integrity: MSE-Bench is clever—121 deterministic edits in a capability-controlled simulator eliminates confounds. The 85-92% cost reduction with 100% success rate is striking, but the benchmark is by construction knowable (the simulator controls what’s needed), so the gains may be upper-bound. The LLM-Case experiment with real pytest runs is more credible, and the authors are honest about the rate-limit being the only failure mode. Baselines are fair—they compare against adaptive retrieval and the strongest baseline. One concern: no comparison against agents that simply use a smaller fixed context window. Is E3 better than just “always read 3 files”?

Writing quality: The paper reads well for a systems paper, but the theoretical framing (formalizing minimum-sufficient execution) feels underdeveloped—it’s more a definition than a theory. Section 3 would benefit from a concrete worked example walking through one task with ACRR calculations. The “Engineering-Grounded AI (EGAI)” framing at the end is a stretch—it’s a good research direction but the paper doesn’t earn the grand name yet.

Verdict: weak accept — The problem is real, the cost savings are dramatic, and the method is simple enough to adopt. But the estimation mechanism needs more scrutiny, and the benchmark may overstate gains due to its controlled nature.

Takeaways

Three things a practitioner can steal today:

1. “Try cheap first” as a default policy for agent loops. Before your agent reads 50 files, have it estimate what it needs with a lightweight prompt. This is a one-prompt change to most agent frameworks and can cut costs 5-10x on simple tasks.

2. ACRR as a diagnostic metric. Instrument your agent pipeline to track context tokens consumed vs. minimum plausible need. If your ACRR is consistently above 3x, your agent is burning money on redundancy. This metric transfers to any LLM-based automation.

3. Verification-triggered expansion instead of preemptive expansion. The pattern of “expand scope only when the first attempt fails” is a general principle. Apply it to RAG systems (don’t retrieve 20 chunks; retrieve 5 and re-retrieve if the answer is uncertain), report generation (draft first, fact-check and expand later), and any multi-step agent workflow.