Agentic Autoresearch: 81 Unattended Experiments and a Provable Structure

Paper: 2608.26093 Authors: Ahmad Khan, Akram Bin Sediq, Sara Azadegi Naeini, Raviraj S. Adve Categories: cs.LG, cs.IT, cs.SY

The Gap

Designing machine learning algorithms for wireless resource management is labour-intensive in a specific way. The architecture, the loss function and the training recipe are all specified by hand, by a researcher, before any experiment runs. The search over design choices is the researcher’s job, and it is slow because each candidate design requires a full training run before it can be judged.

The paper’s claim is that this entire layer can be surrendered. Not partially delegated — the agent is given authority over the architecture family, the input representation, the output parameterization, the loss function and the task-sampling law, which is to say every design decision that a researcher would normally make. What remains for the human is the problem formulation and the metric.

That is a strong claim and it needs two things to be believable: a problem hard enough that success is not trivial, and safeguards strong enough that the reported result is not the agent spinning a narrative.

   TRADITIONAL ML-FOR-WIRELESS DESIGN LOOP

   researcher chooses BY HAND:
     architecture family
     input representation
     output parameterization
     loss function
     task-sampling law
        |
        v
   train -> evaluate -> revise -> repeat
        |
        v
   [BOTTLENECK] every candidate design needs a full
                training run before it can be judged
        |
        v
   [CLAIM UNDER TEST] this whole design layer can be
                      surrendered to an autonomous agent
        |
        v
   NEEDED TO BE BELIEVABLE:
     (1) a problem hard enough that success is non-trivial
     (2) safeguards strong enough that the result is
         not the agent telling a story

The Increment

One sentence: Before this paper, the algorithm-design layer for wireless resource management was a human’s job; after it, an agent with authority over every design choice converged on a solution matching a proved optimum in one inference pass, under safeguards that make the result auditable.

Core Mechanism

The protocol is the autoresearch loop: an AI coding agent edits a training script, runs a fixed-budget experiment, and retains or discards the change according to a single immutable metric. Three properties of that loop are doing the work. Fixed budget means comparisons are fair — a design cannot win by training longer. A single immutable metric means the agent cannot shop for the criterion that flatters it. And edit-run-keep-or-revert means the agent is doing hill-climbing over designs, with the ground truth supplied by experiments rather than by its own judgement.

The problem is chosen for its difficulty, and the choice deserves scrutiny rather than being taken on faith. The target is sum-least-percentile-rate power control across a multicell network — a formulation aimed at cell-edge throughput. It is non-convex, non-smooth, and strongly NP-hard away from its max-min vertex. So the agent is not being handed a task where a reasonable default works; it is being handed one where the design layer matters, and where the human alternative — a converged minorization-maximization procedure — is a serious reference rather than a straw baseline.

The safeguards are what make the results trustworthy, and there are three:

• A hash-pinned evaluator, so the metric cannot drift between experiments.
• An enforced inference contract, so the design must satisfy a fixed cost structure rather than winning by spending more at inference.
• A pre-registered falsifier per experiment, which fixes in advance what would count as a failure. This is the most interesting safeguard: it prevents post hoc reinterpretation of an experiment’s outcome, which is exactly the failure mode an autonomous loop is prone to.

The results are reported in terms of cost and structure. In 81 unattended experiments over 26 hours, the agent reached:

• 99.5% of a converged minorization-maximization reference — in one fixed-cost inference pass
• at roughly 600× lower inference cost
• closing 94% of the gap from its first working architecture
• with one parameter set serving every network size and percentile target

The last point is easy to skim past, and it is the strongest generalisation claim: a single parameter set generalises across network sizes and across percentile targets, so the discovered design is not overfitted to one operating point.

And then the result that elevates the paper beyond engineering. The agent recovered provable structure rather than tuned constants: the output parameterization it discovered reproduces the exact max-min-optimal allocation at the minimum percentile, for every value of the trained weights. This is a qualitatively different kind of outcome from a better number. The agent’s design embodies a theorem about the problem — the optimum at the min-percentile vertex is reproduced exactly, and it holds for any weights, not just the trained ones. Tuned constants would have been evidence of search; a structure with a proof is evidence of insight.

   AUTORESEARCH LOOP
   agent edits training script
        |
        v
   fixed-budget experiment
        |
        v
   keep or revert by ONE IMMUTABLE METRIC
        |
        +--> repeat

   SAFEGUARDS
     hash-pinned evaluator        (metric cannot drift)
     enforced inference contract  (cannot win by spending more)
     pre-registered falsifier     (outcome fixed in advance)

   PROBLEM: sum-least-percentile-rate power control, multicell
     non-convex, non-smooth, strongly NP-hard off the max-min vertex

   RESULT (81 experiments, 26 hours, unattended)
     99.5% of a converged MM reference
     in ONE fixed-cost inference pass
     at ~600x lower inference cost
     closing 94% of the gap from its first working architecture
     one parameter set for every network size and target

   STRUCTURE, NOT CONSTANTS
     the discovered output parameterization REPRODUCES
     the exact max-min-optimal allocation at the min percentile,
     for EVERY value of the trained weights
        -> a theorem about the problem, not a tuned number

Think of it as handing a junior engineer not just the problem but the entire design notebook, and then checking what they write down. Give them enough rope and they will find a solution that works — that alone proves little, because search over a large design space with a fixed evaluation is a known capability. What distinguishes the outcome here is that the notebook ends up containing a derivation: the design’s output parameterization reproduces the optimum exactly at the hardest operating point, for all weights, which is the kind of statement a human would hand in as a proof rather than a submission. The safeguards are the equivalent of a supervisor who pins the test set, forbids changing the evaluation criteria mid-project, and requires the acceptance test to be written before the run.

Key Concepts

• The autoresearch protocol: edit a script, run a fixed-budget experiment, keep or revert by a single immutable metric. Its design choices are what make an autonomous loop’s claims comparable — fixed budget removes the “train longer” confound and the immutable metric removes criterion shopping.
• Pre-registered falsifiers: fixing in advance what counts as a failure for each experiment. It closes the specific loophole autonomous loops are prone to, which is reinterpreting an ambiguous result after seeing it.
• Recovered structure versus tuned constants: the distinction between a design that happens to match the optimum at a point and one that reproduces it exactly for all weights. The second is a property of the problem, and it is why this result reads as insight rather than as a good search outcome.
• One parameter set across operating points: generalisation across network sizes and percentile targets. It is the evidence that the discovered design captures something structural rather than fitting one configuration.

Framework Shift

Before (human designs, machine trains):
  researcher specifies architecture, loss, recipe
  -> each candidate needs a full training run
  -> the search is the researcher's bottleneck
  -> output: a design that works

After (agent designs, experiments adjudicate):
  agent has authority over architecture, representation,
    parameterization, loss, sampling law
  fixed-budget experiments + immutable metric adjudicate
  safeguards: hash-pinned evaluator, inference contract,
              pre-registered falsifier
  -> 99.5% of an MM reference, one pass, ~600x cheaper
  -> and a provably structure-matching parameterization

From a researcher searching the design space while the machine only trains, to the machine searching the design space while experiments supply the ground truth, the core shift is that the design layer can be adjudicated by measurement rather than by authorship — provided the milestones are pinned in advance.

Expert Assessment

Problem choice: Excellent, and unusually well chosen for the claim being made. Wireless resource management is a domain where hand-designed algorithms are the norm and where the problems are genuinely hard — non-convex, non-smooth, NP-hard off a vertex. A weak problem would have made the agent’s success uninformative; this one makes the comparison against a converged reference meaningful.

Method maturity: The three safeguards are the methodological substance, and the pre-registered falsifier is the one that most autonomous-research papers omit. It is the difference between a loop that can be evaluated and one that reports whatever it ends with. The one-parameter-set generalisation across network sizes and targets is the result that would most concern a skeptic, and it is the right thing to have checked.

Experimental integrity: Reporting 99.5% of a converged minorization-maximization reference — rather than beating a weaker baseline — is the honest comparison, and it is notable that the agent did not exceed it, which a hand-designed heuristic is not guaranteed to do either. Reporting that it closed 94% of the gap from its own first working architecture is a useful internal baseline. The strongest element is the structural result, because a claim that holds for every value of the trained weights is checkable in a way that a performance number is not. The open question is transfer: whether this workflow produces comparable results on a problem where the optimum has no such clean vertex structure.

Writing quality: The paper is compact and the results are enumerated with their conditions attached, which suits the claim. Two additions would help: a description of what the agent actually changed across the 81 experiments, since the trajectory of the search is itself informative about where the difficulty lay, and a concrete statement of what the discovered parameterization is, so a reader can see the structure rather than only the fact that it has one.

Verdict: strong accept — a well-safeguarded demonstration that the algorithm-design layer can be handed to an agent, ending in a provable structural result rather than a tuned performance number.

Takeaways

• Pre-register what counts as failure before each autonomous experiment. Without it, a loop that runs unattended will reinterpret ambiguous outcomes after the fact.
• Pin the evaluator and the inference contract. An autonomous search will exploit any freedom it is given, including spending more at inference or drifting the metric.
• Distinguish recovered structure from tuned constants when reading agent results. A parameterization that reproduces an optimum for all weights is a different class of claim from one that matches performance on a benchmark.
• Give an agent authority over the whole design layer, and keep the problem formulation and the metric. The split is what makes the result comparable — the human still owns what “good” means.