Design Docs Are All You Need: A Repository Whose Main Branch Has Almost No Code

Paper: 2609.05364 Authors: Samuel Kushnir, Kimia Noorbakhsh, Kavya Sreedhar, Liqun Cheng, Ming Liu, Parthasarathy Ranganathan, Mohammad Alizadeh, Fred Kjolstad, Suvinay Subramanian Categories: cs.PL, cs.AI

The Gap

Machine-learning performance modelling is hostile terrain for long-lived software. A performance model encodes assumptions about the hardware and the model being served — tile sizes, memory hierarchies, the shape of attention, the cost of a collective. When the next generation of either arrives, those assumptions are invalidated, and the framework has to be refactored. Not sometimes: perpetually, by the nature of the domain.

The paper identifies a change that alters the economics of that loop. AI coding agents have become fast and capable enough that regenerating an entire library is cheaper than paying down the tech debt of incrementally patching it. If regeneration is the cheaper option, then the durable artifact is no longer the code — it is whatever the agents regenerate from. And that reframes the design question: what should the source of truth be, if the code is disposable?

   ML PERFORMANCE MODELLING: HOSTILE TERRAIN

   a performance model ENCODES ASSUMPTIONS about the hardware
   and the model being served:
     tile sizes | memory hierarchies | the shape of attention
     the cost of a collective
        |
        v
   when the next generation of EITHER arrives, those assumptions
   are INVALIDATED and the framework must be REFACTORED
     -> not sometimes, but PERPETUALLY, by the nature of the domain
        |
        v
   [THE CHANGE THAT ALTERS THE ECONOMICS]
     AI coding agents are now fast and capable enough that
     REGENERATING AN ENTIRE LIBRARY IS CHEAPER than paying down
     the tech debt of INCREMENTALLY PATCHING it
        |
        v
   IF regeneration is cheaper, the durable artifact is NO LONGER
   THE CODE -- it is whatever the agents regenerate FROM
        |
        v
   [THE REFRAMED DESIGN QUESTION] what should the SOURCE OF
   TRUTH be, if the code is DISPOSABLE?

The Increment

One sentence: Before this paper, the source of truth in a performance-modelling library was its code and the docs described it; after it, the docs are the source of truth and the code is regenerated from them, reproducing hand-audited reference models to round-off precision.

Core Mechanism

The inverted arrangement is the contribution, and it is stated exactly: the main branch contains almost no code. Instead, the repository is a DAG of self-contained natural-language design docs. Three consequences follow directly from that inversion:

• Coding sub-agents regenerate the implementation from only the docs on new version updates. “Only the docs” is the load-bearing phrase — the agents do not consult the previous implementation, so the docs must be sufficient on their own. That is a much stronger requirement than documentation-as-description.
• Every human change is a natural-language edit to a doc. So the review artifact and the source of truth are the same object; there is no code change without a corresponding doc change, because there is no code to change.
• Self-documenting by construction. This falls out rather than being maintained: if the docs generate the code, the docs cannot go stale.

Two ingredients make regeneration reliable, and they address the two ways this could fail — unclear specification and ambiguous output.

A design-doc style built around step-by-step worked examples that act as in-context demonstrations for the generating agents. Worked examples are specified for a reason: a prose description of a cost model leaves the agent to infer conventions, whereas a worked example shows the convention being applied to concrete numbers. The examples are the interface.

A minimal, recursively defined operator IR with three properties: symbolic (SymPy) cost expressions, a fast analytical roll-up mode for large sweeps, and a slow modulo-scheduling mode for fine-grained schedule studies. The IR is what constrains what a regenerated implementation can be — a minimal core with recursion rather than a large surface — and the two modes reflect the two things people actually do with a performance model: sweep broadly, then zoom in on one schedule. Keeping both in the same IR is what prevents the two use cases from drifting into separate tools.

Then the credibility check, which is the reason to take the claim seriously. Regenerated implementations reproduce hand-audited reference models — including DeepSeek-V3 serving on a TPU pod slice — to round-off precision. Two things about that. The comparison is against models that were hand-audited, so the reference is trustworthy rather than merely a previous output of the same system. And round-off precision on a real serving configuration is a stringent tolerance: this is not “the regenerated code runs and gives similar numbers”, it is numerical agreement.

The paper’s conclusion is carefully scoped and worth quoting for its modesty: the result suggests that design docs — not code — can be the durable artifact for ML-systems co-design tools.

   THE INVERSION: THE MAIN BRANCH CONTAINS ALMOST NO CODE
     instead, the repository is a DAG of SELF-CONTAINED
     NATURAL-LANGUAGE DESIGN DOCS
        |
        v
   THREE CONSEQUENCES
     [a] coding sub-agents REGENERATE THE IMPLEMENTATION
         FROM ONLY THE DOCS on new version updates
           <- "only the docs" is load-bearing: the agents do
              NOT consult the previous implementation
           <- a much stronger requirement than
              documentation-as-description
     [b] EVERY HUMAN CHANGE is a natural-language edit to a doc
           -> the review artifact and the source of truth are the
              SAME OBJECT
     [c] SELF-DOCUMENTING BY CONSTRUCTION
           <- falls out rather than being maintained: if the docs
              generate the code, the docs CANNOT GO STALE

   TWO INGREDIENTS THAT MAKE REGENERATION RELIABLE
     (addressing the two ways it could fail: unclear
      specification, ambiguous output)

     [i] A DESIGN-DOC STYLE BUILT AROUND STEP-BY-STEP WORKED
         EXAMPLES acting as IN-CONTEXT DEMONSTRATIONS
           <- prose leaves the agent to INFER conventions;
              a worked example SHOWS the convention applied
              to concrete numbers
           -> the examples ARE the interface

     [ii] A MINIMAL, RECURSIVELY DEFINED OPERATOR IR with:
            SYMBOLIC (SymPy) cost expressions
            a FAST analytical roll-up mode for LARGE SWEEPS
            a SLOW modulo-scheduling mode for FINE-GRAINED
              schedule studies
           <- the IR constrains what a regenerated implementation
              CAN BE: a minimal recursive core, not a large surface
           <- the two modes reflect the two things people do:
              sweep broadly, then zoom in
           -> keeping both in one IR stops the two use cases from
              drifting into separate tools

   THE CREDIBILITY CHECK
     regenerated implementations reproduce HAND-AUDITED REFERENCE
     MODELS -- including DeepSeek-V3 serving on a TPU pod slice --
     TO ROUND-OFF PRECISION
       <- the reference is HAND-AUDITED, not a previous output of
          the same system
       <- round-off precision on a real serving config is a
          STRINGENT tolerance: not "similar numbers" but
          numerical agreement

   CONCLUSION (carefully scoped)
     suggests that DESIGN DOCS -- not code -- can be the durable
     artifact for ML-systems co-design tools

Think of it as a recipe book that a cook can execute perfectly, kept in place of the prepared meals. If the meals spoil — and in this domain they spoil on a schedule, every time the kitchen equipment changes — then storing meals means storing something that is always going off. Store the recipes instead, and each time the equipment changes you cook again. Two things determine whether that works. The recipe has to be specific enough that a competent cook produces the same dish without watching the last one being made — which is the role of worked examples. And there has to be a constrained vocabulary of operations, so the cook does not invent a different technique each time — which is the role of the IR. What would make this convincing is a taste test against a dish prepared by an expert who never saw the recipe: that is the hand-audited reference at round-off precision.

Key Concepts

• Docs as source of truth, code as build output: the repository’s main branch is a DAG of design docs. It makes the review artifact and the durable artifact the same object, and it makes staleness structurally impossible.
• Regeneration from docs alone: sub-agents do not consult the previous implementation. It is a much stronger requirement than ordinary documentation, and it is what forces the docs to be complete specifications.
• Worked examples as the interface: step-by-step examples that serve as in-context demonstrations. They show conventions applied to numbers, where prose would leave them to be inferred.
• A minimal recursive IR with two modes: symbolic cost expressions, a fast roll-up for sweeps and a slow scheduling mode for fine-grained study. It constrains the space of regenerated implementations and keeps two use cases in one tool.
• Hand-audited references at round-off precision: the credibility check, and a stringent one. It rules out “the regenerated code roughly agrees” and requires numerical agreement against a trusted baseline.

Framework Shift

Before (code is the source of truth):
  performance model lives in code
  docs describe the code
  -> when hardware or models change, the assumptions
     invalidate and the framework must be REFACTORED
  -> perpetual tech debt, incrementally paid down

After (docs are the source of truth):
  main branch is a DAG of natural-language design docs
  sub-agents regenerate implementation from the docs alone
  every human change is a doc edit; self-documenting by construction
  -> regenerated code matches hand-audited references to
     round-off precision

From maintaining a codebase whose assumptions expire to maintaining a specification from which the codebase is rebuilt, the core shift is that when regeneration becomes cheaper than patching, the durable artifact moves one level up and the code becomes an output.

Expert Assessment

Problem choice: Excellent, and the domain choice is what makes it credible rather than speculative. Performance modelling is the strongest possible case for doc-first development, because its assumptions expire on a predictable schedule — so the argument does not rest on a general claim about documentation, but on a domain where code demonstrably has a short half-life.

Method maturity: The two enabling ingredients address the two real failure modes, and both are well chosen. Worked examples as the specification format solve the ambiguity that makes regenerated code drift; the minimal recursive IR with two modes constrains the space so that regeneration is a search over a small region rather than an open-ended generation. The test protocol is the strongest element: comparing against hand-audited references and requiring round-off precision on a real serving configuration is a demanding bar, and reporting it for DeepSeek-V3 on a TPU pod slice is concrete.

Experimental integrity: The comparison is against an independent, hand-audited baseline rather than a previous generation of the same system, which is what makes numerical agreement meaningful. The honest limitation is scope: this is demonstrated for one library in one domain, and the claim that docs can be the durable artifact is offered as a suggestion rather than a general result. Whether the approach survives a domain where the specification itself is contested — rather than formalisable as cost expressions — is left open, and the paper does not overreach it.

Writing quality: The inversion is stated flatly in the title and abstract, which is the right treatment for a claim this counterintuitive. Because the practical question is what a design doc in this style looks like, a short annotated example — one operator, its worked example, and the regenerated implementation — would let a team assess whether their own domain admits this treatment.

Verdict: strong accept — it makes a well-argued case for inverting source and documentation in a domain where code demonstrably expires, and it backs the inversion with numerical agreement against hand-audited references.

Takeaways

• Ask whether regeneration has become cheaper than patching. When it has, the durable artifact moves up a level — and that changes what you should be maintaining.
• Specify with worked examples, not prose. An example applies conventions to concrete numbers where a description leaves them to be inferred.
• Constrain the output space with a minimal core. A small recursive IR limits what a regenerated implementation can be, which is what makes regeneration reliable rather than creative.
• Test regeneration against an independent, hand-audited baseline. Agreement with a previous generation of the same system proves nothing about correctness.