From Function Calls to Function Factories: PAW Compiles Prompts into Tiny Adapters

Paper: 2607.02512 Authors: Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng Categories: cs.LG, cs.AI, cs.CL

The Gap

Existing research has given us powerful tools for two extremes. On one side, we have traditional programming: precise, local, and efficient, but it fails spectacularly at tasks requiring fuzzy judgment or understanding intent (e.g., “flag the alarming log line”). On the other side, we have large language models (LLMs) accessed via APIs: they excel at these fuzzy tasks but are expensive, non-local, slow, and non-reproducible. The gap is a missing middle: a way to get the fuzzy competence of an LLM with the deployment virtues of compiled code. Prior work on distillation or fine-tuning often produces models that are still too large for true local execution or require complex training pipelines for every single task. This paper identifies that specific boundary: the lack of a compilation paradigm that turns a natural-language specification into a tiny, task-specific artifact.

The logical path from problem to conclusion is straightforward:

Problem: Fuzzy tasks need LLMs (big, slow, costly API calls).
Assumption: Knowledge from a big LLM can be *compressed* into a tiny adapter for a specific task.
Method: Train a *compiler* (a 4B model) to convert a text spec into an adapter (PEFT weights) for a small *interpreter* (0.6B model).
Evidence: On FuzzyBench (10M examples), the compiled adapter + interpreter matches the big LLM's performance at 1/50th the memory, locally.
Conclusion: We can now *compile* fuzzy functions instead of *calling* them, enabling efficient, offline, reusable artifacts.

The Increment

One sentence: Before this paper, using an LLM for a fuzzy task meant paying a recurring cost for a large, remote model per execution; after this paper, you can pay a one-time compilation cost to get a small, local, reusable artifact for that task.

Core Mechanism

The system has three main components that form a pipeline. First, there is a Compiler (a 4B parameter model trained on the FuzzyBench dataset). Its job is to take a natural language specification of a fuzzy function (e.g., “a function that rewrites a sentence to sound more professional”) and emit a set of parameter-efficient fine-tuning (PEFT) weights—specifically, low-rank adaptation (LoRA) matrices. Second, these weights are packaged into a PAW Program, which is simply the compiled adapter artifact. Third, a small, frozen Interpreter model (e.g., a 0.6B Qwen3) takes an input (e.g., a sentence) and the PAW Program (the adapter weights) to produce the output (e.g., the professional rewrite). The key is that the adapter is tiny (megabytes) and is applied to the interpreter only during inference for that specific function.

[Input Spec] --> [Compiler (4B)] --> [PAW Program (Adapter Weights)]
                                             |
                                             v
[Task Input] + [Interpreter (0.6B, frozen)] --> [Task Output]

Think of it like a specialized recipe book versus a master chef. The traditional LLM API is like hiring a world-renowned chef (the big LLM) for every meal you cook—you get great results, but it’s expensive and slow. In the PAW paradigm, the Compiler is the master chef who reads your vague request (“I want a dish that’s savory, a bit sweet, and crunchy”) and writes a precise, optimized recipe (the PAW Program/Adapter) for your home kitchen. This recipe is specifically tuned for your home kitchen’s equipment (the small Interpreter model). Once written, you (the Interpreter) can cook that exact dish, perfectly, any number of times, quickly and cheaply, without needing the master chef again. The compiler does the heavy thinking once; the interpreter executes efficiently forever.

Key Concepts

• Fuzzy Function Programming: Imagine programming not with if-else statements, but with natural language descriptions of intent. A “fuzzy function” is one where the input-to-output mapping is complex, ambiguous, and best described by examples and intent (e.g., “rank these search results by relevance to a user looking for home remedies”). Traditional code can’t capture this well. This paradigm says: let’s treat these fuzzy intents as first-class citizens in programming, with a defined way to “compile” them.

• Program-as-Weights (PAW): This is the core mechanical insight. A “program” in the traditional sense is a set of instructions. Here, a “program” *is the set of learned neural network weights (the adapter). The compiler’s output isn’t code; it’s a small file of numbers that, when loaded into the interpreter model, changes its behavior to match the specification. It reframes software distribution: instead of shipping an executable, you ship a weights file that configures a generic executor.

• FuzzyBench: You can’t train a compiler without a massive, diverse set of “fuzzy function specifications” and their expected implementations. FuzzyBench is that dataset—10 million examples pairing natural language task descriptions with input-output examples. It’s the curriculum that teaches the compiler to understand the vast space of possible fuzzy functions.

Framework Shift

Before (mainstream approach):        After (this paper):
                                     
+-------------------+                +-------------------+
|  User Need       |                |  User Need       |
+-------------------+                +-------------------+
         |                                    |
         v                                    v
+-------------------+                +-------------------+
|  Prompt Engineering|               |  Natural Language |
|  (for each query) |               |  Specification    |
+-------------------+                +-------------------+
         |                                    |
         v                                    v
+-------------------+                +-------------------+
|  Large LLM API    |               |  PAW Compiler     |
|  (per-query cost) |               |  (one-time cost)  |
+-------------------+                +-------------------+
         |                                    |
         v                                    v
+-------------------+                +-------------------+
|  Result          |                |  PAW Program      |
|  (Remote, Slow)  |                |  (Adapter File)   |
+-------------------+                +-------------------+
                                             |
                                             v
                                     +-------------------+
                                     |  Small Interpreter|
                                     |  (Local, Fast)    |
                                     +-------------------+
                                             |
                                             v
                                     +-------------------+
                                     |  Result          |
+-------------------+                |  (Local, Fast)    |
|  Cost: O(N) API   |                +-------------------+
+-------------------+                |  Cost: O(1) Comp. |
                                     |  + O(N) local inf.|
                                     +-------------------+

From calling a giant model on every input to compiling a small model once per function, the core shift is from runtime to compile-time intelligence.

Expert Assessment

Problem choice: Excellent. It’s a real and growing pain point. The tension between the power of LLMs and their operational cost/latency is a central issue in applied AI. Framing it as a “programming paradigm” problem is insightful and positions it correctly in the CS landscape.

Method maturity: It’s a clever and elegant brute-force approach. The insight is the “compiler” framing, but the method itself is a large-scale data-driven distillation job (training a model to output adapters). There aren’t simpler overlooked approaches for this level of generality. The use of PEFT/LoRA is standard but perfectly appropriate here.

Experimental integrity: The baselines are fair—they compare against prompting the same base Qwen3-32B model and other compiler models. The performance parity with 1/50th the memory and local execution is a compelling result. A potential red flag is the reliance on FuzzyBench; we need to know how well PAW generalizes to truly novel function types far outside the training distribution.

Writing quality: The writing is clear and well-structured. The weakest section is likely the detailed ablation studies (if any) or failure mode analysis. A deeper dive into *when the compilation fails—e.g., for extremely long-tail or compositional fuzzy functions—would elevate the paper by defining its boundaries more sharply.

Verdict: strong accept. It introduces a clear, novel paradigm with a solid implementation and very strong empirical results on a well-constructed benchmark. It’s likely to spawn follow-up work on more efficient compilers or specialized interpreters.

Takeaways

1. The “Compile-then-Execute” Mindset: Stop thinking of LLMs only as runtime oracles. For repeated tasks, consider if you can front-load the intelligence into a compilation step to create a cheap, fast executor. This is a powerful mental model for system design.
2. Adapters as Distributable Artifacts: This work demonstrates that a small file of adapter weights can be a powerful, portable unit of functionality. This idea transfers to many domains: imagine distributing specialized “skill packs” for a base robotics or vision model.
3. The Value of Task-Specific Curricula: The creation of FuzzyBench is a major contribution. It shows that to build general-purpose tools, you first need a massive, structured curriculum that covers the space of tasks. For any attempt to build a “compiler” for a new domain, dataset construction is job one.