Data Intelligence Agents: Interpreting, Modeling, and Querying Enterprise Data via Autonomous Coding Agents

Paper: 2606.19319 Authors: Anoushka Vyas, Aarushi Dhanuka, Sina Khoshfetrat Pakazad, Henrik Ohlsson Categories: cs.MA, cs.AI, cs.DB

The Gap

Existing data integration in production pipelines relies on a serial handoff: data owners describe schemas to engineers who write ETL scripts, then analysts craft SQL queries. Tools like data catalogs and LLM-based assistants (e.g., SQL-Copilot, DAIL-SQL) have reduced some friction, but they still operate on a generate-then-approve cycle where the artifact (schema, query) is text that must be manually validated and adjusted. The key limitation: these systems have no internal mechanism to execute the generated code, observe failures, and repair errors autonomously. Human bottlenecks remain for every edge case. This paper identifies that gap and proposes an architecture where agents not only generate code but also execute it, evaluate correctness, and iterate—treating the code as a first-class executable object rather than a static text.

[ASCII logic topology]
Problem: Data integration bottleneck from repeated human handoffs
    |
    v
Assumption: LLM can generate correct code if given good prompts
    |
    +---< Prior work: text-only generation, no execution feedback
    |
    v
Paper's method: Agents that generate, execute, validate, repair code
    |
    v
Evidence: 7 SQL benchmarks across 4 dialects, matches/surpasses SOTA
    |
    v
Conclusion: Execution-grounded ACAs + shared memory generalize across data intelligence tasks

The Increment

One sentence: Before this paper, data integration agents could only propose code; after this paper, agents can execute code, catch errors, and fix themselves—then reuse that experience across future tasks.

Core Mechanism

The system (DIA) comprises three specialized agents: Data Interpreter (reads raw files and outputs structured descriptions), Schema Creator (infers and formalizes a schema), and Query Generator (writes and validates SQL queries). Each agent is built on top of an Autonomous Coding Agent (ACA). An ACA operates in a loop: (1) receive a natural-language instruction, (2) generate code (Python or SQL), (3) execute it in a sandbox, (4) capture execution results and errors, (5) if errors or incorrect results, repair the code and repeat, (6) once correct, store both the code and the execution trace into a shared memory. The shared memory acts as an experience database: future agents can retrieve past successes and failures for similar tasks, reducing trial and error. The Query Generator is studied in depth; it uses this loop plus retrieval from shared memory to produce SQL queries that are syntactically and semantically correct.

                      [ASCII diagram of method internals]
  User -> NL instruction
        |
        v
  [Shared Memory] <-----> [ACA core loop]
        ^                    |
        |                    v
        |              [Generate code]
        |                    |
        |                    v
        |              [Execute (sandbox)]
        |                    |
        |                    +---[Success]--> Store trace
        |                    |
        |                    v
        |              [Catch error]
        |                    |
        +---[Retrieve]---> [Repair code] <-- loop back

Here’s the structural metaphor: think of the ACA loop as a tireless apprentice cook working in a kitchen. The apprentice receives a recipe (natural-language instruction). First, they select ingredients (generate code). Then they cook a small test batch (execute in sandbox). If the taste is off or the dish burns (error or wrong output), they adjust the ingredients or technique (repair code). Once the test batch is perfect, they write down the exact process and store it in a shared recipe book (shared memory). The next time someone asks for a similar dish, the apprentice can look up the recipe book instead of starting from scratch. The three agents (Interpreter, Schema Creator, Query Generator) are like three different stations in the kitchen—each apprentice specializes in a different part of the meal, but they all share the same recipe book. The crucial point: the apprentice actually cooks, tastes, and corrects. That’s what makes this system different from a cook who just describes what they *would do.

Key Concepts

• Autonomous Coding Agent (ACA): An LLM-based loop that not only generates code but also executes it, observes the outcome, and self-corrects. It is not a one-shot generator. Example: given “Write a SQL query to find the top 5 customers by total revenue in 2025”, the ACA generates a query, runs it against a test database, sees an error “column ‘revenue’ does not exist”, checks the schema, rewrites using ‘amount’, re-runs, gets a result with 10 rows, filters to top 5, and finally stores the correct query. Without execution, you’d have to manually catch the column name mistake.

• Shared Memory: A persistent store of (instruction, code, execution trace, outcome) triples. It enables experience reuse across agents and tasks. Example: after the Schema Creator has inferred a schema for a dataset, that schema is stored. When the Query Generator later needs to write a query on the same dataset, it retrieves the schema from memory instead of re-inferring it, saving time and reducing errors. The memory is indexed by natural-language similarity so agents can find relevant past examples.

• Execution-grounded code generation: The principle that generated code is only considered correct if it produces the expected result when run. This contrasts with “text-grounded” approaches where correctness is evaluated by human review or abstract metrics (like BLEU). In practice, it means the agent can catch subtle errors like type mismatches, empty results due to incorrect joins, or performance issues (by checking execution time). The paper shows this leads to higher accuracy on SQL benchmarks.

Framework Shift

Before (mainstream approach):        After (this paper):
Human describes task                Human describes task
    |                                   |
    v                                   v
LLM generates code (text)           ACA generates code
    |                                   |
    v                                   v
Human reviews and edits              ACA executes code
    |                                   |
    +---[iterate manually]               +---[error?]--> repair loop
    |                                   |
    v                                   v
Final code stored (static)           Code + trace stored in shared memory
    |                                   |
    v                                   v
Next task: repeat from scratch       Next task: retrieve similar experience

One sentence: From text-only code generation with manual validation to execution-grounded, self-repairing code generation with experience reuse, the core shift is making code a live, testable artifact rather than a frozen proposal.

Expert Assessment

Problem choice: Real gap. The handoff problem in data integration is painful in every enterprise. The paper targets a practical bottleneck, not a fabricated one. It sits at the intersection of “AI for data” and “agentic systems”, a hot area where many papers propose ideas but few validate in production.

Method maturity: Clever insight—execution feedback with shared memory is obvious in hindsight but underexplored in data integration. The implementation appears solid: sandbox execution, repair loop, memory retrieval. However, the approach is computationally expensive: each query goes through multiple execute-repair cycles, and memory retrieval adds overhead. Simpler alternatives like fine-tuning a specialized LLM on SQL benchmarks might achieve similar accuracy without the loop, but the paper’s advantage is adaptability to new dialects and schemas without retraining.

Experimental integrity: Fair baselines—they compare against well-known methods like DAIL-SQL, DIN-SQL, and others on seven benchmarks (Spider, Bird, etc.). The results show matching or surpassing SOTA on all. One red flag: the evaluation is only on the Query Generator, not the full DIA system (Data Interpreter and Schema Creator are not benchmarked). Also, the “fully autonomous mode” means no human feedback, which is good for reproducibility but may not reflect production where a human can intervene. The shared memory component is not isolated in ablations to quantify its contribution; it’s always present.

Writing quality: Clear, well-structured. The paper is long but readable. The authors cut corners in the related work section—it’s a list of prior work without deep comparison. The section that could elevate the whole paper is a dedicated ablation study showing the contribution of execution feedback vs. shared memory vs. baseline LLM. That would make the architecture’s necessity undeniable.

Verdict: weak accept — Solid practical contribution with strong empirical results, but the novelty is incremental (execution + memory) and the evaluation leaves some questions about component-level impact. Worth attention if you build data tools; less so if you study theoretical AI.

Takeaways

• Execution sandbox as a core agent component: If you’re building any code-generating agent, adding a sandbox to run and validate the generated code is a cheap way to boost reliability. Don’t let the LLM hallucinate silently—let it crash and fix.
• Experience reuse via shared memory: Store not just the final code but the full trace (inputs, errors, corrections). This turns the agent into a learning system that gets cheaper over time. For practitioners, this means you can start with a generic LLM and let the memory specialize to your domain.
• Separate agents for separate concerns: The paper decomposes the problem into interpretation, schema, query. This modularity makes each agent simpler and easier to debug. In your own system, consider breaking a complex task into sub-agents with clear interfaces.