FlexSQL: Breaking the Fixed Pipeline in Text-to-SQL with On-Demand Database Interaction

Paper: 2605.02815 Authors: Quang Hieu Pham, Yang He, Ping Nie, Canwen Xu, Davood Rafiei, Yuepeng Wang, Xi Ye, Jocelyn Qiaochu Chen Categories: cs.CL

The Gap

Current text-to-SQL systems follow a rigid pipeline: retrieve schema elements once at the start, generate SQL, execute, and maybe repair if it breaks. This “retrieve-once-then-commit” design creates a recovery ceiling—if the initial schema retrieval misses key tables or the query interpretation is wrong, the system has no way to backtrack and reconsider. Over large analytical databases with hundreds of tables and ambiguous natural language queries, this brittleness compounds: early mistakes propagate through the pipeline with no mechanism for mid-flight correction.

Problem: Complex schemas + ambiguous queries
    |
    v
Assumption: Database interaction is expensive, 
            do it once upfront
    |
    v
Method: Flexible exploration + diverse plans + 
        two-tier repair (code-level + plan-level)
    |
    v
Evidence: 65.4% on Spider2-Snow (gpt-oss-120b),
          beats stronger models with fixed pipelines
    |
    v
Conclusion: Flexibility in when/how to query DB
            > one-shot retrieval + post-hoc repair

The Increment

One sentence: Before FlexSQL, text-to-SQL agents retrieved schema once and repaired SQL syntax errors; after FlexSQL, agents explore the database continuously during reasoning and can revise their interpretation of the query itself, not just the code.

Core Mechanism

FlexSQL has three interlocking components. First, flexible exploration: at any reasoning step, the agent can inspect schema metadata (table names, column types, foreign keys) or sample actual data values to ground its understanding. This isn’t a one-time retrieval—it’s an ongoing dialogue with the database. Second, diverse execution plans: instead of committing to a single SQL query, the agent generates multiple interpretations of the user’s question, each as a structured plan (which tables to join, what filters to apply). It then implements each plan in either SQL or Python, choosing the language that fits the task. Third, two-tier repair: if execution fails, the system first tries code-level fixes (syntax errors, type mismatches). If that doesn’t work, it backtracks to the plan level and revises the query interpretation itself—maybe the user meant a different table, or the join condition was wrong.

User Question
    |
    v
[Flexible Exploration]
    |
    +---> Schema Inspector ----+
    |                          |
    +---> Data Sampler --------+
    |                          |
    v                          v
[Plan Generator] ---------> Multiple Plans
    |                          |
    +---> Plan A (SQL)         |
    +---> Plan B (Python)      |
    +---> Plan C (SQL)         |
    |                          |
    v                          v
[Execution] <------------- [Two-Tier Repair]
    |                          ^
    |                          |
    +---> Code-level fix ------+
    |                          |
    +---> Plan-level revision -+
    |
    v
Results

Think of FlexSQL as a detective investigating a crime scene. Traditional systems are like detectives who read the case file once, form a theory, write a report, and only revise typos if the report has grammatical errors. FlexSQL is the detective who keeps returning to the scene: checking alibis (schema structure), interviewing witnesses (sampling data), considering multiple suspects (diverse plans), and willing to revisit the entire theory (plan-level repair) if new evidence contradicts it. The case file isn’t a static document—it’s an active resource the detective consults throughout the investigation. The key insight: flexibility in when and how you interact with evidence (the database) lets you recover from wrong turns that a one-shot approach would lock you into.

Key Concepts

• Flexible Exploration: Imagine you’re assembling IKEA furniture. A fixed pipeline is like reading the entire manual once, then building without looking back—if you misunderstood step 3, you’re stuck. Flexible exploration is checking the manual whenever you’re unsure: “Wait, does this screw go here or there?” You can inspect the diagram (schema metadata) or compare your half-built shelf to the picture (sample data) at any point. In FlexSQL, the agent doesn’t commit to a schema subset upfront. It queries the database’s metadata tables and samples rows as needed during reasoning, treating the database as a reference it can consult anytime, not a document it reads once.

• Plan-Level Repair: Most text-to-SQL systems repair syntax errors—missing commas, wrong function names. Plan-level repair goes deeper: it revises the semantic interpretation of the query. If the user asks “show me sales by region” and the agent initially joins the sales and regions tables but gets no results, plan-level repair might realize the user actually meant the territories table, not regions. It’s the difference between fixing a typo in your essay versus realizing your thesis statement is wrong and rewriting the argument. FlexSQL maintains a backtracking mechanism: if code-level fixes fail, it returns to the plan generation step and tries a different interpretation of the user’s intent.

• Diverse Execution Plans: Instead of betting everything on one query interpretation, FlexSQL generates multiple plans that cover different readings of the ambiguous question. If the user asks “average price,” does that mean average across all products, or grouped by category? The agent creates plans for both, implements them (some in SQL, some in Python if the logic is complex), and evaluates which results make sense. It’s like a chess player considering multiple opening moves before committing. The diversity isn’t random—it’s structured exploration of the ambiguity space in natural language queries.

Framework Shift

Before (fixed pipeline):              After (FlexSQL):

User Query                            User Query
    |                                     |
    v                                     v
Schema Retrieval (once)               Reasoning Loop:
    |                                     |
    v                                     +---> Explore schema
SQL Generation                            |     (anytime)
    |                                     |
    v                                     +---> Sample data
Execution                                 |     (anytime)
    |                                     |
    v                                     +---> Generate plans
Syntax Repair (if error)                  |     (multiple)
    |                                     |
    v                                     +---> Execute & verify
Done                                      |
                                          +---> Repair (code or plan)
                                          |
                                          v
                                      Done (when verified)

From a linear pipeline with one database touchpoint to a feedback loop where database interaction is woven throughout reasoning, the core shift is treating the database as a continuous resource rather than a one-time input.

Expert Assessment

Problem choice: Real gap. Large analytical databases (hundreds of tables, ambiguous column names) are where text-to-SQL breaks in practice, and the “retrieve once, repair syntax” paradigm genuinely hits a ceiling. This isn’t a manufactured problem—it’s where industry deployments fail. The paper positions itself well in the trajectory from small-schema benchmarks (Spider) to realistic enterprise scenarios (Spider2-Snow).

Method maturity: The core insight—flexible database interaction—is elegant and underexplored. However, the implementation leans on LLM reasoning rather than principled search or planning algorithms. The “generate diverse plans” step is essentially sampling from the LLM with different prompts, not a structured exploration of the query space. This works but feels like a transitional solution. A tighter integration with database query optimizers or symbolic planners could make this more robust. The two-tier repair is clever but the boundary between code-level and plan-level errors is heuristic, not principled.

Experimental integrity: Baselines are fair—they compare against strong open-source systems and even show their method integrated into Claude Code. The Spider2-Snow benchmark is appropriate (real Snowflake databases, complex schemas). One concern: the 65.4% score uses gpt-oss-120b, but the comparison to “stronger models like gpt-o3” is vague—no head-to-head with the same model across methods. The ablation studies (flexible exploration vs. flexible execution) are solid and show both components contribute. No obvious red flags, but I’d want to see error analysis: what types of queries still fail?

Writing quality: The abstract and introduction are crisp. The method section is dense—Figure 2 (the system diagram) does heavy lifting but isn’t self-explanatory. The paper would benefit from a worked example walking through one query end-to-end, showing exactly when the agent explores schema, generates plans, and repairs. The related work section is thorough but reads like a checklist. The conclusion oversells slightly (“key design principle”) without discussing failure modes or when flexibility might hurt (e.g., exploration cost on simple queries).

Verdict: weak accept — Solid contribution with a clear insight (flexible DB interaction) and strong empirical results, but the method feels like an LLM-era hack rather than a durable architectural principle. The paper advances the state of the art but doesn’t fully explore the design space it opens.

Takeaways

For practitioners building text-to-SQL systems: Don’t lock yourself into upfront schema retrieval. Instrument your database so the agent can cheaply query metadata and sample data during reasoning. The cost of a few extra metadata queries is negligible compared to the cost of generating and repairing a wrong SQL query.

For agent builders in any domain: The two-tier repair pattern (surface-level fix → deep revision) transfers. If your agent writes code, tries it, and fails, don’t just patch syntax—consider whether the plan itself is wrong. Build backtracking into your agent loop.

For researchers: The “diverse plans” approach is undertheorized. There’s a paper waiting to be written on how to systematically explore the ambiguity space in natural language queries, rather than relying on LLM sampling. The connection to query optimization and database theory is underdeveloped—text-to-SQL could borrow more from decades of work on query planning.

Steal this: The idea of treating external resources (databases, APIs, file systems) as continuous dialogue partners rather than one-shot inputs. Most agent architectures retrieve context once and then reason in isolation. FlexSQL shows that interleaving retrieval and reasoning can break through brittleness ceilings.