GeoContra: Turning Fluent GIS Code into Verifiable Spatial Analysis

Paper: 2605.00782 Authors: Yinhao Xiao, Rongbo Xiao, Yihan Zhang Categories: cs.SE, cs.AI

The Gap

LLMs can generate syntactically correct GIS code that compiles and runs. The problem: geographic correctness is invisible to syntax checkers. A script might calculate distances in degrees instead of meters, mix coordinate systems, or produce negative travel times—all while passing unit tests. Prior work (CodeGeeX, Copilot for GIS) focuses on code fluency but lacks domain-specific verification. The gap is between executable code and geographically valid results.

Problem: LLM generates fluent but geographically wrong GIS code
         (CRS mismatch, unit errors, topology violations)
            |
            v
Assumption: Geographic correctness requires domain rules
            that syntax alone cannot enforce
            |
            v
Method: Encode spatial contracts (CRS, topology, predicates)
        + static/runtime/semantic verification
        + bounded repair loop
            |
            v
Evidence: 7,079 tasks, 11 models, 600 runs each
          Correctness: 47.6% -> 77.5% (DeepSeek-V4)
                       57.7% -> 81.5% (Kimi-K2.5)
                       +26.6% average across open models
            |
            v
Conclusion: Contract-based verification bridges fluency
            and geographic validity

The Increment

One sentence: Before GeoContra, LLMs produced executable GIS scripts with silent geographic errors; after, a contract layer catches CRS mismatches, topology violations, and unit errors before they corrupt analysis.

Core Mechanism

GeoContra wraps each GIS task in a geospatial contract: natural-language question, data schemas, CRS metadata, expected output structure, spatial predicates (e.g., “result must be within Boston bounds”), topology constraints (e.g., “polygons must not self-intersect”), required operations (e.g., “must reproject before distance calculation”), and forbidden shortcuts (e.g., “no direct coordinate arithmetic”). The LLM generates code against this contract.

Verification happens in three stages. Static inspection checks for CRS consistency, required library calls, and forbidden patterns (like treating lat/lon as Cartesian). Runtime validation executes the code and checks output schema, data types, and bounds. Semantic verification tests spatial predicates: does the output satisfy topology rules, unit expectations, and domain constraints? Violations trigger a repair loop where the contract, error trace, and failed checks are fed back to the LLM with a bounded retry budget.

Task -> Contract Encoding -> LLM Generation -> Code
         (CRS, schema,                           |
          predicates,                            v
          topology rules)              Static Inspection
                                       (CRS, patterns)
                                                |
                                                v
                                       Runtime Validation
                                       (schema, bounds)
                                                |
                                                v
                                       Semantic Verification
                                       (predicates, topology)
                                                |
                                    +-----------+-----------+
                                    |                       |
                                  Pass                    Fail
                                    |                       |
                                    v                       v
                                 Accept              Repair Loop
                                                  (feedback + retry)

Think of GeoContra as a building inspector for geographic code. The contract is the building code: it specifies load-bearing requirements (CRS must match), safety rules (no negative distances), and structural integrity (topology must be valid). The LLM is the contractor who submits plans. Static inspection is the blueprint review—checking for obvious code violations before construction starts. Runtime validation is the framing inspection—does the structure match the approved plans? Semantic verification is the final walkthrough—does the building actually function as intended (doors open, plumbing works)? If inspections fail, the contractor gets a punch list of violations and must revise within a limited number of attempts. The inspector doesn’t build the house, but ensures what gets built won’t collapse.

Key Concepts

• Geospatial Contract: A formal specification that bundles task requirements with geographic constraints. Unlike a unit test that checks “does this function return 42,” a geospatial contract checks “does this result lie within valid geographic bounds, use consistent coordinate systems, and satisfy spatial predicates like containment or adjacency?” It’s executable documentation that encodes domain knowledge LLMs lack. Example: for “find parks within 1km of subway stations,” the contract specifies input CRS (EPSG

  ), required reprojection to a metric CRS before distance calculation, output schema (park names + distances in meters), and a predicate that all distances must be positive and ≤1000.

• Bounded Repair Loop: A feedback mechanism with a hard limit on retry attempts. When verification fails, the system doesn’t just say “wrong”—it returns the contract, the generated code, the specific violation (e.g., “CRS mismatch: input is EPSG

  but distance calculated without reprojection”), and the failed predicate. The LLM gets this context to generate a fix. The “bounded” part prevents infinite loops: after N attempts (paper uses 3), the system accepts failure rather than burning compute. This is critical because some tasks are genuinely ambiguous or underspecified, and no amount of repair will fix them.

• Semantic Verification vs Runtime Validation: Runtime validation checks mechanical correctness—did the code crash? Is the output a DataFrame with the right columns? Semantic verification checks geographic meaning—are the coordinates plausible? Do polygons have valid topology? Is the calculated area positive? A script can pass runtime checks (no exceptions, correct types) while producing nonsense geography (negative travel times, self-intersecting polygons, distances in degrees). Semantic verification is where domain knowledge enters: it evaluates whether the result makes sense in the real world, not just in Python’s type system.

Framework Shift

Before (mainstream LLM-GIS):        After (GeoContra):

User Query                          User Query
    |                                   |
    v                                   v
LLM generates code              Contract Encoding
    |                            (CRS, predicates, topology)
    v                                   |
Execute script                          v
    |                           LLM generates code
    v                                   |
Output                                  v
(may be geographically         Verification Pipeline
 invalid but executable)        (static + runtime + semantic)
                                        |
                            +-----------+-----------+
                            |                       |
                          Pass                    Fail
                            |                       |
                            v                       v
                         Output              Repair Loop
                      (verified)          (bounded retries)

From “generate and hope” to “generate, verify, repair,” the core shift is making geographic correctness a first-class constraint enforced by automated checks, not post-hoc manual inspection.

Expert Assessment

Problem choice: Real and underexplored. GIS practitioners know LLM-generated code often looks right but produces garbage results. The gap between syntactic fluency and domain validity is genuine across specialized fields (bioinformatics, finance, physics), but GIS is a good testbed because violations are concrete and checkable. This sits at the intersection of program synthesis and domain-specific verification—a natural next step after code generation became commoditized.

Method maturity: Solid engineering, not a research breakthrough. The contract idea is borrowed from design-by-contract (Eiffel, Dafny) and adapted to geospatial domains. The three-stage verification is standard software testing (static analysis, runtime checks, property testing). The novelty is in the domain encoding and the repair loop’s integration with LLM feedback. No deep algorithmic insight, but that’s fine—this is infrastructure work. The bounded repair loop is pragmatic; unbounded repair would be more elegant but impractical.

Experimental integrity: Strong. 7,079 tasks across 15 geographic zones and 9 task families (buffering, intersection, routing, etc.) is comprehensive. Testing 11 models with 600 runs each shows the method generalizes. The baselines are fair: vanilla LLM generation without verification. The correctness metric (spatial predicates + topology + units) is well-defined. One weakness: no comparison to other verification approaches (e.g., symbolic execution, fuzzing, or human-in-the-loop validation). The paper treats “no verification” as the only alternative, which overstates the contribution. Also, the 26.6% average improvement hides variance—some models barely improve, suggesting the method’s effectiveness depends on base model quality.

Writing quality: Clear structure, but the contract encoding section is dense and would benefit from a worked example earlier. The evaluation buries important details (what counts as a “violation”? how are predicates specified?) in supplementary material. The related work section name-drops tools (QGIS, ArcPy) without explaining why they don’t solve this problem. Rewriting Section 3 (Method) with a concrete running example—showing the contract, generated code, violation, and repair for one task—would make the paper 30% more accessible.

Verdict: weak accept — Solid engineering contribution with strong evaluation, but limited conceptual novelty. The value is in demonstrating that domain-specific verification can tame LLM code generation in specialized fields. Useful for practitioners, incremental for researchers.

Takeaways

Contract-as-context pattern: When LLMs generate domain-specific code, encode constraints as structured contracts (schemas, predicates, forbidden patterns) and include them in the prompt. This is cheaper than fine-tuning and more maintainable than post-hoc validation. Transferable to any domain with checkable invariants: financial calculations (no negative prices), scientific simulations (energy conservation), database queries (referential integrity).

Three-stage verification: Static (pattern matching) catches obvious errors cheaply. Runtime (execution) catches crashes and type errors. Semantic (domain predicates) catches silent wrongness. Run them in order of cost. This hierarchy applies beyond code: validating LLM-generated SQL, configuration files, or API calls.

Bounded repair with rich feedback: Don’t just say “wrong”—return the spec, the attempt, and the specific violation. But cap retries to avoid infinite loops. The “bounded” part is critical: it acknowledges that some tasks are unsolvable with current methods and moves on rather than thrashing. This is a general pattern for LLM-in-the-loop systems.

Domain-specific verification beats general correctness: Unit tests check “does this function return the right type?” Domain verification checks “does this result make sense in the real world?” For specialized fields, the latter catches more bugs. If you’re building LLM tools for a technical domain, invest in encoding domain invariants, not just syntax checks.