GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL

Paper: 2602.22190 Authors: Rui Yang, Qianhui Wu, Zhaoyang Wang, Hanyang Chen, Ke Yang, Hao Cheng, Huaxiu Yao, Baoling Peng, Huan Zhang, Jianfeng Gao Categories: cs.LG, cs.AI, cs.CL

Abstract

Open-source GUI agents continue to underperform compared to their closed-source counterparts on complex, long-horizon navigation tasks. GUI-Libra addresses this performance gap by tackling two critical limitations in current training approaches: the scarcity of high-quality action-aligned reasoning data and the misapplication of generic post-training pipelines that fail to account for GUI-specific challenges. The framework introduces three key innovations: a curated 81K GUI reasoning dataset with action-aligned annotations, an action-aware supervised fine-tuning approach that balances reasoning with grounding through strategic data mixing and token reweighting, and a stabilized reinforcement learning method that handles partial verifiability through KL regularization and success-adaptive scaling. Experimental results across web and mobile benchmarks demonstrate consistent improvements in both step-wise accuracy and end-to-end task completion rates.

Key Contributions

• Curated GUI Reasoning Dataset: Release of an 81K high-quality dataset with action-aligned reasoning traces, constructed through a systematic data construction and filtering pipeline
• Action-aware Supervised Fine-tuning: A novel SFT approach that mixes reasoning-then-action and direct-action data with token reweighting to preserve grounding capabilities while enabling reasoning
• Stabilized RL under Partial Verifiability: Introduction of KL trust region regularization and success-adaptive scaling to address the challenge where multiple correct actions exist but only one is used for verification
• Comprehensive Empirical Validation: Demonstration of consistent improvements across diverse web and mobile benchmarks without requiring costly online data collection

The Problem: Why Open-Source GUI Agents Lag Behind

The performance gap between open-source and closed-source GUI agents stems from fundamental issues in how these systems are trained. Standard supervised fine-tuning with chain-of-thought (CoT) reasoning, while effective for general language tasks, often degrades the grounding capabilities essential for GUI interaction. The model learns to generate elaborate reasoning but loses precision in identifying and executing the correct UI actions.

Additionally, step-wise reinforcement learning from verifiable rewards (RLVR) faces a unique challenge in GUI environments: partial verifiability. In many scenarios, multiple actions can correctly advance toward the goal, but traditional training only verifies against a single demonstrated action. This creates a mismatch where offline step-wise metrics become poor predictors of actual online task success, leading to unstable training dynamics and suboptimal policies.

Methodology: The GUI-Libra Training Recipe

Data Construction and Curation

GUI-Libra begins with addressing the data scarcity problem through a systematic pipeline that constructs action-aligned reasoning traces. The process involves:

1. Trajectory Collection: Gathering GUI interaction trajectories from diverse web and mobile environments
2. Reasoning Annotation: Generating step-by-step reasoning that explicitly connects observations to actions
3. Quality Filtering: Applying strict filtering criteria to ensure action-grounding alignment and reasoning quality
4. Dataset Balancing: Curating a final 81K dataset that balances reasoning depth with action precision

The key insight is that reasoning traces must be action-aware—they should not just explain what to do, but explicitly ground the reasoning in observable UI elements and actionable targets.

Action-aware Supervised Fine-tuning

To reconcile reasoning with grounding, GUI-Libra proposes a mixed training approach:

$\mathcal{L}_{\text{SFT}} = \alpha \mathcal{L}_{\text{reasoning}} + (1-\alpha) \mathcal{L}_{\text{direct}}$

where $\mathcal{L}_{\text{reasoning}}$ trains on reasoning-then-action sequences and $\mathcal{L}_{\text{direct}}$ trains on direct action prediction. The framework further applies token reweighting to emphasize action and grounding tokens:

$w_i = \begin{cases} \lambda_{\text{action}} & \text{if token } i \text{ is action-related} \\ \lambda_{\text{ground}} & \text{if token } i \text{ is grounding-related} \\ 1 & \text{otherwise} \end{cases}$

This approach prevents the common pitfall where verbose reasoning dilutes the model’s attention to critical action execution details.

Stabilized RL with KL Regularization

For reinforcement learning under partial verifiability, GUI-Libra identifies that standard RLVR suffers from gradient instability when negative rewards are assigned to potentially correct actions. The solution involves two components:

KL Trust Region: Maintaining a trust region around the SFT policy to prevent catastrophic policy shifts:

$\mathcal{L}_{\text{RL}} = \mathbb{E}_{s,a \sim \pi_\theta}[r(s,a)] - \beta \text{KL}(\pi_\theta || \pi_{\text{SFT}})$

The KL regularization term $\beta \text{KL}(\pi_\theta || \pi_{\text{SFT}})$ ensures that the policy doesn’t deviate too far from the supervised baseline, which is critical when verification signals are noisy.

Success-adaptive Scaling: Downweighting negative gradients based on task success rates to reduce the impact of false negatives:

$abla_\theta \mathcal{L} = \begin{cases} abla_\theta \log \pi_\theta(a|s) & \text{if } r > 0 \\ \gamma(s) abla_\theta \log \pi_\theta(a|s) & \text{if } r \leq 0 \end{cases}$

where $\gamma(s) \in [0,1]$ is computed based on the empirical success rate of similar states, effectively reducing the learning signal from potentially incorrect negative examples.

Experimental Results and Analysis

GUI-Libra was evaluated across multiple benchmarks spanning web navigation (Mind2Web, WebArena) and mobile interaction (AndroidWorld, MobileControl) tasks. The results demonstrate consistent improvements:

• Step-wise Accuracy: 12-18% improvement over baseline SFT models across benchmarks
• Task Completion Rate: 15-25% improvement in end-to-end task success
• Offline-Online Correlation: Significantly improved predictability of online performance from offline metrics after applying KL regularization

Ablation studies reveal that each component contributes meaningfully:

• Removing action-aware token reweighting reduces grounding accuracy by ~8%
• Disabling KL regularization leads to training instability and 10-15% performance degradation
• Success-adaptive scaling provides an additional 3-5% improvement in task completion

The framework also demonstrates strong data efficiency—achieving competitive performance with 81K curated examples compared to systems trained on millions of trajectories.

Implications for GUI Agent Development

GUI-Libra’s success highlights several important principles for training reasoning-capable GUI agents:

Reasoning Must Be Grounded: Generic CoT reasoning, while powerful for language tasks, can actually harm GUI agent performance if not carefully integrated with action grounding. The solution isn’t to abandon reasoning but to make it action-aware through data curation and training objectives.

Partial Verifiability Requires Special Handling: GUI environments present unique RL challenges where standard reward signals are inherently noisy. Techniques like KL regularization and success-adaptive scaling are not optional refinements but essential components for stable learning.

Data Quality Over Quantity: The 81K curated dataset outperforms much larger but lower-quality datasets, suggesting that careful data construction and filtering can be more valuable than scale alone for GUI agent training.

Post-training Matters: The gap between open-source and closed-source systems isn’t solely about model architecture or scale—carefully designed post-training recipes can unlock substantial capabilities without requiring massive computational resources or online data collection.

Takeaways

1. Open-source GUI agents can achieve competitive performance through tailored training approaches that address GUI-specific challenges rather than applying generic post-training pipelines
2. Action-aware supervised fine-tuning with mixed data and token reweighting successfully balances reasoning capabilities with grounding precision
3. KL regularization is critical for stable reinforcement learning under partial verifiability, dramatically improving offline-to-online performance predictability
4. High-quality, action-aligned reasoning data (81K examples) can be more effective than much larger datasets lacking careful curation
5. Success-adaptive scaling provides a principled way to handle noisy negative signals in partially verifiable environments
6. The framework’s data efficiency suggests that careful post-training design can reduce the need for expensive online data collection
7. Future GUI agent research should prioritize action-grounding alignment in both data construction and training objectives