Scaling Verification Can Be More Effective than Scaling Policy Learning for Vision-Language-Action Alignment

Paper: 2602.12281 Authors: Jacky Kwok, Xilun Zhang, Mengdi Xu, Yuejiang Liu, Azalia Mirhoseini, Chelsea Finn, Marco Pavone Categories: cs.RO, cs.AI, eess.SY

Abstract

Vision-Language-Action (VLA) models represent a significant step toward general-purpose robots that can understand and execute natural language instructions. However, a persistent challenge remains: the “intention-action gap” where generated actions misalign with given instructions. This paper investigates test-time verification as a solution, characterizing scaling laws for embodied instruction following and demonstrating that jointly scaling rephrased instructions and generated actions increases sample diversity more efficiently than independent scaling. The authors present CoVer, a contrastive verifier for vision-language-action alignment that scales gracefully with computational resources and data. Their framework introduces “boot-time compute” and hierarchical verification, precomputing diverse instruction rephrasings, generating multiple action candidates, and using verification to select optimal prompts and action chunks. Results show 22% in-distribution and 13% out-of-distribution gains on SIMPLER benchmark, with 45% improvement in real-world experiments compared to scaling policy pre-training.

Key Contributions

• Test-Time Scaling Laws: Characterization of scaling behavior for embodied instruction following, showing that joint scaling of instruction rephrasings and action candidates outperforms independent scaling
• CoVer Architecture: A contrastive verifier designed specifically for vision-language-action alignment that demonstrates strong scaling properties with additional compute and data
• Boot-Time Compute Framework: Novel deployment strategy that precomputes diverse instruction rephrasings and generates multiple action candidates for hierarchical verification
• Empirical Validation: Comprehensive evaluation showing verification scaling outperforms policy pre-training scaling across multiple benchmarks and real-world scenarios

Methodology and Technical Approach

The paper’s core insight revolves around shifting computational investment from pre-training to test-time verification. Traditional approaches focus on scaling policy models during pre-training, but this work demonstrates that test-time compute can be more efficiently allocated.

CoVer Architecture: The contrastive verifier operates by learning to distinguish between aligned and misaligned vision-language-action triplets. Unlike generative models that must learn the full action distribution, the verifier only needs to rank candidates, making it more sample-efficient and scalable.

Hierarchical Verification Pipeline: The inference process operates in two stages:

1. High-level verification: Select the best instruction rephrasing from VLM-generated alternatives
2. Low-level verification: Choose optimal action chunks from multiple policy-generated candidates

This hierarchical approach allows the system to correct misalignments at both the semantic (instruction understanding) and execution (action generation) levels.

Boot-Time Compute: By precomputing instruction rephrasings before deployment, the system amortizes computational cost across multiple queries while maintaining diversity in the action space. This strategy proves particularly effective when deployment scenarios allow for preprocessing.

Experimental Results and Analysis

The empirical evaluation spans simulation benchmarks and real-world robotic experiments, providing comprehensive evidence for the verification scaling hypothesis.

SIMPLER Benchmark Performance:

• 22% improvement in-distribution over baseline policy scaling
• 13% improvement out-of-distribution, demonstrating generalization
• Consistent gains across different task categories and complexity levels

PolaRiS Benchmark Results:

• 14% increase in task progress metrics
• 9% improvement in overall success rate
• Particularly strong performance on long-horizon manipulation tasks

Real-World Experiments: The 45% improvement in physical robot experiments represents the most compelling evidence. Real-world settings introduce noise, uncertainty, and distribution shift that simulation cannot fully capture, making these gains especially significant.

Scaling Analysis: The paper demonstrates that verification scales more favorably than policy pre-training when given equivalent computational budgets. The test-time scaling curve shows continued improvement with additional samples, while policy scaling exhibits diminishing returns beyond certain model sizes.

Implications for Robotics and AI

This work challenges the prevailing paradigm in embodied AI that emphasizes ever-larger pre-trained models. Several implications emerge:

Computational Efficiency: Test-time verification offers a more accessible path to improved performance, as it requires less infrastructure than massive pre-training runs. Organizations with limited computational resources can leverage verification to enhance existing models.

Deployment Flexibility: The boot-time compute framework enables adaptation to specific deployment contexts without retraining. Different environments or task distributions can be addressed by adjusting verification strategies rather than model parameters.

Interpretability and Safety: Verification provides explicit reasoning about action selection, offering transparency that pure end-to-end policies lack. This interpretability becomes crucial for safety-critical applications where understanding failure modes matters.

Future Research Directions: The success of verification scaling suggests investigating similar approaches in other domains where test-time compute can be strategically allocated, such as code generation, mathematical reasoning, or multi-agent coordination.

Takeaways

1. Test-time verification scaling can outperform policy pre-training scaling for vision-language-action alignment, achieving 22% in-distribution and 45% real-world improvements
2. Joint scaling of instruction rephrasings and action candidates proves more efficient than scaling either dimension independently
3. CoVer’s contrastive verification architecture demonstrates strong scaling properties with additional computational resources and training data
4. Boot-time compute strategies enable preprocessing that amortizes computational cost while maintaining action diversity
5. Hierarchical verification at both instruction and action levels effectively addresses the intention-action gap in embodied AI systems