RVR: Retrieve-Verify-Retrieve for Comprehensive Question Answering

Paper: 2602.18425 Authors: Deniz Qian, Hung-Ting Chen, Eunsol Choi Categories: cs.CL, cs.IR

Abstract

This paper introduces Retrieve-Verify-Retrieve (RVR), a novel multi-round retrieval framework designed to comprehensively answer queries with multiple valid answers. The approach addresses a critical limitation in traditional retrieval systems: their inability to capture the full diversity of valid answers. RVR operates through iterative cycles where a retriever fetches candidate documents, a verifier identifies high-quality subsets, and subsequent rounds augment the query with previously verified documents to discover uncovered answers. The method demonstrates substantial improvements over baselines, achieving at least 10% relative and 3% absolute gains in complete recall on the QAMPARI dataset, with consistent performance across out-of-domain datasets.

Key Contributions

• Introduction of a multi-round retrieval framework that systematically maximizes answer coverage through iterative query augmentation
• A verification mechanism that identifies high-quality document subsets to guide subsequent retrieval rounds
• Demonstration that the approach works effectively with off-the-shelf retrievers and benefits further from fine-tuning
• Empirical validation showing significant improvements over baseline methods including agentic search approaches
• Generalization across multiple datasets (QAMPARI, QUEST, WebQuestionsSP) and different base retriever architectures

Methodology and Framework Design

The RVR framework operates in three distinct phases per iteration. In the initial retrieval phase, the system takes the original query and retrieves a candidate document set using a standard retriever. The verification phase then employs a verifier model to assess document quality and identify which documents contain valid, high-quality answers to the query. This verification step is crucial as it filters noise and ensures only relevant information propagates to subsequent rounds.

For subsequent retrieval rounds, RVR implements query augmentation by incorporating previously verified documents into the search context. This augmentation strategy enables the retriever to discover answers that differ from those already found, effectively exploring the answer space more comprehensively. The iterative nature of this process allows the system to progressively build a diverse set of valid answers rather than converging on a single answer type.

The framework’s design is retriever-agnostic, meaning it can work with various retrieval architectures. However, the authors demonstrate that fine-tuning retrievers specifically for this multi-round inference procedure yields additional performance gains, suggesting that retrievers can learn to better support iterative exploration when trained with this objective in mind.

Experimental Results and Performance Analysis

RVR demonstrates strong empirical performance across multiple benchmarks. On the QAMPARI dataset, which specifically tests multi-answer retrieval capabilities, the method achieves at least 10% relative improvement and 3% absolute gain in complete recall percentage compared to baseline approaches. This improvement is particularly significant as complete recall measures whether the system successfully retrieves all valid answers, not just some subset.

The framework’s robustness is evidenced by consistent gains on out-of-domain datasets QUEST and WebQuestionsSP. This cross-dataset performance indicates that RVR’s iterative approach generalizes well beyond its training distribution. The method outperforms not only traditional retrieval baselines but also more sophisticated agentic search approaches, suggesting that the structured iterative framework provides advantages over less constrained exploration strategies.

Ablation studies reveal that both the verification component and query augmentation mechanism contribute substantially to performance. The verification step prevents error propagation by filtering low-quality documents, while query augmentation enables the discovery of diverse answer types that might be missed in single-round retrieval.

Implications for Information Retrieval Systems

RVR represents a paradigm shift in how retrieval systems approach comprehensive question answering. Traditional retrieval systems optimize for relevance to a single query formulation, which inherently limits their ability to capture answer diversity. By introducing structured iteration with verification, RVR addresses this fundamental limitation while maintaining computational efficiency through its focused exploration strategy.

The framework’s compatibility with existing retrieval architectures makes it practically deployable. Organizations can enhance their current retrieval systems by adding the verification and iteration components without requiring complete system redesigns. The additional computational cost of multiple retrieval rounds is offset by the verification step’s ability to focus subsequent searches, preventing exponential growth in retrieved document sets.

For applications requiring comprehensive information gathering—such as research assistance, fact-checking, or exploratory search—RVR offers a principled approach to maximizing answer coverage. The method’s success suggests that future retrieval systems should consider answer diversity as a first-class optimization objective rather than a secondary concern.

Takeaways

1. Multi-round retrieval with verification significantly outperforms single-round approaches for queries with multiple valid answers, achieving 10%+ relative improvements in complete recall
2. Query augmentation with previously verified documents enables systematic exploration of diverse answer spaces without redundant retrieval
3. The framework generalizes across different retrieval architectures and datasets, demonstrating robustness beyond specific implementation choices
4. Verification mechanisms are critical for preventing error propagation in iterative retrieval systems
5. Fine-tuning retrievers for multi-round inference scenarios provides additional performance gains beyond using off-the-shelf models