GRIL: Grounded Reasoning via Interactive Reinforcement Learning

Paper: 2604.19656 Authors: Yiwen Qiu, Linjuan Wu, et al. (Zhejiang University, Tencent, Xiaohongshu) Categories: cs.CL, cs.AI

Problem: Ungrounded Reasoning

Large language models often fabricate information when inputs are incomplete, producing confident but unreliable conclusions. This failure mode arises from lack of inferential boundary awareness - the ability to recognize when necessary premises for valid inference are missing.

Key Insight

The issue is NOT insufficient reasoning capability, but the lack of awareness of inference boundaries. Models need to:

1. Detect when information is insufficient
2. Stop and ask for clarification
3. Resume reasoning after clarification

GRIL Framework

Grounded Reasoning via Interactive Reinforcement Learning decomposes reasoning into two stages:

1. Clarify and Pause: Identifies whether available information is sufficient
2. Grounded Reasoning: Performs task solving once necessary premises are established

Stage-Specific Rewards

• Penalize hallucinations when model fabricates without flagging
• Reward proactive stopping when premise detection fails
• Reward natural resumption after clarification

Results

On GSM8K-Insufficient and MetaMATH-Insufficient:

• Premise detection improved by 45%
• Task success increased by 30%
• Average response length reduced by 20%+

Also shows robustness to noisy user responses and generalization to OOD tasks.

Takeaways

• Teaching models to “know what they don’t know” is crucial
• Interactive RL can instill inferential boundary awareness
• Reducing unnecessary generation improves both efficiency and accuracy