Discovering Shared Logical Subspace for LLM Logical Reasoning

Paper: 2604.19716 Authors: Feihao Fang, My T. Thai, Yuanyuan Lei Categories: cs.CL, cs.AI

Problem

LLMs struggle with multi-step logical reasoning. Existing approaches either:

1. Purely refine reasoning chain in natural language
2. Attach a symbolic solver as an external module

Neither approach leverages internal reasoning capabilities across modalities.

Key Insight

LLMs contain a shared internal logical subspace that simultaneously aligns natural-language and symbolic-language views of the reasoning process. This logical subspace captures reasoning capabilities shared across views while remaining independent of surface forms.

Method: CCA-based Logical Subspace Discovery

1. Collect paired activations: Extract residual activations from both natural-language and symbolic-language reasoning chains
2. Canonical Correlation Analysis: Learn a low-dimensional subspace with maximum cross-view correlation
3. Training-free steering: Use the discovered subspace to guide reasoning chain generation

Results

On four logical reasoning benchmarks:

• Up to 11 percentage points accuracy improvement
• Generalizes well on out-of-domain problems

Why This Works

By aligning the two modalities, the model can:

• Leverage complementary reasoning signals from both views
• Transfer logical capabilities learned in one modality to another
• Maintain reasoning consistency across different representation styles

Takeaways

• Internal logical representations exist independent of language form
• Cross-modal alignment reveals hidden reasoning capabilities
• Training-free steering is effective and efficient