Improving Lean4 Autoformalization via Cycle Consistency Fine-tuning

Paper: 2603.24372 Authors: Arsen Shebzukhov Categories: cs.CL

Abstract

This paper studies autoformalization from natural language to Lean4. It fine-tunes Qwen3.5-2B with LoRA on FineLeanCorpus and compares three setups: supervised fine-tuning with curriculum, supervised fine-tuning without curriculum, and GRPO reinforcement learning using a cycle-consistency reward based on sentence-embedding similarity between the original statement and a round-tripped NL version. The RL approach gives the strongest results on both an unseen FineLeanCorpus subset and PutnamBench, with little change in formalization quality.

Key Contributions

• Uses Qwen3.5-2B with LoRA for natural-language-to-Lean4 autoformalization
• Compares curriculum SFT, shuffled SFT, and GRPO with cycle-consistency reward
• Defines cycle consistency through embedding similarity after NL → Lean4 → NL round-tripping
• Shows the RL approach performs best on both evaluation sets
• Finds no measurable benefit from curriculum ordering over shuffled training

Methodology

The work treats autoformalization as a sequence-to-sequence generation problem and adapts Qwen3.5-2B with LoRA on FineLeanCorpus. The model is trained under three regimes so the authors can isolate the effect of curriculum ordering and reinforcement learning. In the GRPO setup, the reward is computed from a cycle-consistency signal: a generated Lean4 statement is translated back to natural language, and the resulting text is compared with the original statement using sentence-embedding cosine similarity.

This makes the optimization target less about matching formal syntax alone and more about preserving the meaning of the original theorem statement. That matters because a formal proof script can look plausible while quietly drifting from the intended semantics.

Results

The reinforcement-learning variant outperforms both supervised baselines on the unseen FineLeanCorpus subset and on PutnamBench. The paper reports that formalization quality remains largely stable, so the improvement comes from better semantic preservation rather than a trade-off with output quality.

The curriculum experiment is also informative: ordering the training examples by difficulty does not provide a clear advantage over shuffled training. That suggests the cycle-consistency objective is doing more of the useful work than dataset ordering in this setting.

Takeaways

1. Autoformalization benefits from reward signals that check meaning, not just surface-form correctness
2. Cycle consistency is a practical way to measure whether formalization preserved the original statement
3. GRPO can improve Lean4 autoformalization without hurting output quality
4. Curriculum learning did not show a clear gain in this experiment
5. Semantic round-tripping is a useful pattern for training language-to-formal-language systems