Paper: 2602.04118 Authors: John X. Morris, Niloofar Mireshghallah, Mark Ibrahim, Saeed Mahloujifar Affiliations: Meta Categories: cs.LG

Abstract

TinyLoRA scales low-rank adapters to extremely small sizes, demonstrating that Qwen2.5 8B can achieve 91% accuracy on GSM8K with only 13 trained parameters in bf16 (26 total bytes). The work recovers 90% of performance improvements while training 1000x fewer parameters across benchmarks including AIME, AMC, and MATH500.

Key Contributions

  • Extreme parameter efficiency: 91% GSM8K accuracy with just 13 trainable parameters
  • RL vs SFT insight: Models trained with SFT require 100-1000x larger updates to reach the same performance as RL-trained models
  • Challenges LoRA assumptions: Questions whether even rank=1 LoRA is necessary for reasoning tasks

The Core Finding

The most striking result is the dramatic difference between RL and SFT training regimes. When fine-tuning for reasoning tasks, RL-based training (like GRPO) can achieve strong performance with vanishingly small parameter counts, while SFT requires orders of magnitude more parameters for the same result.

This suggests that RL training doesn’t need to learn new knowledge — it primarily needs to activate existing capabilities in the pretrained model. The update required is so small that 13 parameters suffice.

Why This Matters

  1. Rethinking fine-tuning: If 13 parameters can unlock reasoning, the pretrained model already “knows” how to reason — it just needs a tiny nudge
  2. RL is fundamentally different from SFT: RL fine-tuning operates in a qualitatively different regime, requiring far less parametric change
  3. Practical implications: Extreme parameter efficiency means fine-tuning can be done with negligible storage and compute overhead
  4. Theoretical questions: What exactly are those 13 parameters doing? Understanding this could reveal how reasoning is encoded in LLMs

Takeaways

  1. Less is more: Extreme parameter reduction reveals that RL fine-tuning for reasoning is about activation, not learning
  2. Training method matters more than adapter size: The choice between RL and SFT has a larger impact than the number of trainable parameters
  3. Implications for deployment: Near-zero storage overhead for task-specific adapters

论文: 2602.04118 作者: John X. Morris, Niloofar Mireshghallah, Mark Ibrahim, Saeed Mahloujifar 机构: Meta 分类: cs.LG

摘要

TinyLoRA将低秩适配器缩放到极小规模，证明Qwen2.5 8B仅用13个bf16训练参数（共26字节）即可在GSM8K上达到91%准确率。该工作在AIME、AMC和MATH500等基准测试中，用少1000倍的参数恢复了90%的性能提升。

主要贡献

  • 极致参数效率：仅13个可训练参数即达91% GSM8K准确率
  • RL vs SFT洞察：SFT训练的模型需要100-1000倍更大的更新才能达到与RL训练模型相同的性能
  • 挑战LoRA假设：质疑推理任务是否需要rank=1的LoRA

核心发现

最引人注目的结果是RL和SFT训练方式之间的巨大差异。在微调推理任务时，基于RL的训练（如GRPO）可以用极少的参数量达到强大的性能，而SFT需要多几个数量级的参数才能达到相同效果。

这表明RL训练不需要学习新知识——它主要需要激活预训练模型中已有的能力。所需的更新如此之小，以至于13个参数就足够了。

重要意义

  1. 重新思考微调：如果13个参数就能解锁推理能力，那么预训练模型已经”知道”如何推理——只需要一个微小的推动
  2. RL与SFT本质不同：RL微调在质上处于不同的范畴，需要的参数变化远少于SFT
  3. 实际意义：极致的参数效率意味着微调可以以几乎可忽略的存储和计算开销完成
  4. 理论问题：这13个参数究竟在做什么？理解这一点可能揭示推理能力在LLM中的编码方式

要点总结

  1. 少即是多：极端参数缩减揭示了RL推理微调的本质是激活而非学习
  2. 训练方法比适配器大小更重要：RL和SFT之间的选择比可训练参数数量的影响更大
  3. 部署意义：任务特定适配器几乎零存储开销