The Art of Efficient Reasoning: Data, Reward, and Optimization
arXiv:2602.20945v1 Announce Type: new Abstract: Large Language Models (LLMs) consistently benefit from scaled Chain-of-Thought (CoT) reasoning, but also suffer from heavy computational overhead. To address this issue, efficient reasoning aims to incentivize short yet accurate thinking trajectories, typically through reward shaping with Reinforcement Learning (RL). In this paper, we systematically investigate the mechanics of efficient reasoning for LLMs. For comprehensive evaluation, we advocate for more fine-grained metrics, including length distribution conditioned on correctness and performance across a wide spectrum of token budgets ranging from 2k to 32k. First, we reveal that the training process follows a two-stage paradigm: length adaptation and reasoning refinement. After that, we conduct extensive experiments (about 0.2 million GPU hours) in a unified protocol, deconstructing training prompts and rollouts, reward shaping, and optimization strategies. In particular, a key fin
arXiv:2602.20945v1 Announce Type: new Abstract: Large Language Models (LLMs) consistently benefit from scaled Chain-of-Thought (CoT) reasoning, but also suffer from heavy computational overhead. To address this issue, efficient reasoning aims to incentivize short yet accurate thinking trajectories, typically through reward shaping with Reinforcement Learning (RL). In this paper, we systematically investigate the mechanics of efficient reasoning for LLMs. For comprehensive evaluation, we advocate for more fine-grained metrics, including length distribution conditioned on correctness and performance across a wide spectrum of token budgets ranging from 2k to 32k. First, we reveal that the training process follows a two-stage paradigm: length adaptation and reasoning refinement. After that, we conduct extensive experiments (about 0.2 million GPU hours) in a unified protocol, deconstructing training prompts and rollouts, reward shaping, and optimization strategies. In particular, a key finding is to train on relatively easier prompts, ensuring the density of positive reward signals and thus avoiding the length collapse. Meanwhile, the learned length bias can be generalized across domains. We distill all findings into valuable insights and practical guidelines, and further validate them across the Qwen3 series, ranging from 0.6B to 30B, demonstrating the robustness and generalization.
Executive Summary
The article 'The Art of Efficient Reasoning: Data, Reward, and Optimization' explores the mechanics of efficient reasoning for Large Language Models (LLMs) to mitigate the computational overhead associated with scaled Chain-of-Thought (CoT) reasoning. The authors propose a two-stage training paradigm, comprising length adaptation and reasoning refinement, and advocate for fine-grained metrics to evaluate performance. Experimental results demonstrate the efficacy of training on relatively easier prompts, ensuring the density of positive reward signals and avoiding length collapse. The study's findings are validated across the Qwen3 series, ranging from 0.6B to 30B, highlighting the robustness and generalization of the approach. This research contributes to the development of more efficient and effective LLMs, with practical implications for various applications.
Key Points
- ▸ Efficient reasoning aims to incentivize short yet accurate thinking trajectories in LLMs.
- ▸ A two-stage training paradigm is proposed, consisting of length adaptation and reasoning refinement.
- ▸ Training on relatively easier prompts ensures the density of positive reward signals and avoids length collapse.
Merits
Strengths in Methodological Approach
The article employs a systematic and comprehensive evaluation of efficient reasoning for LLMs, incorporating fine-grained metrics and extensive experiments. This approach allows for a thorough understanding of the mechanics underlying efficient reasoning and provides valuable insights into optimizing LLM performance.
Robustness and Generalization
The study's findings are validated across the Qwen3 series, ranging from 0.6B to 30B, demonstrating the robustness and generalization of the approach. This suggests that the proposed methods can be applied to a wide range of LLM architectures and sizes.
Demerits
Limitation in Generalizability
While the study demonstrates the effectiveness of the proposed approach across the Qwen3 series, it is unclear whether the findings can be generalized to other LLM architectures or domains. Further research is needed to explore the applicability of the proposed methods to more diverse settings.
Expert Commentary
The article presents a well-structured and comprehensive investigation of efficient reasoning for LLMs. The proposed two-stage training paradigm and the emphasis on fine-grained metrics provide valuable insights into optimizing LLM performance. However, the limitation in generalizability of the findings highlights the need for further research to explore the applicability of the proposed methods to more diverse settings. The study's contributions to the development of more efficient and effective LLMs are significant, with practical implications for various applications and policy implications for the development of AI systems.
Recommendations
- ✓ Future research should explore the generalizability of the proposed approach to other LLM architectures and domains.
- ✓ The development of more fine-grained metrics and evaluation protocols is essential to further optimize LLM performance and ensure the reliability of AI systems.
