Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs
arXiv:2603.00578v1 Announce Type: new Abstract: Long chain-of-thought~(CoT) has become a dominant paradigm for enhancing the reasoning capability of large reasoning models~(LRMs); however, the performance gains often come with a substantial increase in reasoning budget. Recent studies show that existing CoT paradigms tend to induce systematic overthinking, unnecessarily coupling reasoning capability with reasoning cost. Most prior approaches reduce token usage through post hoc techniques such as token compression, truncation, or length penalties, without explicitly addressing the core mechanisms of reasoning. We propose \textbf{Draft-Thinking}, which guides models to first learn a concise \textit{draft-style} reasoning structure that retains only the critical reasoning steps. Through a \textit{progressive curriculum learning}, the model stably internalizes this efficient reasoning pattern as its capability scales. Moreover, Draft-Thinking introduces adaptive prompting, which elevates
arXiv:2603.00578v1 Announce Type: new Abstract: Long chain-of-thought~(CoT) has become a dominant paradigm for enhancing the reasoning capability of large reasoning models~(LRMs); however, the performance gains often come with a substantial increase in reasoning budget. Recent studies show that existing CoT paradigms tend to induce systematic overthinking, unnecessarily coupling reasoning capability with reasoning cost. Most prior approaches reduce token usage through post hoc techniques such as token compression, truncation, or length penalties, without explicitly addressing the core mechanisms of reasoning. We propose \textbf{Draft-Thinking}, which guides models to first learn a concise \textit{draft-style} reasoning structure that retains only the critical reasoning steps. Through a \textit{progressive curriculum learning}, the model stably internalizes this efficient reasoning pattern as its capability scales. Moreover, Draft-Thinking introduces adaptive prompting, which elevates reasoning depth to a flexible, model-selectable behavior. Extensive experiments demonstrate that Draft-Thinking substantially reduces reasoning budget while largely preserving reasoning performance; for example, on MATH500, it achieves an 82.6\% reduction in reasoning budget at the cost of only a 2.6\% performance drop.
Executive Summary
This article introduces Draft-Thinking, a novel paradigm for enhancing the reasoning capability of Large Reasoning Models (LRMs) in a Long Chain-of-Thought (CoT) setting. Draft-Thinking learns an efficient, concise reasoning structure through progressive curriculum learning and adaptive prompting, reducing token usage and reasoning budget. The approach is demonstrated to achieve an 82.6% reduction in reasoning budget with a mere 2.6% performance drop on the MATH500 dataset. Draft-Thinking addresses the issue of systematic overthinking in existing CoT paradigms, offering a more scalable and efficient solution for LRM development. The approach has significant implications for AI model development, particularly in resource-constrained environments.
Key Points
- ▸ Draft-Thinking learns an efficient, concise reasoning structure through progressive curriculum learning and adaptive prompting.
- ▸ The approach substantially reduces token usage and reasoning budget in a Long Chain-of-Thought (CoT) setting.
- ▸ Draft-Thinking achieves a significant reduction in reasoning budget with minimal performance drop on the MATH500 dataset.
Merits
Strength in Scalability
Draft-Thinking's progressive curriculum learning enables the model to stably internalize efficient reasoning patterns as its capability scales, addressing the issue of overthinking in existing CoT paradigms.
Flexibility in Reasoning Depth
The adaptive prompting mechanism allows for flexible, model-selectable reasoning depth, enhancing the model's ability to adapt to different tasks and environments.
Demerits
Overreliance on Curricula
The effectiveness of Draft-Thinking relies heavily on the design of the curriculum learning process, which may be specific to the task or dataset at hand.
Limited Generalizability
The approach may not generalize well to other domains or tasks, requiring further adaptation and fine-tuning to achieve optimal performance.
Expert Commentary
The introduction of Draft-Thinking marks a significant step forward in the development of Large Reasoning Models, addressing the issue of overthinking in existing CoT paradigms. The approach's focus on scalability and efficiency makes it an attractive solution for resource-constrained environments. However, further research is needed to fully understand the limitations and generalizability of Draft-Thinking. The potential implications of this work extend beyond AI model development, influencing the broader landscape of AI research and policy.
Recommendations
- ✓ Future research should focus on adapting Draft-Thinking to other domains and tasks, exploring its generalizability and limitations.
- ✓ The development of AI policies and regulations should consider the implications of Draft-Thinking for model efficiency and scalability.