Academic

RouteGoT: Node-Adaptive Routing for Cost-Efficient Graph of Thoughts Reasoning

arXiv:2603.05818v1 Announce Type: new Abstract: Large Language Models (LLMs) excel at multi-step reasoning, yet increasing the structural complexity of inference does not consistently improve system-level returns. Methods such as Tree of Thoughts (ToT), Graph of Thoughts (GoT), and Adaptive Graph of Thoughts (AGoT) can boost accuracy on some benchmarks, but often introduce substantial overhead in token consumption and latency, and their gains can be unstable across task distributions-sometimes underperforming simpler Chain-of-Thought (CoT) or direct input-output prompting (IO). We attribute this inefficiency to stage-wise and node-wise heterogeneity inside GoT-style reasoning pipelines: high-quality planning and final synthesis are globally coupled and typically benefit from strong models, whereas many intermediate subtasks are localized and can be solved accurately by lighter models with far fewer tokens. Motivated by these observations, we propose RouteGoT, a budget-controllable, no

Y
Yuhang Liu, Ruijie Wang, Yunlong Chu, Bing Hao, Yumeng Lin, Shengzhong Liu, Minglai Shao · · 1 min read · 8 views
#cs.CL

arXiv:2603.05818v1 Announce Type: new Abstract: Large Language Models (LLMs) excel at multi-step reasoning, yet increasing the structural complexity of inference does not consistently improve system-level returns. Methods such as Tree of Thoughts (ToT), Graph of Thoughts (GoT), and Adaptive Graph of Thoughts (AGoT) can boost accuracy on some benchmarks, but often introduce substantial overhead in token consumption and latency, and their gains can be unstable across task distributions-sometimes underperforming simpler Chain-of-Thought (CoT) or direct input-output prompting (IO). We attribute this inefficiency to stage-wise and node-wise heterogeneity inside GoT-style reasoning pipelines: high-quality planning and final synthesis are globally coupled and typically benefit from strong models, whereas many intermediate subtasks are localized and can be solved accurately by lighter models with far fewer tokens. Motivated by these observations, we propose RouteGoT, a budget-controllable, node-adaptive routing framework for graph-structured reasoning. RouteGoT performs in-graph routing by prioritizing strong models for planning and synthesis, while dynamically allocating lightweight models and cost-effective strategies to leaf subtasks based on predicted difficulty. It further integrates explicit budget constraints into a global inference scheduler to control graph expansion under a user-specified token budget, enabling predictable performance-cost trade-offs. Experiments across reasoning, retrieval, and multi-hop QA benchmarks show that RouteGoT matching or improving accuracy while substantially reducing token usage; specifically, it achieves an average 8.1 percentage points accuracy improvement and 79.1\% output token reduction compared to AGoT. Furthermore, RouteGoT outperforms existing routing baselines by maintaining a superior cost-accuracy trade-off, demonstrating improved robustness under varying budget targets and tasks.

Executive Summary

RouteGoT: A Novel Node-Adaptive Routing Framework for Efficient Graph of Thoughts Reasoning. This study proposes RouteGoT, a routing framework that dynamically allocates lightweight models to intermediate subtasks, while prioritizing strong models for planning and synthesis. By integrating explicit budget constraints, RouteGoT achieves predictable performance-cost trade-offs, reducing token usage by 79.1% and improving accuracy by 8.1 percentage points compared to Adaptive Graph of Thoughts (AGoT). The framework's superior cost-accuracy trade-off outperforms existing routing baselines, demonstrating improved robustness under varying budget targets and tasks. This breakthrough has significant implications for the development of large language models, enabling more efficient and accurate graph-structured reasoning.

Key Points

  • ▸ RouteGoT introduces a novel, node-adaptive routing framework for graph-structured reasoning.
  • ▸ The framework dynamically allocates lightweight models to intermediate subtasks, improving efficiency.
  • ▸ RouteGoT achieves predictable performance-cost trade-offs by integrating explicit budget constraints.

Merits

Strength in Efficiency

RouteGoT significantly reduces token usage by 79.1% compared to AGoT, making it a highly efficient framework for graph-structured reasoning.

Improved Accuracy

RouteGoT achieves an average 8.1 percentage points accuracy improvement compared to AGoT, showcasing its potential for improved performance in large language models.

Robustness in Budget Targets and Tasks

RouteGoT demonstrates improved robustness under varying budget targets and tasks, making it a reliable choice for graph-structured reasoning applications.

Demerits

Limited Scalability

RouteGoT's performance may be limited by the complexity of the graph-structured reasoning pipeline, requiring further research to scale the framework for larger applications.

Overreliance on Model Quality

RouteGoT's performance is heavily dependent on the quality of the models used for planning and synthesis, which may introduce variability in performance across different tasks and budget targets.

Expert Commentary

RouteGoT represents a groundbreaking contribution to the field of natural language processing, specifically in the area of graph-structured reasoning. The framework's ability to dynamically allocate lightweight models, prioritize strong models for planning and synthesis, and integrate explicit budget constraints makes it a highly efficient and accurate solution for graph-structured reasoning applications. While the study highlights several merits of RouteGoT, including its improved efficiency, accuracy, and robustness, it also identifies limitations, such as its potential for limited scalability and overreliance on model quality. Overall, RouteGoT has significant implications for the development of large language models and AI applications, and its findings have far-reaching consequences for the field of natural language processing.

Recommendations

  • ✓ Further research is needed to scale RouteGoT for larger applications and explore its potential for other types of graph-structured reasoning pipelines.
  • ✓ Developing more robust and efficient models for planning and synthesis is crucial to unlock the full potential of RouteGoT and improve its performance in a wide range of tasks and budget targets.

Sources

Related Articles