Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution
arXiv:2602.16154v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning sometimes fails to faithfully reflect the true computation of a large language model (LLM), hampering its utility in explaining how LLMs arrive at their answers. Moreover, optimizing for faithfulness and interpretability in reasoning often degrades task performance. To address this tradeoff and improve CoT faithfulness, we propose Reasoning Execution by Multiple Listeners (REMUL), a multi-party reinforcement learning approach. REMUL builds on the hypothesis that reasoning traces which other parties can follow will be more faithful. A speaker model generates a reasoning trace, which is truncated and passed to a pool of listener models who "execute" the trace, continuing the trace to an answer. Speakers are rewarded for producing reasoning that is clear to listeners, with additional correctness regularization via masked supervised finetuning to counter the tradeoff between faithfulness and performance. On m
arXiv:2602.16154v1 Announce Type: new Abstract: Chain-of-thought (CoT) reasoning sometimes fails to faithfully reflect the true computation of a large language model (LLM), hampering its utility in explaining how LLMs arrive at their answers. Moreover, optimizing for faithfulness and interpretability in reasoning often degrades task performance. To address this tradeoff and improve CoT faithfulness, we propose Reasoning Execution by Multiple Listeners (REMUL), a multi-party reinforcement learning approach. REMUL builds on the hypothesis that reasoning traces which other parties can follow will be more faithful. A speaker model generates a reasoning trace, which is truncated and passed to a pool of listener models who "execute" the trace, continuing the trace to an answer. Speakers are rewarded for producing reasoning that is clear to listeners, with additional correctness regularization via masked supervised finetuning to counter the tradeoff between faithfulness and performance. On multiple reasoning benchmarks (BIG-Bench Extra Hard, MuSR, ZebraLogicBench, and FOLIO), REMUL consistently and substantially improves three measures of faithfulness -- hint attribution, early answering area over the curve (AOC), and mistake injection AOC -- while also improving accuracy. Our analysis finds that these gains are robust across training domains, translate to legibility gains, and are associated with shorter and more direct CoTs.
Executive Summary
This article proposes the Reasoning Execution by Multiple Listeners (REMUL) approach to improve chain-of-thought (CoT) faithfulness in large language models (LLMs). By leveraging multiple listener models, REMUL rewards speakers for generating clear and understandable reasoning traces, while also incorporating correctness regularization to balance faithfulness and performance. The results show substantial improvements in faithfulness on multiple reasoning benchmarks, including hint attribution, early answering area over the curve, and mistake injection area over the curve. Notably, these gains are robust across training domains and translate to legibility improvements. The REMUL approach has the potential to enhance the explainability and transparency of LLMs, making them more trustworthy and reliable in practical applications.
Key Points
- ▸ REMUL improves CoT faithfulness by leveraging multiple listener models.
- ▸ REMUL balances faithfulness and performance through correctness regularization.
- ▸ REMUL achieves substantial improvements in faithfulness on multiple reasoning benchmarks.
Merits
Improved Faithfulness
REMUL consistently improves CoT faithfulness on multiple benchmarks, making it a valuable contribution to the field of explainable AI.
Balanced Performance
REMUL's use of correctness regularization allows for a balance between faithfulness and performance, making it a practical solution for real-world applications.
Demerits
Scalability Limitations
REMUL may face scalability challenges with large numbers of listener models, which could impact its practicality in real-world applications.
Dependence on Training Data
REMUL's performance may be highly dependent on the quality and diversity of the training data, which could impact its robustness and generalizability.
Expert Commentary
The REMUL approach is a significant contribution to the field of explainable AI, as it addresses a critical challenge in the development of LLMs. By leveraging multiple listener models, REMUL provides a novel solution to the tradeoff between faithfulness and performance. However, further research is needed to address scalability limitations and dependence on training data. Additionally, the implications of REMUL's focus on balancing faithfulness and performance are far-reaching, particularly in areas such as policy and regulation.
Recommendations
- ✓ Further research is needed to address scalability limitations and dependence on training data.
- ✓ REMUL's approach should be explored in other areas of AI, such as computer vision and decision-making.
