Illocutionary Explanation Planning for Source-Faithful Explanations in Retrieval-Augmented Language Models
arXiv:2604.06211v1 Announce Type: new Abstract: Natural language explanations produced by large language models (LLMs) are often persuasive, but not necessarily scrutable: users cannot easily verify whether the claims in an explanation are supported by evidence. In XAI, this motivates a focus on faithfulness and traceability, i.e., the extent to which an explanation's claims can be grounded in, and traced back to, an explicit source. We study these desiderata in retrieval-augmented generation (RAG) for programming education, where textbooks provide authoritative evidence. We benchmark six LLMs on 90 Stack Overflow questions grounded in three programming textbooks and quantify source faithfulness via source adherence metrics. We find that non Retrieval-Augmented Generation (RAG) models have median source adherence of 0%, while baseline RAG systems still exhibit low median adherence (22-40%, depending on the model). Motivated by Achinstein's illocutionary theory of explanation, we intro
arXiv:2604.06211v1 Announce Type: new Abstract: Natural language explanations produced by large language models (LLMs) are often persuasive, but not necessarily scrutable: users cannot easily verify whether the claims in an explanation are supported by evidence. In XAI, this motivates a focus on faithfulness and traceability, i.e., the extent to which an explanation's claims can be grounded in, and traced back to, an explicit source. We study these desiderata in retrieval-augmented generation (RAG) for programming education, where textbooks provide authoritative evidence. We benchmark six LLMs on 90 Stack Overflow questions grounded in three programming textbooks and quantify source faithfulness via source adherence metrics. We find that non Retrieval-Augmented Generation (RAG) models have median source adherence of 0%, while baseline RAG systems still exhibit low median adherence (22-40%, depending on the model). Motivated by Achinstein's illocutionary theory of explanation, we introduce illocutionary macro-planning as a descriptive design principle for source-faithful explanations and instantiate it with chain-of-illocution prompting (CoI), which expands a query into implicit explanatory questions that drive retrieval. Across models, CoI yields statistically significant gains (up to 63%) in source adherence, although absolute adherence remains moderate and the gains are weak or non-significant for some models. A user study with 165 retained participants (220 recruited) indicates that these gains do not harm satisfaction, relevance, or perceived correctness.
Executive Summary
This article addresses the critical challenge of ensuring source faithfulness in explanations generated by Retrieval-Augmented Language Models (RAG-LLMs), particularly in programming education where authoritative textbooks serve as evidence. The authors benchmark six LLMs on Stack Overflow questions, revealing alarmingly low source adherence in both non-RAG (0%) and baseline RAG systems (22-40%). Drawing on Achinstein's illocutionary theory, they propose 'illocutionary macro-planning' and instantiate it with 'chain-of-illocution' (CoI) prompting, which significantly improves source adherence (up to 63%) across models. A user study confirms these gains do not diminish user satisfaction or perceived correctness, highlighting a promising avenue for more trustworthy AI explanations.
Key Points
- ▸ LLMs, even RAG-augmented ones, exhibit low source faithfulness in their explanations, making verification difficult for users.
- ▸ The study benchmarks LLMs on programming education tasks, using textbooks as ground truth sources.
- ▸ Achinstein's illocutionary theory informs the development of 'illocutionary macro-planning' and 'chain-of-illocution' (CoI) prompting.
- ▸ CoI prompting significantly improves source adherence (up to 63%) but absolute adherence remains moderate.
- ▸ User studies indicate that increased source faithfulness via CoI does not negatively impact user satisfaction or perceived correctness.
Merits
Novel Theoretical Grounding
The application of Achinstein's illocutionary theory of explanation to LLM explanation planning is a sophisticated and novel contribution, moving beyond purely technical fixes to a more philosophical understanding of what constitutes a 'good' explanation.
Empirical Rigor
The benchmarking of six LLMs across 90 questions and the quantification of source adherence provide robust empirical evidence of the problem's severity and the effectiveness of the proposed solution.
Practical Intervention
Chain-of-Illocution (CoI) prompting offers a concrete, implementable strategy for improving source faithfulness, directly addressing a critical limitation of current RAG systems.
User-Centric Validation
The inclusion of a user study with a substantial participant count demonstrates a commitment to evaluating the intervention's impact on user experience, ensuring that technical improvements do not come at the cost of usability or perceived quality.
Demerits
Moderate Absolute Adherence
Despite significant gains, the 'moderate' absolute adherence achieved (even with CoI) suggests that substantial work remains to achieve truly robust source faithfulness, particularly in high-stakes domains.
Varying Model Efficacy
The observation that gains are 'weak or non-significant for some models' indicates that CoI's effectiveness is not universally applicable and may depend on underlying model architecture or training, requiring further investigation.
Domain Specificity
While programming education is a valid testbed, the generalizability of these findings and the CoI approach to other domains (e.g., law, medicine) with different evidentiary structures needs further exploration.
Subjectivity of 'Explanation'
Achinstein's theory, while valuable, introduces a layer of philosophical interpretation. The operationalization of 'illocutionary macro-planning' into concrete prompts may still carry inherent subjective elements, potentially limiting its universal application or requiring extensive domain-specific tuning.
Expert Commentary
This paper makes a significant theoretical and practical stride in addressing a foundational challenge for AI: generating explanations that are not merely plausible but demonstrably true to their sources. The application of Achinstein's illocutionary theory is particularly insightful, elevating the discourse beyond purely algorithmic optimizations to consider the communicative intent and structure of a genuine explanation. The empirical findings are stark, underscoring the pervasive issue of unfaithful explanations even in RAG systems. While the gains from CoI are encouraging, the 'moderate' absolute adherence highlights that this is an ongoing battle. For legal and academic applications, where textual fidelity is paramount, this work offers a compelling blueprint. The call for 'implicit explanatory questions' resonates with how experienced legal professionals construct arguments, implicitly retrieving and synthesizing evidence. Further research should focus on extending this framework to more complex, multi-source legal reasoning, potentially involving conflicting evidence or diverse interpretative frameworks, and exploring its robustness across different legal domains and languages.
Recommendations
- ✓ Investigate the application of illocutionary planning in highly adversarial or ambiguous domains, such as legal interpretation where multiple valid interpretations of a source may exist.
- ✓ Explore methods to dynamically adapt CoI prompting based on the complexity of the query and the nature of the source material, potentially incorporating meta-cognitive prompting strategies.
- ✓ Develop more granular metrics for source adherence that can distinguish between minor deviations and fundamental misrepresentations, especially in the context of nuanced legal or scientific argumentation.
- ✓ Conduct further research into the 'weak or non-significant' performance of CoI on certain models to understand underlying architectural limitations or biases, and to develop model-agnostic improvements.
- ✓ Examine the scalability of CoI for very large and diverse knowledge bases, addressing potential computational overheads and retrieval efficiency challenges.
Sources
Original: arXiv - cs.CL