Contrastive explanations of BDI agents
arXiv:2602.13323v1 Announce Type: new Abstract: The ability of autonomous systems to provide explanations is important for supporting transparency and aiding the development of (appropriate) trust. Prior work has defined a mechanism for Belief-Desire-Intention (BDI) agents to be able to answer questions of the form ``why did you do action $X$?''. However, we know that we ask \emph{contrastive} questions (``why did you do $X$ \emph{instead of} $F$?''). We therefore extend previous work to be able to answer such questions. A computational evaluation shows that using contrastive questions yields a significant reduction in explanation length. A human subject evaluation was conducted to assess whether such contrastive answers are preferred, and how well they support trust development and transparency. We found some evidence for contrastive answers being preferred, and some evidence that they led to higher trust, perceived understanding, and confidence in the system's correctness. We also e
arXiv:2602.13323v1 Announce Type: new Abstract: The ability of autonomous systems to provide explanations is important for supporting transparency and aiding the development of (appropriate) trust. Prior work has defined a mechanism for Belief-Desire-Intention (BDI) agents to be able to answer questions of the form ``why did you do action $X$?''. However, we know that we ask \emph{contrastive} questions (``why did you do $X$ \emph{instead of} $F$?''). We therefore extend previous work to be able to answer such questions. A computational evaluation shows that using contrastive questions yields a significant reduction in explanation length. A human subject evaluation was conducted to assess whether such contrastive answers are preferred, and how well they support trust development and transparency. We found some evidence for contrastive answers being preferred, and some evidence that they led to higher trust, perceived understanding, and confidence in the system's correctness. We also evaluated the benefit of providing explanations at all. Surprisingly, there was not a clear benefit, and in some situations we found evidence that providing a (full) explanation was worse than not providing any explanation.
Executive Summary
The article 'Contrastive explanations of BDI agents' explores the importance of autonomous systems providing explanations to enhance transparency and trust. It builds on prior work by extending the ability of Belief-Desire-Intention (BDI) agents to answer contrastive questions, such as 'why did you do X instead of F?'. The study demonstrates that contrastive explanations significantly reduce explanation length and are preferred by users, leading to higher trust, perceived understanding, and confidence in the system's correctness. Surprisingly, the study also finds that providing full explanations may not always be beneficial and can sometimes be worse than providing no explanation at all.
Key Points
- ▸ Extension of BDI agents to answer contrastive questions.
- ▸ Contrastive explanations reduce explanation length significantly.
- ▸ Human subject evaluation shows preference for contrastive answers.
- ▸ Contrastive answers may lead to higher trust and perceived understanding.
- ▸ Providing full explanations may not always be beneficial.
Merits
Innovative Approach
The article introduces a novel method for BDI agents to provide contrastive explanations, addressing a gap in the literature.
Empirical Evidence
The study provides empirical evidence from both computational and human subject evaluations, strengthening the validity of the findings.
Practical Implications
The findings have practical implications for the design of autonomous systems, emphasizing the importance of concise and relevant explanations.
Demerits
Limited Scope
The study focuses on a specific type of agent (BDI) and a particular form of explanation, which may limit the generalizability of the findings.
Mixed Results
The finding that providing full explanations may not always be beneficial is counterintuitive and requires further investigation to understand the underlying mechanisms.
Sample Size
The human subject evaluation may have a limited sample size, which could affect the robustness of the conclusions drawn.
Expert Commentary
The article 'Contrastive explanations of BDI agents' presents a significant advancement in the field of explainable AI by addressing the need for autonomous systems to provide contrastive explanations. The study's innovative approach and empirical evidence provide valuable insights into how different forms of explanations can impact user trust and understanding. However, the mixed results regarding the benefit of providing full explanations warrant further investigation. The study's findings have important practical and policy implications, highlighting the need for designers and policymakers to consider the effectiveness of different types of explanations in the design and regulation of autonomous systems. Overall, the article makes a strong contribution to the literature and opens up new avenues for research in the field of human-AI interaction.
Recommendations
- ✓ Conduct further studies to explore the conditions under which providing full explanations may be detrimental.
- ✓ Investigate the generalizability of the findings to other types of autonomous systems and explanation forms.