Games That Teach, Chats That Convince: Comparing Interactive and Static Formats for Persuasive Learning
arXiv:2602.17905v1 Announce Type: cross Abstract: Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held constant. Grounding on identical arguments and factual content across conditions, we present a controlled user study comparing three modes of information delivery: static essays, conversational chatbots, and narrative text-based games. Across subjective measures, the chatbot condition consistently outperformed the other modes and increased perceived importance of the topic. However, perceived learning did not reliably align with objective outcomes: participants in the text-based game condition reported learning less than those reading essays, yet achieved higher scores on a delayed (24-hour) knowledge quiz. Additional exploratory analyses further suggest that common engagement proxies, such as verbo
arXiv:2602.17905v1 Announce Type: cross Abstract: Interactive systems such as chatbots and games are increasingly used to persuade and educate on sustainability-related topics, yet it remains unclear how different delivery formats shape learning and persuasive outcomes when content is held constant. Grounding on identical arguments and factual content across conditions, we present a controlled user study comparing three modes of information delivery: static essays, conversational chatbots, and narrative text-based games. Across subjective measures, the chatbot condition consistently outperformed the other modes and increased perceived importance of the topic. However, perceived learning did not reliably align with objective outcomes: participants in the text-based game condition reported learning less than those reading essays, yet achieved higher scores on a delayed (24-hour) knowledge quiz. Additional exploratory analyses further suggest that common engagement proxies, such as verbosity and interaction length, are more closely related to subjective experience than to actual learning. These findings highlight a dissociation between how persuasive experiences feel and what participants retain, and point to important design trade-offs between interactivity, realism, and learning in persuasive systems and serious games.
Executive Summary
This study compares the effectiveness of interactive and static formats for persuasive learning, including chatbots, games, and essays, on sustainability-related topics. The results show that while chatbots outperformed other formats in subjective measures, the narrative text-based game condition achieved higher scores on a delayed knowledge quiz. The study highlights a dissociation between perceived learning and actual retention, emphasizing the importance of design trade-offs in persuasive systems and serious games.
Key Points
- ▸ Chatbots outperformed other formats in subjective measures, increasing perceived importance of the topic
- ▸ Narrative text-based games achieved higher scores on a delayed knowledge quiz despite lower reported learning
- ▸ A dissociation exists between perceived learning and actual retention, with engagement proxies not reliably predicting learning outcomes
Merits
Controlled Study Design
The study's controlled design allows for a direct comparison of different formats, holding content constant, which strengthens the validity of the findings.
Demerits
Limited Generalizability
The study's focus on sustainability-related topics may limit the generalizability of the findings to other domains, and the use of a specific population may not be representative of all learners.
Expert Commentary
This study contributes to our understanding of the complex relationships between format, engagement, and learning outcomes in persuasive systems and serious games. The findings highlight the importance of moving beyond subjective measures of learning and engagement, and instead using objective outcomes to evaluate the effectiveness of different formats. The study's results have significant implications for the design of educational experiences, emphasizing the need for a nuanced approach that balances interactivity, realism, and learning outcomes.
Recommendations
- ✓ Future studies should investigate the mechanisms underlying the dissociation between perceived learning and actual retention, and explore the potential for interactive formats to promote transfer of learning to real-world contexts.
- ✓ Designers of persuasive systems and serious games should prioritize the use of objective outcomes to evaluate the effectiveness of different formats, and consider using a combination of subjective and objective measures to gain a more comprehensive understanding of learning and engagement.
