Investigating Co-Constructive Behavior of Large Language Models in Explanation Dialogues

The ability to generate explanations that are understood by explainees is the quintessence of explainable artificial intelligence. Since un- derstanding depends on the explainee’s back- ground and needs, recent research focused on co-constructive explanation dialogues, where an explainer continuously monitors the ex- plainee’s understanding and adapts their expla- nations dynamically. We investigate the ability of large language models (LLMs) to engage as explainers in co-constructive explanation dia- logues. In particular, we present a user study in which explainees interact with an LLM in two settings, one of which involves the LLM being instructed to explain a topic co-constructively. We evaluate the explainees’ understanding be- fore and after the dialogue, as well as their perception of the LLMs’ co-constructive behav- ior. Our results suggest that LLMs show some co-constructive behaviors, such as asking ver- ification questions, that foster the explainees’ engagement and can improve understanding of a topic. However, their ability to effectively monitor the current understanding and scaffold the explanations accordingly remains limited.