conference paper published yes Olcay Türk author Stefan Teodorov Lazarov author 903450009-0009-0892-9483 Yu Wang author Hendrik Buschmeier author 764560000-0002-9613-5713 Angela Grimminger author 57578 Petra Wagner author 74505 660 department The 26th ACM International Conference on Multimodal Interaction TRR 318 - A02: TRR 318 - Verstehensprozess einer Erklärung beobachten und auswerten (Teilprojekt A02) project In explanatory interactions, explainees are expected to continuously provide feedback to explainers by signaling whether they understand an ongoing explanation. The study presented in this paper is based on the hypothesis that explainees use a set of multimodal cues, including vocalizations, facial expressions, and movements of the torso, head, and hands, to do so. We test this hypothesis by building a random forest classifier based on a multimodal corpus of dyadic explanations (21 explainers and explainees), in which windows of understanding or non-understanding were identified by participants in a retrospective video recall task. Results show that sequences of understanding can indeed be differentiated from those of non-understanding, and that a diverse set of predictors covering a wide range of modalities contributes to this classification. Due to data sparsity and a high degree of individual variation, the generalizability of our results is currently limited, but they support our hypothesis of the relevance of multimodal display in explanatory interactions. 2024San José, Costa Rica eng 10.1145/3678957.3685741 449-458 O. Türk, S. T. Lazarov, Y. Wang, H. Buschmeier, A. Grimminger, and P. Wagner, “Predictability of understanding in explanatory interactions based on multimodal cues,” in <i>Proceedings of the 26th ACM International Conference on Multimodal Interaction</i>, San José, Costa Rica, 2024, pp. 449–458, doi: <a href="https://doi.org/10.1145/3678957.3685741">10.1145/3678957.3685741</a>. Türk, O., Lazarov, S. T., Wang, Y., Buschmeier, H., Grimminger, A., &#38; Wagner, P. (2024). Predictability of understanding in explanatory interactions based on multimodal cues. <i>Proceedings of the 26th ACM International Conference on Multimodal Interaction</i>, 449–458. <a href="https://doi.org/10.1145/3678957.3685741">https://doi.org/10.1145/3678957.3685741</a> O. Türk, S.T. Lazarov, Y. Wang, H. Buschmeier, A. Grimminger, P. Wagner, in: Proceedings of the 26th ACM International Conference on Multimodal Interaction, San José, Costa Rica, 2024, pp. 449–458. Türk, Olcay, Stefan Teodorov Lazarov, Yu Wang, Hendrik Buschmeier, Angela Grimminger, and Petra Wagner. “Predictability of Understanding in Explanatory Interactions Based on Multimodal Cues.” In <i>Proceedings of the 26th ACM International Conference on Multimodal Interaction</i>, 449–58. San José, Costa Rica, 2024. <a href="https://doi.org/10.1145/3678957.3685741">https://doi.org/10.1145/3678957.3685741</a>. Türk, Olcay, et al. “Predictability of Understanding in Explanatory Interactions Based on Multimodal Cues.” <i>Proceedings of the 26th ACM International Conference on Multimodal Interaction</i>, 2024, pp. 449–58, doi:<a href="https://doi.org/10.1145/3678957.3685741">10.1145/3678957.3685741</a>. @inproceedings{Türk_Lazarov_Wang_Buschmeier_Grimminger_Wagner_2024, place={San José, Costa Rica}, title={Predictability of understanding in explanatory interactions based on multimodal cues}, DOI={<a href="https://doi.org/10.1145/3678957.3685741">10.1145/3678957.3685741</a>}, booktitle={Proceedings of the 26th ACM International Conference on Multimodal Interaction}, author={Türk, Olcay and Lazarov, Stefan Teodorov and Wang, Yu and Buschmeier, Hendrik and Grimminger, Angela and Wagner, Petra}, year={2024}, pages={449–458} } Türk O, Lazarov ST, Wang Y, Buschmeier H, Grimminger A, Wagner P. Predictability of understanding in explanatory interactions based on multimodal cues. In: <i>Proceedings of the 26th ACM International Conference on Multimodal Interaction</i>. ; 2024:449-458. doi:<a href="https://doi.org/10.1145/3678957.3685741">10.1145/3678957.3685741</a> 559122024-08-30T07:42:55Z2025-09-02T14:46:03Z