Predictability of understanding in explanatory interactions based on multimodal cues

O. Türk, S. Lazarov, Y. Wang, H. Buschmeier, A. Grimminger, P. Wagner, in: Proceedings of the 26th ACM International Conference on Multimodal Interaction, San José, Costa Rica, n.d.

Download
No fulltext has been uploaded.
Conference Paper | Accepted | English
Author
Abstract
In explanatory interactions, explainees are expected to continuously provide feedback to explainers by signaling whether they understand an ongoing explanation. The study presented in this paper is based on the hypothesis that explainees use a set of multimodal cues, including vocalizations, facial expressions, and movements of the torso, head, and hands, to do so. We test this hypothesis by building a random forest classifier based on a multimodal corpus of dyadic explanations (21 explainers and explainees), in which windows of understanding or non-understanding were identified by participants in a retrospective video recall task. Results show that sequences of understanding can indeed be differentiated from those of non-understanding, and that a diverse set of predictors covering a wide range of modalities contributes to this classification. Due to data sparsity and a high degree of individual variation, the generalizability of our results is currently limited, but they support our hypothesis of the relevance of multimodal display in explanatory interactions.
Publishing Year
Proceedings Title
Proceedings of the 26th ACM International Conference on Multimodal Interaction
Conference
The 26th ACM International Conference on Multimodal Interaction
Conference Location
San José, Costa Rica
LibreCat-ID

Cite this

Türk O, Lazarov S, Wang Y, Buschmeier H, Grimminger A, Wagner P. Predictability of understanding in explanatory interactions based on multimodal cues. In: Proceedings of the 26th ACM International Conference on Multimodal Interaction. doi:10.1145/3678957.3685741
Türk, O., Lazarov, S., Wang, Y., Buschmeier, H., Grimminger, A., & Wagner, P. (n.d.). Predictability of understanding in explanatory interactions based on multimodal cues. Proceedings of the 26th ACM International Conference on Multimodal Interaction. The 26th ACM International Conference on Multimodal Interaction, San José, Costa Rica. https://doi.org/10.1145/3678957.3685741
@inproceedings{Türk_Lazarov_Wang_Buschmeier_Grimminger_Wagner, place={San José, Costa Rica}, title={Predictability of understanding in explanatory interactions based on multimodal cues}, DOI={10.1145/3678957.3685741}, booktitle={Proceedings of the 26th ACM International Conference on Multimodal Interaction}, author={Türk, Olcay and Lazarov, Stefan and Wang, Yu and Buschmeier, Hendrik and Grimminger, Angela and Wagner, Petra} }
Türk, Olcay, Stefan Lazarov, Yu Wang, Hendrik Buschmeier, Angela Grimminger, and Petra Wagner. “Predictability of Understanding in Explanatory Interactions Based on Multimodal Cues.” In Proceedings of the 26th ACM International Conference on Multimodal Interaction. San José, Costa Rica, n.d. https://doi.org/10.1145/3678957.3685741.
O. Türk, S. Lazarov, Y. Wang, H. Buschmeier, A. Grimminger, and P. Wagner, “Predictability of understanding in explanatory interactions based on multimodal cues,” presented at the The 26th ACM International Conference on Multimodal Interaction, San José, Costa Rica, doi: 10.1145/3678957.3685741.
Türk, Olcay, et al. “Predictability of Understanding in Explanatory Interactions Based on Multimodal Cues.” Proceedings of the 26th ACM International Conference on Multimodal Interaction, doi:10.1145/3678957.3685741.

Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar