Towards a Computational Model of Child Gesture-Speech Production

O. Abramov, S. Kopp, A. Nemeth, F. Kern, U. Mertens, K. Rohlfing, in: KOGWIS2018: Computational Approaches to Cognitive Science, 2018.

Download
No fulltext has been uploaded.
Conference Paper | English
Author
; ; ; ; ;
Abstract
Previous work by [1] studied gesture-speech interaction in adults. [1] focussed on temporal and semantic coordination of gesture and speech and found that while adult speech is mostly coordinated (or redundant) with gestures, semantic coordination increases the temporal synchrony. These observations do not necessarily hold for children (in particular with respect to iconic gestures, see [2]), where the speech and gesture systems are still under development. We studied the semantic and temporal coordination of speech and gesture in 4-year old children using a corpus of 40 children producing action descriptions in task oriented dialogues. In particular, we examined what kinds of information are transmitted verbally vs. non-verbally and how they are related. To account for this, we extended the semantic features (SFs) developed in [3] for object descriptions in order to include the semantics of actions. We coded the SFs on the children’s speech and gestures separately using video data. In our presentation, we will focus on the quantitative distribution of SFs across gesture and speech. Our results indicate that speech and gestures of 4-year olds are less integrated than those of the adults, although there is a large variability among the children. We will discuss the results with respect to the cognitive processes (e.g., visual memory, language) underlying children’s abilities at this stage of development. Our work paves the way for the cognitive architecture of speech-gesture interaction in preschoolers which to our knowledge is missing so far.
Publishing Year
Proceedings Title
KOGWIS2018: Computational Approaches to Cognitive Science
LibreCat-ID

Cite this

Abramov O, Kopp S, Nemeth A, Kern F, Mertens U, Rohlfing K. Towards a Computational Model of Child Gesture-Speech Production. In: KOGWIS2018: Computational Approaches to Cognitive Science. ; 2018.
Abramov, O., Kopp, S., Nemeth, A., Kern, F., Mertens, U., & Rohlfing, K. (2018). Towards a Computational Model of Child Gesture-Speech Production. In KOGWIS2018: Computational Approaches to Cognitive Science.
@inproceedings{Abramov_Kopp_Nemeth_Kern_Mertens_Rohlfing_2018, title={Towards a Computational Model of Child Gesture-Speech Production}, booktitle={KOGWIS2018: Computational Approaches to Cognitive Science}, author={Abramov, Olga and Kopp, Stefan and Nemeth, Anne and Kern, Friederike and Mertens, Ulrich and Rohlfing, Katharina}, year={2018} }
Abramov, Olga, Stefan Kopp, Anne Nemeth, Friederike Kern, Ulrich Mertens, and Katharina Rohlfing. “Towards a Computational Model of Child Gesture-Speech Production.” In KOGWIS2018: Computational Approaches to Cognitive Science, 2018.
O. Abramov, S. Kopp, A. Nemeth, F. Kern, U. Mertens, and K. Rohlfing, “Towards a Computational Model of Child Gesture-Speech Production,” in KOGWIS2018: Computational Approaches to Cognitive Science, 2018.
Abramov, Olga, et al. “Towards a Computational Model of Child Gesture-Speech Production.” KOGWIS2018: Computational Approaches to Cognitive Science, 2018.

Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar