---
_id: '64840'
abstract:
- lang: eng
  text: "Multilingual picture books are considered to have great potential for language
    reflection and language learning, partly due to their multimodality, in which
    writing(s), language(s) and image(s) interact in the construction of\r\nmeaning.
    What possibilities multilingual picture books offer for grammatical learning is
    still a research desidera-\r\ntum. In a multi-perspective qualitative analysis
    that brings together categories from picture book and typography\r\nresearch as
    well as graphemics and grammar, three variants of multilingual picture books are
    analysed. The results show that all of them – to varying degrees – open up possibilities
    for the further development of multiliteracies and offer potential that could
    be used in classroom practice for both implicit and explicit grammatical learning.\r\n"
- lang: ger
  text: "Mehrsprachigen Bilderbüchern wird ein hohes Potential für sprachreflexives
    und sprachliches Lernen zugespro-\r\nchen, auch aufgrund ihrer Multimodalität,
    bei der Schrift(en), Sprache(n) und Bild(er) bei der Bedeutungskonstruk-\r\ntion
    zusammenspielen. Welche Möglichkeiten Bilderbücher, bei denen Mehrsprachigkeit
    auf Multimodalität trifft, für grammatisches Lernen bieten, stellt noch immer
    ein Forschungsdesiderat dar. In einer mehrperspektivisch ausgerichteten qualitativen
    Analyse, die Kategorien aus der Bilderbuch- und Typographieforschung sowie Graphe-\r\nmatik
    und Grammatik zusammenführt, werden drei Varianten mehrsprachiger Bilderbücher
    (additiv, parallel und\r\nintegrativ mehrsprachige) untersucht. Die Ergebnisse
    zeigen, dass alle drei – zum Teil in unterschiedlichem Maße – Möglichkeiten für
    die Weiterentwicklung von Multiliteracies (Multiliteralität und Multiliterarität)
    eröffnen und Anregungspotentiale bieten, die in der Unterrichtspraxis – sowohl
    für implizites als auch explizites – multimodales grammatisches Lernen genutzt
    werden könnten.\r\n"
author:
- first_name: Elvira
  full_name: Topalovic, Elvira
  id: '23088'
  last_name: Topalovic
  orcid: 0009-0002-7576-3558
- first_name: Kira
  full_name: Härtel, Kira
  id: '23198'
  last_name: Härtel
citation:
  ama: 'Topalovic E, Härtel K. Grammatik und Multimodalität im mehrsprachigen Bilderbuch.
    Qualitative Analyse an der Schnittstelle von Schrift, Sprache und Bild. In: Geyer
    S, Cristante V, eds. <i>Grammatikdidaktik und Mehrsprachigkeit: Theoretische und
    empirische Perspektiven</i>. ; 2026:190-207. doi:<a href="https://doi.org/10.46586/SLLD.462">https://doi.org/10.46586/SLLD.462</a>'
  apa: 'Topalovic, E., &#38; Härtel, K. (2026). Grammatik und Multimodalität im mehrsprachigen
    Bilderbuch. Qualitative Analyse an der Schnittstelle von Schrift, Sprache und
    Bild. In S. Geyer &#38; V. Cristante (Eds.), <i>Grammatikdidaktik und Mehrsprachigkeit:
    Theoretische und empirische Perspektiven</i> (pp. 190–207). <a href="https://doi.org/10.46586/SLLD.462">https://doi.org/10.46586/SLLD.462</a>'
  bibtex: '@inbook{Topalovic_Härtel_2026, title={Grammatik und Multimodalität im mehrsprachigen
    Bilderbuch. Qualitative Analyse an der Schnittstelle von Schrift, Sprache und
    Bild}, DOI={<a href="https://doi.org/10.46586/SLLD.462">https://doi.org/10.46586/SLLD.462</a>},
    booktitle={Grammatikdidaktik und Mehrsprachigkeit: Theoretische und empirische
    Perspektiven}, author={Topalovic, Elvira and Härtel, Kira}, editor={Geyer, Sabrina
    and Cristante, Valentina}, year={2026}, pages={190–207} }'
  chicago: 'Topalovic, Elvira, and Kira Härtel. “Grammatik und Multimodalität im mehrsprachigen
    Bilderbuch. Qualitative Analyse an der Schnittstelle von Schrift, Sprache und
    Bild.” In <i>Grammatikdidaktik und Mehrsprachigkeit: Theoretische und empirische
    Perspektiven</i>, edited by Sabrina Geyer and Valentina Cristante, 190–207, 2026.
    <a href="https://doi.org/10.46586/SLLD.462">https://doi.org/10.46586/SLLD.462</a>.'
  ieee: 'E. Topalovic and K. Härtel, “Grammatik und Multimodalität im mehrsprachigen
    Bilderbuch. Qualitative Analyse an der Schnittstelle von Schrift, Sprache und
    Bild,” in <i>Grammatikdidaktik und Mehrsprachigkeit: Theoretische und empirische
    Perspektiven</i>, S. Geyer and V. Cristante, Eds. 2026, pp. 190–207.'
  mla: 'Topalovic, Elvira, and Kira Härtel. “Grammatik und Multimodalität im mehrsprachigen
    Bilderbuch. Qualitative Analyse an der Schnittstelle von Schrift, Sprache und
    Bild.” <i>Grammatikdidaktik und Mehrsprachigkeit: Theoretische und empirische
    Perspektiven</i>, edited by Sabrina Geyer and Valentina Cristante, 2026, pp. 190–207,
    doi:<a href="https://doi.org/10.46586/SLLD.462">https://doi.org/10.46586/SLLD.462</a>.'
  short: 'E. Topalovic, K. Härtel, in: S. Geyer, V. Cristante (Eds.), Grammatikdidaktik
    und Mehrsprachigkeit: Theoretische und empirische Perspektiven, 2026, pp. 190–207.'
date_created: 2026-03-05T09:13:53Z
date_updated: 2026-03-05T09:29:27Z
doi: https://doi.org/10.46586/SLLD.462
editor:
- first_name: Sabrina
  full_name: Geyer, Sabrina
  last_name: Geyer
- first_name: Valentina
  full_name: Cristante, Valentina
  last_name: Cristante
keyword:
- grammar
- multimodality
- multiliteracies
- picture book
- typography
- Grammatik
- Multimodalität
- Multiliteracies
- Bilderbuch
- Typographie
language:
- iso: ger
main_file_link:
- open_access: '1'
  url: https://omp.ub.rub.de/index.php/SLLD/catalog/view/462/379/2391
oa: '1'
page: 190 - 207
publication: 'Grammatikdidaktik und Mehrsprachigkeit: Theoretische und empirische
  Perspektiven'
publication_status: published
quality_controlled: '1'
status: public
title: Grammatik und Multimodalität im mehrsprachigen Bilderbuch. Qualitative Analyse
  an der Schnittstelle von Schrift, Sprache und Bild
type: book_chapter
user_id: '23088'
year: '2026'
...
---
_id: '43437'
abstract:
- lang: eng
  text: '<jats:p>In virtual reality (VR), participants may not always have hands,
    bodies, eyes, or even voices—using VR helmets and two controllers, participants
    control an avatar through virtual worlds that do not necessarily obey familiar
    laws of physics; moreover, the avatar’s bodily characteristics may not neatly
    match our bodies in the physical world. Despite these limitations and specificities,
    humans get things done through collaboration and the creative use of the environment.
    While multiuser interactive VR is attracting greater numbers of participants,
    there are currently few attempts to analyze the in situ interaction systematically.
    This paper proposes a video-analytic detail-oriented methodological framework
    for studying virtual reality interaction. Using multimodal conversation analysis,
    the paper investigates a nonverbal, embodied, two-person interaction: two players
    in a survival game strive to gesturally resolve a misunderstanding regarding an
    in-game mechanic—however, both of their microphones are turned off for the duration
    of play. The players’ inability to resort to complex language to resolve this
    issue results in a dense sequence of back-and-forth activity involving gestures,
    object manipulation, gaze, and body work. Most crucially, timing and modified
    repetitions of previously produced actions turn out to be the key to overcome
    both technical and communicative challenges. The paper analyzes these action sequences,
    demonstrates how they generate intended outcomes, and proposes a vocabulary to
    speak about these types of interaction more generally. The findings demonstrate
    the viability of multimodal analysis of VR interaction, shed light on unique challenges
    of analyzing interaction in virtual reality, and generate broader methodological
    insights about the study of nonverbal action.</jats:p>'
article_type: original
author:
- first_name: Nils
  full_name: Klowait, Nils
  id: '98454'
  last_name: Klowait
  orcid: 0000-0002-7347-099X
citation:
  ama: Klowait N. On the Multimodal Resolution of a Search Sequence in Virtual Reality.
    <i>Human Behavior and Emerging Technologies</i>. 2023;2023:1-15. doi:<a href="https://doi.org/10.1155/2023/8417012">10.1155/2023/8417012</a>
  apa: Klowait, N. (2023). On the Multimodal Resolution of a Search Sequence in Virtual
    Reality. <i>Human Behavior and Emerging Technologies</i>, <i>2023</i>, 1–15. <a
    href="https://doi.org/10.1155/2023/8417012">https://doi.org/10.1155/2023/8417012</a>
  bibtex: '@article{Klowait_2023, title={On the Multimodal Resolution of a Search
    Sequence in Virtual Reality}, volume={2023}, DOI={<a href="https://doi.org/10.1155/2023/8417012">10.1155/2023/8417012</a>},
    journal={Human Behavior and Emerging Technologies}, publisher={Hindawi Limited},
    author={Klowait, Nils}, year={2023}, pages={1–15} }'
  chicago: 'Klowait, Nils. “On the Multimodal Resolution of a Search Sequence in Virtual
    Reality.” <i>Human Behavior and Emerging Technologies</i> 2023 (2023): 1–15. <a
    href="https://doi.org/10.1155/2023/8417012">https://doi.org/10.1155/2023/8417012</a>.'
  ieee: 'N. Klowait, “On the Multimodal Resolution of a Search Sequence in Virtual
    Reality,” <i>Human Behavior and Emerging Technologies</i>, vol. 2023, pp. 1–15,
    2023, doi: <a href="https://doi.org/10.1155/2023/8417012">10.1155/2023/8417012</a>.'
  mla: Klowait, Nils. “On the Multimodal Resolution of a Search Sequence in Virtual
    Reality.” <i>Human Behavior and Emerging Technologies</i>, vol. 2023, Hindawi
    Limited, 2023, pp. 1–15, doi:<a href="https://doi.org/10.1155/2023/8417012">10.1155/2023/8417012</a>.
  short: N. Klowait, Human Behavior and Emerging Technologies 2023 (2023) 1–15.
date_created: 2023-04-06T10:57:28Z
date_updated: 2024-03-26T09:40:53Z
ddc:
- '300'
department:
- _id: '9'
doi: 10.1155/2023/8417012
file:
- access_level: closed
  content_type: application/pdf
  creator: nklowait
  date_created: 2023-04-06T11:00:01Z
  date_updated: 2023-04-06T11:00:01Z
  file_id: '43438'
  file_name: Klowait_2023a.pdf
  file_size: 2877385
  relation: main_file
  success: 1
file_date_updated: 2023-04-06T11:00:01Z
funded_apc: '1'
has_accepted_license: '1'
intvolume: '      2023'
keyword:
- Human-Computer Interaction
- General Social Sciences
- Social Psychology
- 'Virtual Reality : Multimodality'
- Nonverbal Interaction
- Search Sequence
- Gesture
- Co-Operative Action
- Goodwin
- Ethnomethodology
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://doi.org/10.1155/2023/8417012
oa: '1'
page: 1-15
project:
- _id: '119'
  name: 'TRR 318 - Ö: TRR 318 - Project Area Ö'
publication: Human Behavior and Emerging Technologies
publication_identifier:
  issn:
  - 2578-1863
publication_status: published
publisher: Hindawi Limited
quality_controlled: '1'
status: public
title: On the Multimodal Resolution of a Search Sequence in Virtual Reality
type: journal_article
user_id: '98454'
volume: 2023
year: '2023'
...
---
_id: '48882'
abstract:
- lang: eng
  text: In multimodal multi-objective optimization (MMMOO), the focus is not solely
    on convergence in objective space, but rather also on explicitly ensuring diversity
    in decision space. We illustrate why commonly used diversity measures are not
    entirely appropriate for this task and propose a sophisticated basin-based evaluation
    (BBE) method. Also, BBE variants are developed, capturing the anytime behavior
    of algorithms. The set of BBE measures is tested by means of an algorithm configuration
    study. We show that these new measures also transfer properties of the well-established
    hypervolume (HV) indicator to the domain of MMMOO, thus also accounting for objective
    space convergence. Moreover, we advance MMMOO research by providing insights into
    the multimodal performance of the considered algorithms. Specifically, algorithms
    exploiting local structures are shown to outperform classical evolutionary multi-objective
    optimizers regarding the BBE variants and respective trade-off with HV.
author:
- first_name: Jonathan
  full_name: Heins, Jonathan
  last_name: Heins
- first_name: Jeroen
  full_name: Rook, Jeroen
  last_name: Rook
- first_name: Lennart
  full_name: Schäpermeier, Lennart
  last_name: Schäpermeier
- first_name: Pascal
  full_name: Kerschke, Pascal
  last_name: Kerschke
- first_name: Jakob
  full_name: Bossek, Jakob
  id: '102979'
  last_name: Bossek
  orcid: 0000-0002-4121-4668
- first_name: Heike
  full_name: Trautmann, Heike
  last_name: Trautmann
citation:
  ama: 'Heins J, Rook J, Schäpermeier L, Kerschke P, Bossek J, Trautmann H. BBE: Basin-Based
    Evaluation of Multimodal Multi-objective Optimization Problems. In: Rudolph G,
    Kononova AV, Aguirre H, Kerschke P, Ochoa G, Tusar T, eds. <i>Parallel Problem
    Solving from Nature (PPSN XVII)</i>. Lecture Notes in Computer Science. Springer
    International Publishing; 2022:192–206. doi:<a href="https://doi.org/10.1007/978-3-031-14714-2_14">10.1007/978-3-031-14714-2_14</a>'
  apa: 'Heins, J., Rook, J., Schäpermeier, L., Kerschke, P., Bossek, J., &#38; Trautmann,
    H. (2022). BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization
    Problems. In G. Rudolph, A. V. Kononova, H. Aguirre, P. Kerschke, G. Ochoa, &#38;
    T. Tusar (Eds.), <i>Parallel Problem Solving from Nature (PPSN XVII)</i> (pp.
    192–206). Springer International Publishing. <a href="https://doi.org/10.1007/978-3-031-14714-2_14">https://doi.org/10.1007/978-3-031-14714-2_14</a>'
  bibtex: '@inproceedings{Heins_Rook_Schäpermeier_Kerschke_Bossek_Trautmann_2022,
    place={Cham}, series={Lecture Notes in Computer Science}, title={BBE: Basin-Based
    Evaluation of Multimodal Multi-objective Optimization Problems}, DOI={<a href="https://doi.org/10.1007/978-3-031-14714-2_14">10.1007/978-3-031-14714-2_14</a>},
    booktitle={Parallel Problem Solving from Nature (PPSN XVII)}, publisher={Springer
    International Publishing}, author={Heins, Jonathan and Rook, Jeroen and Schäpermeier,
    Lennart and Kerschke, Pascal and Bossek, Jakob and Trautmann, Heike}, editor={Rudolph,
    Günter and Kononova, Anna V. and Aguirre, Hernán and Kerschke, Pascal and Ochoa,
    Gabriela and Tusar, Tea}, year={2022}, pages={192–206}, collection={Lecture Notes
    in Computer Science} }'
  chicago: 'Heins, Jonathan, Jeroen Rook, Lennart Schäpermeier, Pascal Kerschke, Jakob
    Bossek, and Heike Trautmann. “BBE: Basin-Based Evaluation of Multimodal Multi-Objective
    Optimization Problems.” In <i>Parallel Problem Solving from Nature (PPSN XVII)</i>,
    edited by Günter Rudolph, Anna V. Kononova, Hernán Aguirre, Pascal Kerschke, Gabriela
    Ochoa, and Tea Tusar, 192–206. Lecture Notes in Computer Science. Cham: Springer
    International Publishing, 2022. <a href="https://doi.org/10.1007/978-3-031-14714-2_14">https://doi.org/10.1007/978-3-031-14714-2_14</a>.'
  ieee: 'J. Heins, J. Rook, L. Schäpermeier, P. Kerschke, J. Bossek, and H. Trautmann,
    “BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems,”
    in <i>Parallel Problem Solving from Nature (PPSN XVII)</i>, 2022, pp. 192–206,
    doi: <a href="https://doi.org/10.1007/978-3-031-14714-2_14">10.1007/978-3-031-14714-2_14</a>.'
  mla: 'Heins, Jonathan, et al. “BBE: Basin-Based Evaluation of Multimodal Multi-Objective
    Optimization Problems.” <i>Parallel Problem Solving from Nature (PPSN XVII)</i>,
    edited by Günter Rudolph et al., Springer International Publishing, 2022, pp.
    192–206, doi:<a href="https://doi.org/10.1007/978-3-031-14714-2_14">10.1007/978-3-031-14714-2_14</a>.'
  short: 'J. Heins, J. Rook, L. Schäpermeier, P. Kerschke, J. Bossek, H. Trautmann,
    in: G. Rudolph, A.V. Kononova, H. Aguirre, P. Kerschke, G. Ochoa, T. Tusar (Eds.),
    Parallel Problem Solving from Nature (PPSN XVII), Springer International Publishing,
    Cham, 2022, pp. 192–206.'
date_created: 2023-11-14T15:58:58Z
date_updated: 2023-12-13T10:47:50Z
department:
- _id: '819'
doi: 10.1007/978-3-031-14714-2_14
editor:
- first_name: Günter
  full_name: Rudolph, Günter
  last_name: Rudolph
- first_name: Anna V.
  full_name: Kononova, Anna V.
  last_name: Kononova
- first_name: Hernán
  full_name: Aguirre, Hernán
  last_name: Aguirre
- first_name: Pascal
  full_name: Kerschke, Pascal
  last_name: Kerschke
- first_name: Gabriela
  full_name: Ochoa, Gabriela
  last_name: Ochoa
- first_name: Tea
  full_name: Tusar, Tea
  last_name: Tusar
extern: '1'
keyword:
- Anytime behavior
- Benchmarking
- Continuous optimization
- Multi-objective optimization
- Multimodality
- Performance metric
language:
- iso: eng
page: 192–206
place: Cham
publication: Parallel Problem Solving from Nature (PPSN XVII)
publication_identifier:
  isbn:
  - 978-3-031-14714-2
publisher: Springer International Publishing
series_title: Lecture Notes in Computer Science
status: public
title: 'BBE: Basin-Based Evaluation of Multimodal Multi-objective Optimization Problems'
type: conference
user_id: '102979'
year: '2022'
...
---
_id: '34564'
abstract:
- lang: eng
  text: To provide user interfaces for a rich set of devices and interaction modalities,
    we follow a model-based development methodology. We devised an architecture which
    deploys user interfaces specified as dialogue models with abstract interaction
    objects and allows context-based adaptations by means of an external transcoding
    process. For the validation of the applicability of this methodology for developing
    usable multimodal multi-device systems, we present two case studies based on proof-of-concept
    implementations and assessed them with a large set of established design principles
    and different types of modality cooperation.
author:
- first_name: Robbie
  full_name: Schäfer, Robbie
  last_name: Schäfer
- first_name: Wolfgang
  full_name: Müller, Wolfgang
  id: '16243'
  last_name: Müller
citation:
  ama: Schäfer R, Müller W. Assessment of a Multimodal Interaction and Rendering System
    against Established Design Principles. <i>Journal on Multimodal User Interfaces</i>.
    2008;2(1):25-41. doi:<a href="https://doi.org/10.1007/s12193-008-0003-3">10.1007/s12193-008-0003-3</a>
  apa: Schäfer, R., &#38; Müller, W. (2008). Assessment of a Multimodal Interaction
    and Rendering System against Established Design Principles. <i>Journal on Multimodal
    User Interfaces</i>, <i>2</i>(1), 25–41. <a href="https://doi.org/10.1007/s12193-008-0003-3">https://doi.org/10.1007/s12193-008-0003-3</a>
  bibtex: '@article{Schäfer_Müller_2008, title={Assessment of a Multimodal Interaction
    and Rendering System against Established Design Principles}, volume={2}, DOI={<a
    href="https://doi.org/10.1007/s12193-008-0003-3">10.1007/s12193-008-0003-3</a>},
    number={1}, journal={Journal on Multimodal User Interfaces}, publisher={Springer-Verlag},
    author={Schäfer, Robbie and Müller, Wolfgang}, year={2008}, pages={25–41} }'
  chicago: 'Schäfer, Robbie, and Wolfgang Müller. “Assessment of a Multimodal Interaction
    and Rendering System against Established Design Principles.” <i>Journal on Multimodal
    User Interfaces</i> 2, no. 1 (2008): 25–41. <a href="https://doi.org/10.1007/s12193-008-0003-3">https://doi.org/10.1007/s12193-008-0003-3</a>.'
  ieee: 'R. Schäfer and W. Müller, “Assessment of a Multimodal Interaction and Rendering
    System against Established Design Principles,” <i>Journal on Multimodal User Interfaces</i>,
    vol. 2, no. 1, pp. 25–41, 2008, doi: <a href="https://doi.org/10.1007/s12193-008-0003-3">10.1007/s12193-008-0003-3</a>.'
  mla: Schäfer, Robbie, and Wolfgang Müller. “Assessment of a Multimodal Interaction
    and Rendering System against Established Design Principles.” <i>Journal on Multimodal
    User Interfaces</i>, vol. 2, no. 1, Springer-Verlag, 2008, pp. 25–41, doi:<a href="https://doi.org/10.1007/s12193-008-0003-3">10.1007/s12193-008-0003-3</a>.
  short: R. Schäfer, W. Müller, Journal on Multimodal User Interfaces 2 (2008) 25–41.
date_created: 2022-12-19T12:22:50Z
date_updated: 2022-12-19T12:23:05Z
department:
- _id: '672'
doi: 10.1007/s12193-008-0003-3
intvolume: '         2'
issue: '1'
keyword:
- Interaction architecture     Abstract interaction objects     Dialogue model     Transformations     Multimodality     Multi-device     Design
  principles
language:
- iso: eng
page: 25-41
publication: Journal on Multimodal User Interfaces
publisher: Springer-Verlag
status: public
title: Assessment of a Multimodal Interaction and Rendering System against Established
  Design Principles
type: journal_article
user_id: '5786'
volume: 2
year: '2008'
...