{"status":"public","author":[{"first_name":"Milena","last_name":"Belosevic","full_name":"Belosevic, Milena"},{"first_name":"Hendrik","id":"76456","orcid":"0000-0002-9613-5713","full_name":"Buschmeier, Hendrik","last_name":"Buschmeier"}],"doi":"10.1145/3686215.3689203","publication_status":"published","place":"San José, Costa Rica","quality_controlled":"1","language":[{"iso":"eng"}],"abstract":[{"lang":"eng","text":"Starting from the assumption that LLMs are systems bearing only formal but not functional linguistic competence, this short paper explores how the understanding capabilities of LLMs could be implicitly explained based on a “pause and refect” strategy. Specifically, we propose to include a virtual embodied agent in human interactions with LLM-based chatbots. The agent will use air quotes as multimodal metalinguistic markers to explicitly point to those parts of the LLM’s output that are relevant to explaining the LLM’s meaning understanding capabilities. At the same time, by scafolding users to perceive the output as ‘mentioned language’ inferred from a metalinguistic function of multimodal markers, the agent implicitly explains how the meaning of the output should be understood. In this proposal, users will actively participate in the co-construction of the implicit explanation by providing feedback and deciding when and to what extent the agent’s scafold (e.g., the air quotes) is used."}],"extern":"1","date_created":"2025-09-11T07:32:23Z","date_updated":"2025-09-12T06:25:45Z","conference":{"name":"Multimodal Co-Construction of Explanations with XAI Workshop at ICMI 2024","end_date":"2024-11-04","location":"San José, Costa Rica","start_date":"2024-11-04"},"type":"conference","_id":"61180","title":"Quote to explain: Using multimodal metalinguistic markers to explain large language models’ understanding capabilities","citation":{"ieee":"M. Belosevic and H. Buschmeier, “Quote to explain: Using multimodal metalinguistic markers to explain large language models’ understanding capabilities,” in <i>ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction</i>, San José, Costa Rica, 2024, pp. 225–227, doi: <a href=\"https://doi.org/10.1145/3686215.3689203\">10.1145/3686215.3689203</a>.","mla":"Belosevic, Milena, and Hendrik Buschmeier. “Quote to Explain: Using Multimodal Metalinguistic Markers to Explain Large Language Models’ Understanding Capabilities.” <i>ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction</i>, ACM, 2024, pp. 225–227, doi:<a href=\"https://doi.org/10.1145/3686215.3689203\">10.1145/3686215.3689203</a>.","bibtex":"@inproceedings{Belosevic_Buschmeier_2024, place={San José, Costa Rica}, title={Quote to explain: Using multimodal metalinguistic markers to explain large language models’ understanding capabilities}, DOI={<a href=\"https://doi.org/10.1145/3686215.3689203\">10.1145/3686215.3689203</a>}, booktitle={ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction}, publisher={ACM}, author={Belosevic, Milena and Buschmeier, Hendrik}, year={2024}, pages={225–227} }","short":"M. Belosevic, H. Buschmeier, in: ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction, ACM, San José, Costa Rica, 2024, pp. 225–227.","apa":"Belosevic, M., &#38; Buschmeier, H. (2024). Quote to explain: Using multimodal metalinguistic markers to explain large language models’ understanding capabilities. <i>ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction</i>, 225–227. <a href=\"https://doi.org/10.1145/3686215.3689203\">https://doi.org/10.1145/3686215.3689203</a>","chicago":"Belosevic, Milena, and Hendrik Buschmeier. “Quote to Explain: Using Multimodal Metalinguistic Markers to Explain Large Language Models’ Understanding Capabilities.” In <i>ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction</i>, 225–227. San José, Costa Rica: ACM, 2024. <a href=\"https://doi.org/10.1145/3686215.3689203\">https://doi.org/10.1145/3686215.3689203</a>.","ama":"Belosevic M, Buschmeier H. Quote to explain: Using multimodal metalinguistic markers to explain large language models’ understanding capabilities. In: <i>ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction</i>. ACM; 2024:225–227. doi:<a href=\"https://doi.org/10.1145/3686215.3689203\">10.1145/3686215.3689203</a>"},"year":"2024","page":"225–227","publisher":"ACM","publication":"ICMI Companion ’24: Companion Proceedings of the 26th International Conference on Multimodal Interaction","user_id":"76456"}