Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles - Muschalik, Maximilian; Fumagalli, Fabian; Hammer, Barbara; Huellermeier, Eyke (Proceedings of the AAAI Conference on Artificial Intelligence)
While shallow decision trees may be interpretable, larger ensemble models like gradient-boosted trees, which often set the state of the art in machine learning problems involving tabular data, still remain black box models. As a remedy, the Shapley value (SV) is a well-known concept in explainable artificial intelligence (XAI) research for quantifying additive feature attributions of predictions. The model-specific TreeSHAP methodology solves the exponential complexity for retrieving exact SVs from tree-based models. Expanding beyond individual feature attribution, Shapley interactions reveal the impact of intricate feature interactions of any order. In this work, we present TreeSHAP-IQ, an efficient method to compute any-order additive Shapley interactions for predictions of tree-based models. TreeSHAP-IQ is supported by a mathematical framework that exploits polynomial arithmetic to compute the interaction scores in a single recursive traversal of the tree, akin to Linear TreeSHAP. We apply TreeSHAP-IQ on state-of-the-art tree ensembles and explore interactions on well-established benchmark datasets.
Keywords: Explainable Artificial Intelligence

Effects of task difficulty on visual processing speed - Banh, Ngoc Chi; Scharlau, Ingrid

Contrastiveness in the context of action demonstration: an eye-tracking study on its effects on action perception and action recall - Singh, Amit; Rohlfing, Katharina J. (Proceedings of the Annual Meeting of the Cognitive Science Society 45 (45))
The study investigates two different ways of guiding the addressee of an explanation - an explainee, through action demonstration: contrastive and non-contrastive. Their effect was tested on attention to specific action elements (goal) as well as on event memory. In an eye-tracking experiment, participants were shown different motion videos that were either contrastive or non-contrastive with respect to the segments of movement presentation. Given that everyday action demonstration is often multimodal, the stimuli were created with re- spect to their visual and verbal presentation. For visual presentation, a video combined two movements in a contrastive (e.g., Up-motion following a Down-motion) or non-contrastive way (e.g., two Up-motions following each other). For verbal presentation, each video was combined with a sequence of instruction descriptions in the form of negative (i.e., contrastive) or assertive (i.e., non-contrastive) guidance. It was found that a) attention to the event goal increased for this condition in the later time window, and b) participants’ recall of the event was facilitated when a visually contrastive motion was combined with a verbal contrast.
Keywords: Attention; negation; contrastive guidance; eye-movements; action understanding; event representation

On Feature Importance and Interpretability of Speaker Representations - Rautenberg, Frederik; Kuhlmann, Michael; Wiechmann, Jana; Seebauer, Fritz; Wagner, Petra; Haeb-Umbach, Reinhold (ITG Conference on Speech Communication)
Unsupervised speech disentanglement aims at separating fast varying from slowly varying components of a speech signal. In this contribution, we take a closer look at the embedding vector representing the slowly varying signal components, commonly named the speaker embedding vector. We ask, which properties of a speaker's voice are captured and investigate to which extent do individual embedding vector components sign responsible for them, using the concept of Shapley values. Our findings show that certain speaker-specific acoustic-phonetic properties can be fairly well predicted from the speaker embedding, while the investigated more abstract voice quality features cannot.

Explaining voice characteristics to novice voice practitioners-How successful is it? - Wiechmann, Jana; Rautenberg, Frederik; Wagner, Petra; Haeb-Umbach, Reinhold (20th International Congress of the Phonetic Sciences (ICPhS))

The Importance of Distrust in AI - Peters, Tobias Martin; Visser, Roel W. (Communications in Computer and Information Science)

Incremental permutation feature importance (iPFI): towards online explanations on data streams - Fumagalli, Fabian; Muschalik, Maximilian; Hüllermeier, Eyke; Hammer, Barbara (Machine Learning)
Abstract Explainable artificial intelligence has mainly focused on static learning scenarios so far. We are interested in dynamic scenarios where data is sampled progressively, and learning is done in an incremental rather than a batch mode. We seek efficient incremental algorithms for computing feature importance (FI). Permutation feature importance (PFI) is a well-established model-agnostic measure to obtain global FI based on feature marginalization of absent features. We propose an efficient, model-agnostic algorithm called iPFI to estimate this measure incrementally and under dynamic modeling conditions including concept drift. We prove theoretical guarantees on the approximation quality in terms of expectation and variance. To validate our theoretical findings and the efficacy of our approaches in incremental scenarios dealing with streaming data rather than traditional batch settings, we conduct multiple experimental studies on benchmark data with and without concept drift.
Keywords: Artificial Intelligence; Software

“I do not know! but why?” — Local model-agnostic example-based explanations of reject - Artelt, André; Visser, Roel; Hammer, Barbara (Neurocomputing)
Keywords: Artificial Intelligence; Cognitive Neuroscience; Computer Science Applications

Adding Why to What? Analyses of an Everyday Explanation - Terfloth, Lutz; Schaffer, Michael; Buhl, Heike M.; Schulte, Carsten
In XAI it is important to consider that, in contrast to explanations for professional audiences, one cannot assume common expertise when explaining for laypeople. But such explanations between humans vary greatly, making it difficult to research commonalities across explanations. We used the dual nature theory, a techno-philosophical approach, to cope with these challenges. According to it, one can explain, for example, an XAI's decision by addressing its dual nature: by focusing on the Architecture (e.g., the logic of its algorithms) or the Relevance (e.g., the severity of a decision, the implications of a recommendation). We investigated 20 game explanations using the theory as an analytical framework. We elaborate how we used the theory to quickly structure and compare explanations of technological artifacts. We supplemented results from analyzing the explanation contents with results from a video recall to explore how explainers justified their explanation. We found that explainers were focusing on the physical aspects of the game first (Architecture) and only later on aspects of the Relevance. Reasoning in the video recalls indicated that EX regarded the focus on the Architecture as important for structuring the explanation initially by explaining the basic components before focusing on more complex, intangible aspects. Shifting between addressing the two sides was justified by explanation goals, emerging misunderstandings, and the knowledge needs of the explainee. We discovered several commonalities that inspire future research questions which, if further generalizable, provide first ideas for the construction of synthetic explanations.

What is AI Ethics? Ethics as means of self-regulation and the need for critical reflection - Alpsancar, Suzana (International Conference on Computer Ethics 2023)

A new algorithmic imaginary - Schulz, Christian (Media, Culture & Society)
The algorithmic imaginary as a theoretical concept has received increasing attention in recent years as it aims at users’ appropriation of algorithmic processes operating in opacity. But the concept originally only starts from the users’ point of view, while the processes on the platforms’ side are largely left out. In contrast, this paper argues that what is true for users is also valid for algorithmic processes and the designers behind. On the one hand, the algorithm imagines users’ future behavior via machine learning, which is supposed to predict all their future actions. On the other hand, the designers anticipate different actions that could potentially performed by users with every new implementation of features such as social media feeds. In order to bring into view this permanently reciprocal interplay coupled to the imaginary, in which not only the users are involved, I will argue for a more comprehensive and theoretically precise algorithmic imaginary referring to the theory of Cornelius Castoriadis. In such a perspective, an important contribution can be formulated for a theory of social media platforms that goes beyond praxeocentrism or structural determinism.
Keywords: Sociology and Political Science; Communication

Towards Interpretability in Audio and Visual Affective Machine Learning: A Review - Johnson, David; Hakobyan, Olya; Drimalla, Hanna
Machine learning is frequently used in affective computing, but presents challenges due the opacity of state-of-the-art machine learning methods. Because of the impact affective machine learning systems may have on an individual's life, it is important that models be made transparent to detect and mitigate biased decision making. In this regard, affective machine learning could benefit from the recent advancements in explainable artificial intelligence (XAI) research. We perform a structured literature review to examine the use of interpretability in the context of affective machine learning. We focus on studies using audio, visual, or audiovisual data for model training and identified 29 research articles. Our findings show an emergence of the use of interpretability methods in the last five years. However, their use is currently limited regarding the range of methods used, the depth of evaluations, and the consideration of use-cases. We outline the main gaps in the research and provide recommendations for researchers that aim to implement interpretable methods for affective machine learning.

EEG Correlates of Distractions and Hesitations in Human–Robot Interaction: A LabLinking Pilot Study - Richter, Birte; Putze, Felix; Ivucic, Gabriel; Brandt, Mara; Schütze, Christian; Reisenhofer, Rafael; Wrede, Britta; Schultz, Tanja (Multimodal Technologies and Interaction)
In this paper, we investigate the effect of distractions and hesitations as a scaffolding strategy. Recent research points to the potential beneficial effects of a speaker’s hesitations on the listeners’ comprehension of utterances, although results from studies on this issue indicate that humans do not make strategic use of them. The role of hesitations and their communicative function in human-human interaction is a much-discussed topic in current research. To better understand the underlying cognitive processes, we developed a human–robot interaction (HRI) setup that allows the measurement of the electroencephalogram (EEG) signals of a human participant while interacting with a robot. We thereby address the research question of whether we find effects on single-trial EEG based on the distraction and the corresponding robot’s hesitation scaffolding strategy. To carry out the experiments, we leverage our LabLinking method, which enables interdisciplinary joint research between remote labs. This study could not have been conducted without LabLinking, as the two involved labs needed to combine their individual expertise and equipment to achieve the goal together. The results of our study indicate that the EEG correlates in the distracted condition are different from the baseline condition without distractions. Furthermore, we could differentiate the EEG correlates of distraction with and without a hesitation scaffolding strategy. This proof-of-concept study shows that LabLinking makes it possible to conduct collaborative HRI studies in remote laboratories and lays the first foundation for more in-depth research into robotic scaffolding strategies.
Keywords: Computer Networks and Communications; Computer Science Applications; Human-Computer Interaction; Neuroscience (miscellaneous)

SNAPE: A Sequential Non-Stationary Decision Process Model for Adaptive Explanation Generation - Robrecht, Amelie; Kopp, Stefan (Proceedings of the 15th International Conference on Agents and Artificial Intelligence)

Technical Transparency for Robot Navigation Through AR Visualizations - Dyck, Leonie; Beierling, Helen; Helmert, Robin; Vollmer, Anna-Lisa (Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction)

Does Explainability Require Transparency? - Esposito, Elena (Sociologica)
Dealing with opaque algorithms, the frequent overlap between transparency and explainability produces seemingly unsolvable dilemmas, as the much-discussed trade-off between model performance and model transparency. Referring to Niklas Luhmann's notion of communication, the paper argues that explainability does not necessarily require transparency and proposes an alternative approach. Explanations as communicative processes do not imply any disclosure of thoughts or neural processes, but only reformulations that provide the partners with additional elements and enable them to understand (from their perspective) what has been done and why. Recent computational approaches aiming at post-hoc explainability reproduce what happens in communication, producing explanations of the working of algorithms that can be different from the processes of the algorithms.
Keywords: Explainable AI; Transparency; Explanation; Communication; Sociological systems theory

Explaining Machines: Social Management of Incomprehensible Algorithms. Introduction - Esposito, Elena (Sociologica)
This short introduction presents the symposium ‘Explaining Machines’. It locates the debate about Explainable AI in the history of the reflection about AI and outlines the issues discussed in the contributions.
Keywords: Explainable AI; Inexplicability; Transparency; Explanation; Opacity; Contestability

Speech Disentanglement for Analysis and Modification of Acoustic and Perceptual Speaker Characteristics - Rautenberg, Frederik; Kuhlmann, Michael; Ebbers, Janek; Wiechmann, Jana; Seebauer, Fritz; Wagner, Petra; Haeb-Umbach, Reinhold (Fortschritte der Akustik - DAGA 2023)

iSAGE: An Incremental Version of SAGE for Online Explanation on Data Streams - Muschalik, Maximilian; Fumagalli, Fabian; Hammer, Barbara; Huellermeier, Eyke (Machine Learning and Knowledge Discovery in Databases: Research Track)

iPDP: On Partial Dependence Plots in Dynamic Modeling Scenarios - Muschalik, Maximilian; Fumagalli, Fabian; Jagtani, Rohit; Hammer, Barbara; Huellermeier, Eyke (Communications in Computer and Information Science)

Incremental permutation feature importance (iPFI): towards online explanations on data streams - Fumagalli, Fabian; Muschalik, Maximilian; Hüllermeier, Eyke; Hammer, Barbara (Machine Learning)
Abstract Explainable artificial intelligence has mainly focused on static learning scenarios so far. We are interested in dynamic scenarios where data is sampled progressively, and learning is done in an incremental rather than a batch mode. We seek efficient incremental algorithms for computing feature importance (FI). Permutation feature importance (PFI) is a well-established model-agnostic measure to obtain global FI based on feature marginalization of absent features. We propose an efficient, model-agnostic algorithm called iPFI to estimate this measure incrementally and under dynamic modeling conditions including concept drift. We prove theoretical guarantees on the approximation quality in terms of expectation and variance. To validate our theoretical findings and the efficacy of our approaches in incremental scenarios dealing with streaming data rather than traditional batch settings, we conduct multiple experimental studies on benchmark data with and without concept drift.
Keywords: Artificial Intelligence; Software

On Feature Removal for Explainability in Dynamic Environments - Fumagalli, Fabian; Muschalik, Maximilian; Hüllermeier, Eyke; Hammer, Barbara (ESANN 2023 proceedings)

SHAP-IQ: Unified Approximation of any-order Shapley Interactions - Fumagalli, Fabian; Muschalik, Maximilian; Kolpaczki, Patrick; Hüllermeier, Eyke; Hammer, Barbara (NeurIPS 2023 - Advances in Neural Information Processing Systems)

Folgen wiederholter Negation auf die Aufmerksamkeit - Banh, Ngoc Chi; Scharlau, Ingrid; Rohlfing, Katharina J. (52. Kongress der Deutschen Gesellschaft für Psychologie)

Gesundes Misstrauen gegenüber Erklärungen - Peters, Tobias Martin; Scharlau, Ingrid (52. Kongress der Deutschen Gesellschaft für Psychologie)

Technically enabled explaining of voice characteristics - Wiechmann, Jana; Glarner, Thomas; Rautenberg, Frederik; Wagner, Petra; Haeb-Umbach, Reinhold (18. Phonetik und Phonologie im deutschsprachigen Raum (P&P))

What is Missing in XAI So Far? - Schmid, Ute; Wrede, Britta (KI - Künstliche Intelligenz)
Abstract With the perspective on applications of AI-technology, especially data intensive deep learning approaches, the need for methods to control and understand such models has been recognized and gave rise to a new research domain labeled explainable artificial intelligence (XAI). In this overview paper we give an interim appraisal of what has been achieved so far and where there are still gaps in the research. We take an interdisciplinary perspective to identify challenges on XAI research and point to open questions with respect to the quality of the explanations regarding faithfulness and consistency of explanations. On the other hand we see a need regarding the interaction between XAI and user to allow for adaptability to specific information needs and explanatory dialog for informed decision making as well as the possibility to correct models and explanations by interaction. This endeavor requires an integrated interdisciplinary perspective and rigorous approaches to empirical evaluation based on psychological, linguistic and even sociological theories.
Keywords: Artificial Intelligence

Explainable AI - Schmid, Ute; Wrede, Britta (KI - Künstliche Intelligenz)
Keywords: Artificial Intelligence

AI: Back to the Roots? - Wrede, Britta (KI - Künstliche Intelligenz)
Keywords: Artificial Intelligence

Exploring Monological and Dialogical Phases in Naturally Occurring Explanations - Fisher, Josephine B.; Lohmer, Vivien; Kern, Friederike; Barthlen, Winfried; Gaus, Sebastian; Rohlfing, Katharina (KI - Künstliche Intelligenz)
Abstract Recent approaches to Explainable AI (XAI) promise to satisfy diverse user expectations by allowing them to steer the interaction in order to elicit content relevant to them. However, little is known about how and to what extent the explainee takes part actively in the process of explaining. To tackle this empirical gap, we exploratively examined naturally occurring everyday explanations in doctor–patient interactions ( N = 11). Following the social design of XAI, we view explanations as emerging in interactions: first, we identified the verbal behavior of both the explainer and the explainee in the sequential context, which we could assign to phases that were either monological or dialogical; second, we investigated in particular who was responsible for the initiation of the different phases. Finally, we took a closer look at the global conversational structure of explanations by applying a context-sensitive model of organizational jobs, thus adding a third layer of analysis. Results show that in our small sample of conversational explanations, both monological and dialogical phases varied in their length, timing of occurrence (at the early or later stages of the interaction) and their initiation (by the explainer or the explainee). They alternated several times in the course of the interaction. However, we also found some patterns suggesting that all interactions started with a monological phase initiated by the explainer. Both conversational partners contributed to the core organizational job that constitutes an explanation. We interpret the results as an indication for naturally occurring everyday explanations in doctor–patient interactions to be co-constructed on three levels of linguistic description: (1) by switching back and forth between monological to dialogical phases that (2) can be initiated by both partners and (3) by the mutual accomplishment and thus responsibility for an explanation’s core job that is crucial for the success of the explanation. Because of the explorative nature of our study, these results need to be investigated (a) with a larger sample and (b) in other contexts. However, our results suggest that future designs of artificial explainable systems should design the explanatory dialogue in such a way that it includes monological and dialogical phases that can be initiated not only by the explainer but also by the explainee, as both contribute to the core job of explicating procedural, clausal, or conceptual relations in explanations.
Keywords: Artificial Intelligence

Which “motionese” parameters change with children's age? Disentangling attention-getting from action-structuring modifications - Rohlfing, Katharina; Vollmer, Anna-Lisa; Fritsch, Jannik; Wrede, Britta (Frontiers in Communication)
Modified action demonstration—dubbed motionese— has been proposed as a way to help children recognize the structure and meaning of actions. However, until now, it has been investigated only in young infants. This brief research report presents findings from a cross-sectional study of parental action demonstrations to three groups of 8–11, 12–23, and 24–30-month-old children that applied seven motionese parameters; a second study investigated the youngest group of participants longitudinally to corroborate the cross-sectional results. Results of both studies suggested that four motionese parameters (Motion Pauses, Pace, Velocity, Acceleration) seem to structure the action by organizing it in motion pauses. Whereas these parameters persist over different ages, three other parameters (Demonstration Length, Roundness, and Range) occur predominantly in the younger group and seem to serve to organize infants' attention on the basis of movement. Results are discussed in terms of facilitative vs. pedagogical learning.
Keywords: Social Sciences (miscellaneous); Communication

Formalizing cognitive biases in medical diagnostic reasoning - Battefeld, Dominik; Kopp, Stefan (Presented at the 8th Workshop on Formal and Cognitive Reasoning (FCR), Trier.)
This paper presents preliminary work on the formalization of three prominent cognitive biases in the diagnostic reasoning process over epileptic seizures, psychogenic seizures and syncopes. Diagnostic reasoning is understood as iterative exploration of medical evidence. This exploration is represented as a partially observable Markov decision process where the state (i.e., the correct diagnosis) is uncertain. Observation likelihoods and belief updates are computed using a Bayesian network which defines the interrelation between medical risk factors, diagnoses and potential findings. The decision problem is solved via partially observable upper confidence bounds for trees in Monte-Carlo planning. We compute a biased diagnostic exploration policy by altering the generated state transition, observation and reward during look ahead simulations. The resulting diagnostic policies reproduce reasoning errors which have only been described informally in the medical literature. We plan to use this formal representation in the future to inversely detect and classify biased reasoning in actual diagnostic trajectories obtained from physicians.
Keywords: Diagnostic reasoning; Cognitive bias; Cognitive model; POMDP; Bayesian network; Epilepsy; CDSS

An Architecture Supporting Configurable Autonomous Multimodal Joint-Attention-Therapy for Various Robotic Systems - Groß, André; Schütze, Christian; Wrede, Britta; Richter, Birte (INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION)

Enabling Non-Technical Domain Experts to Create Robot-Assisted Therapeutic Scenarios via Visual Programming - Schütze, Christian; Groß, André; Wrede, Britta; Richter, Birte (INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION)

Agnostic Explanation of Model Change based on Feature Importance - Muschalik, Maximilian; Fumagalli, Fabian; Hammer, Barbara; Huellermeier, Eyke (KI - Künstliche Intelligenz)
Explainable Artificial Intelligence (XAI) has mainly focused on static learning tasks so far. In this paper, we consider XAI in the context of online learning in dynamic environments, such as learning from real-time data streams, where models are learned incrementally and continuously adapted over the course of time. More specifically, we motivate the problem of explaining model change, i.e. explaining the difference between models before and after adaptation, instead of the models themselves. In this regard, we provide the first efficient model-agnostic approach to dynamically detecting, quantifying, and explaining significant model changes. Our approach is based on an adaptation of the well-known Permutation Feature Importance (PFI) measure. It includes two hyperparameters that control the sensitivity and directly influence explanation frequency, so that a human user can adjust the method to individual requirements and application needs. We assess and validate our method’s efficacy on illustrative synthetic data streams with three popular model classes.
Keywords: Artificial Intelligence