Unveiling the Hidden Layer: Ambivalent Cognitive Effects of XAI in Augmented Decision-Making under Uncertainty

This study investigates whether explainable AI (XAI), although intended to improve calibrated reliance in augmented decision-making (ADM), may also increase negative consultations. In a between-subjects online experiment (n = 200), participants completed a price classification task supported by AI, while we manipulated the presence of local feature-based explanations. Drawing on dual-process theory as an interpretive lens, we conceptualize explainability as a cognitively ambivalent intervention that reshapes reliance behavior. Our results show that XAI-induced transparency exerts competing effects: while it directly reduces negative consultations, it simultaneously increases competence-based trust and shifts responsibility attribution toward the AI. These mechanisms foster overreliance and significantly increase negative consultations, resulting in a competitive mediation pattern. Notably, comparable effects do not emerge for positive consultations, suggesting that transparency more readily amplifies reliance than it promotes corrective belief updating. By empirically demonstrating this paradox, our study develops IS and Decision Sciences research on automation bias in transparent ADM settings and identifies key psychological mechanisms through which explainability influences reliance. These findings inform the design of XAI systems that prioritize calibrated reliance rather than merely increasing perceived transparency.