{"funded_apc":"1","oa":"1","has_accepted_license":"1","year":"2023","author":[{"id":"58701","full_name":"Kersting, Joschka","last_name":"Kersting","first_name":"Joschka"},{"full_name":"Maoro, Falk","last_name":"Maoro","first_name":"Falk"},{"first_name":"Michaela","full_name":"Geierhos, Michaela","last_name":"Geierhos"}],"department":[{"_id":"579"}],"publisher":"Elsevier","file":[{"success":1,"date_created":"2024-04-30T12:34:35Z","creator":"jkers","content_type":"application/pdf","file_id":"53802","file_name":"Kersting 2023.pdf","date_updated":"2024-04-30T12:34:35Z","file_size":1381398,"relation":"main_file","access_level":"closed"}],"date_created":"2024-04-30T12:30:56Z","main_file_link":[{"open_access":"1","url":" https://doi.org/10.1016/j.datak.2023.102235 "}],"article_number":"102235","citation":{"ieee":"J. Kersting, F. Maoro, and M. Geierhos, “Towards comparable ratings: Exploring bias in German physician reviews,” Data & Knowledge Engineering, vol. 148, Art. no. 102235, 2023, doi: 10.1016/j.datak.2023.102235.","short":"J. Kersting, F. Maoro, M. Geierhos, Data & Knowledge Engineering 148 (2023).","bibtex":"@article{Kersting_Maoro_Geierhos_2023, title={Towards comparable ratings: Exploring bias in German physician reviews}, volume={148}, DOI={10.1016/j.datak.2023.102235}, number={102235}, journal={Data & Knowledge Engineering}, publisher={Elsevier}, author={Kersting, Joschka and Maoro, Falk and Geierhos, Michaela}, year={2023} }","ama":"Kersting J, Maoro F, Geierhos M. Towards comparable ratings: Exploring bias in German physician reviews. Data & Knowledge Engineering. 2023;148. doi:10.1016/j.datak.2023.102235","chicago":"Kersting, Joschka, Falk Maoro, and Michaela Geierhos. “Towards Comparable Ratings: Exploring Bias in German Physician Reviews.” Data & Knowledge Engineering 148 (2023). https://doi.org/10.1016/j.datak.2023.102235.","apa":"Kersting, J., Maoro, F., & Geierhos, M. (2023). Towards comparable ratings: Exploring bias in German physician reviews. Data & Knowledge Engineering, 148, Article 102235. https://doi.org/10.1016/j.datak.2023.102235","mla":"Kersting, Joschka, et al. “Towards Comparable Ratings: Exploring Bias in German Physician Reviews.” Data & Knowledge Engineering, vol. 148, 102235, Elsevier, 2023, doi:10.1016/j.datak.2023.102235."},"intvolume":" 148","file_date_updated":"2024-04-30T12:34:35Z","project":[{"_id":"1","grant_number":"160364472","name":"SFB 901: SFB 901: On-The-Fly Computing - Individualisierte IT-Dienstleistungen in dynamischen Märkten "},{"_id":"3","name":"SFB 901 - B: SFB 901 - Project Area B"},{"_id":"9","grant_number":"160364472","name":"SFB 901 - B1: SFB 901 - Parametrisierte Servicespezifikation (Subproject B1)"}],"keyword":["Language model fairness","Aspect phrase classification","Grade prediction","Physician reviews"],"status":"public","language":[{"iso":"eng"}],"article_type":"original","volume":148,"publication_status":"published","user_id":"58701","type":"journal_article","doi":"10.1016/j.datak.2023.102235","_id":"53801","ddc":["004"],"date_updated":"2024-04-30T12:41:14Z","publication_identifier":{"issn":["0169-023X"]},"abstract":[{"text":"In this study, we evaluate the impact of gender-biased data from German-language physician reviews on the fairness of fine-tuned language models. For two different downstream tasks, we use data reported to be gender biased and aggregate it with annotations. First, we propose a new approach to aspect-based sentiment analysis that allows identifying, extracting, and classifying implicit and explicit aspect phrases and their polarity within a single model. The second task we present is grade prediction, where we predict the overall grade of a review on the basis of the review text. For both tasks, we train numerous transformer models and evaluate their performance. The aggregation of sensitive attributes, such as a physician’s gender and migration background, with individual text reviews allows us to measure the performance of the models with respect to these sensitive groups. These group-wise performance measures act as extrinsic bias measures for our downstream tasks. In addition, we translate several gender-specific templates of the intrinsic bias metrics into the German language and evaluate our fine-tuned models. Based on this set of tasks, fine-tuned models, and intrinsic and extrinsic bias measures, we perform correlation analyses between intrinsic and extrinsic bias measures. In terms of sensitive groups and effect sizes, our bias measure results show different directions. Furthermore, correlations between measures of intrinsic and extrinsic bias can be observed in different directions. This leads us to conclude that gender-biased data does not inherently lead to biased models. Other variables, such as template dependency for intrinsic measures and label distribution in the data, must be taken into account as they strongly influence the metric results. Therefore, we suggest that metrics and templates should be chosen according to the given task and the biases to be assessed. ","lang":"eng"}],"title":"Towards comparable ratings: Exploring bias in German physician reviews","publication":"Data & Knowledge Engineering"}