---
_id: '53801'
abstract:
- lang: eng
  text: 'In this study, we evaluate the impact of gender-biased data from German-language
    physician reviews on the fairness of fine-tuned language models. For two different
    downstream tasks, we use data reported to be gender biased and aggregate it with
    annotations. First, we propose a new approach to aspect-based sentiment analysis
    that allows identifying, extracting, and classifying implicit and explicit aspect
    phrases and their polarity within a single model. The second task we present is
    grade prediction, where we predict the overall grade of a review on the basis
    of the review text. For both tasks, we train numerous transformer models and evaluate
    their performance. The aggregation of sensitive attributes, such as a physician’s
    gender and migration background, with individual text reviews allows us to measure
    the performance of the models with respect to these sensitive groups. These group-wise
    performance measures act as extrinsic bias measures for our downstream tasks.
    In addition, we translate several gender-specific templates of the intrinsic bias
    metrics into the German language and evaluate our fine-tuned models. Based on
    this set of tasks, fine-tuned models, and intrinsic and extrinsic bias measures,
    we perform correlation analyses between intrinsic and extrinsic bias measures.
    In terms of sensitive groups and effect sizes, our bias measure results show different
    directions. Furthermore, correlations between measures of intrinsic and extrinsic
    bias can be observed in different directions. This leads us to conclude that gender-biased
    data does not inherently lead to biased models. Other variables, such as template
    dependency for intrinsic measures and label distribution in the data, must be
    taken into account as they strongly influence the metric results. Therefore, we
    suggest that metrics and templates should be chosen according to the given task
    and the biases to be assessed. '
article_number: '102235'
article_type: original
author:
- first_name: Joschka
  full_name: Kersting, Joschka
  id: '58701'
  last_name: Kersting
- first_name: Falk
  full_name: Maoro, Falk
  last_name: Maoro
- first_name: Michaela
  full_name: Geierhos, Michaela
  last_name: Geierhos
citation:
  ama: 'Kersting J, Maoro F, Geierhos M. Towards comparable ratings: Exploring bias
    in German physician reviews. <i>Data &#38; Knowledge Engineering</i>. 2023;148.
    doi:<a href="https://doi.org/10.1016/j.datak.2023.102235">10.1016/j.datak.2023.102235</a>'
  apa: 'Kersting, J., Maoro, F., &#38; Geierhos, M. (2023). Towards comparable ratings:
    Exploring bias in German physician reviews. <i>Data &#38; Knowledge Engineering</i>,
    <i>148</i>, Article 102235. <a href="https://doi.org/10.1016/j.datak.2023.102235">https://doi.org/10.1016/j.datak.2023.102235</a>'
  bibtex: '@article{Kersting_Maoro_Geierhos_2023, title={Towards comparable ratings:
    Exploring bias in German physician reviews}, volume={148}, DOI={<a href="https://doi.org/10.1016/j.datak.2023.102235">10.1016/j.datak.2023.102235</a>},
    number={102235}, journal={Data &#38; Knowledge Engineering}, publisher={Elsevier},
    author={Kersting, Joschka and Maoro, Falk and Geierhos, Michaela}, year={2023}
    }'
  chicago: 'Kersting, Joschka, Falk Maoro, and Michaela Geierhos. “Towards Comparable
    Ratings: Exploring Bias in German Physician Reviews.” <i>Data &#38; Knowledge
    Engineering</i> 148 (2023). <a href="https://doi.org/10.1016/j.datak.2023.102235">https://doi.org/10.1016/j.datak.2023.102235</a>.'
  ieee: 'J. Kersting, F. Maoro, and M. Geierhos, “Towards comparable ratings: Exploring
    bias in German physician reviews,” <i>Data &#38; Knowledge Engineering</i>, vol.
    148, Art. no. 102235, 2023, doi: <a href="https://doi.org/10.1016/j.datak.2023.102235">10.1016/j.datak.2023.102235</a>.'
  mla: 'Kersting, Joschka, et al. “Towards Comparable Ratings: Exploring Bias in German
    Physician Reviews.” <i>Data &#38; Knowledge Engineering</i>, vol. 148, 102235,
    Elsevier, 2023, doi:<a href="https://doi.org/10.1016/j.datak.2023.102235">10.1016/j.datak.2023.102235</a>.'
  short: J. Kersting, F. Maoro, M. Geierhos, Data &#38; Knowledge Engineering 148
    (2023).
date_created: 2024-04-30T12:30:56Z
date_updated: 2024-04-30T12:41:14Z
ddc:
- '004'
department:
- _id: '579'
doi: 10.1016/j.datak.2023.102235
file:
- access_level: closed
  content_type: application/pdf
  creator: jkers
  date_created: 2024-04-30T12:34:35Z
  date_updated: 2024-04-30T12:34:35Z
  file_id: '53802'
  file_name: Kersting 2023.pdf
  file_size: 1381398
  relation: main_file
  success: 1
file_date_updated: 2024-04-30T12:34:35Z
funded_apc: '1'
has_accepted_license: '1'
intvolume: '       148'
keyword:
- Language model fairness
- Aspect phrase classification
- Grade prediction
- Physician reviews
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
main_file_link:
- open_access: '1'
  url: ' https://doi.org/10.1016/j.datak.2023.102235 '
oa: '1'
project:
- _id: '1'
  grant_number: '160364472'
  name: 'SFB 901: SFB 901: On-The-Fly Computing - Individualisierte IT-Dienstleistungen
    in dynamischen Märkten '
- _id: '3'
  name: 'SFB 901 - B: SFB 901 - Project Area B'
- _id: '9'
  grant_number: '160364472'
  name: 'SFB 901 - B1: SFB 901 - Parametrisierte Servicespezifikation (Subproject
    B1)'
publication: Data & Knowledge Engineering
publication_identifier:
  issn:
  - 0169-023X
publication_status: published
publisher: Elsevier
status: public
title: 'Towards comparable ratings: Exploring bias in German physician reviews'
type: journal_article
user_id: '58701'
volume: 148
year: '2023'
...