---
_id: '52958'
author:
- first_name: Christoph
  full_name: Boeddeker, Christoph
  id: '40767'
  last_name: Boeddeker
- first_name: Aswin Shanmugam
  full_name: Subramanian, Aswin Shanmugam
  last_name: Subramanian
- first_name: Gordon
  full_name: Wichern, Gordon
  last_name: Wichern
- first_name: Reinhold
  full_name: Haeb-Umbach, Reinhold
  id: '242'
  last_name: Haeb-Umbach
- first_name: Jonathan
  full_name: Le Roux, Jonathan
  last_name: Le Roux
citation:
  ama: 'Boeddeker C, Subramanian AS, Wichern G, Haeb-Umbach R, Le Roux J. TS-SEP:
    Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings.
    <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>. 2024;32:1185-1197.
    doi:<a href="https://doi.org/10.1109/taslp.2024.3350887">10.1109/taslp.2024.3350887</a>'
  apa: 'Boeddeker, C., Subramanian, A. S., Wichern, G., Haeb-Umbach, R., &#38; Le
    Roux, J. (2024). TS-SEP: Joint Diarization and Separation Conditioned on Estimated
    Speaker Embeddings. <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>,
    <i>32</i>, 1185–1197. <a href="https://doi.org/10.1109/taslp.2024.3350887">https://doi.org/10.1109/taslp.2024.3350887</a>'
  bibtex: '@article{Boeddeker_Subramanian_Wichern_Haeb-Umbach_Le Roux_2024, title={TS-SEP:
    Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings},
    volume={32}, DOI={<a href="https://doi.org/10.1109/taslp.2024.3350887">10.1109/taslp.2024.3350887</a>},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing}, publisher={Institute
    of Electrical and Electronics Engineers (IEEE)}, author={Boeddeker, Christoph
    and Subramanian, Aswin Shanmugam and Wichern, Gordon and Haeb-Umbach, Reinhold
    and Le Roux, Jonathan}, year={2024}, pages={1185–1197} }'
  chicago: 'Boeddeker, Christoph, Aswin Shanmugam Subramanian, Gordon Wichern, Reinhold
    Haeb-Umbach, and Jonathan Le Roux. “TS-SEP: Joint Diarization and Separation Conditioned
    on Estimated Speaker Embeddings.” <i>IEEE/ACM Transactions on Audio, Speech, and
    Language Processing</i> 32 (2024): 1185–97. <a href="https://doi.org/10.1109/taslp.2024.3350887">https://doi.org/10.1109/taslp.2024.3350887</a>.'
  ieee: 'C. Boeddeker, A. S. Subramanian, G. Wichern, R. Haeb-Umbach, and J. Le Roux,
    “TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings,”
    <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>, vol. 32,
    pp. 1185–1197, 2024, doi: <a href="https://doi.org/10.1109/taslp.2024.3350887">10.1109/taslp.2024.3350887</a>.'
  mla: 'Boeddeker, Christoph, et al. “TS-SEP: Joint Diarization and Separation Conditioned
    on Estimated Speaker Embeddings.” <i>IEEE/ACM Transactions on Audio, Speech, and
    Language Processing</i>, vol. 32, Institute of Electrical and Electronics Engineers
    (IEEE), 2024, pp. 1185–97, doi:<a href="https://doi.org/10.1109/taslp.2024.3350887">10.1109/taslp.2024.3350887</a>.'
  short: C. Boeddeker, A.S. Subramanian, G. Wichern, R. Haeb-Umbach, J. Le Roux, IEEE/ACM
    Transactions on Audio, Speech, and Language Processing 32 (2024) 1185–1197.
date_created: 2024-03-26T16:11:54Z
date_updated: 2025-04-16T10:21:45Z
ddc:
- '000'
department:
- _id: '54'
doi: 10.1109/taslp.2024.3350887
file:
- access_level: open_access
  content_type: application/pdf
  creator: cbj
  date_created: 2025-04-16T10:14:47Z
  date_updated: 2025-04-16T10:21:45Z
  file_id: '59602'
  file_name: main.pdf
  file_size: 3432879
  relation: main_file
- access_level: open_access
  content_type: application/pdf
  creator: cbj
  date_created: 2025-04-16T10:15:08Z
  date_updated: 2025-04-16T10:21:45Z
  file_id: '59603'
  file_name: slides.pdf
  file_size: 2838635
  relation: main_file
- access_level: open_access
  content_type: application/pdf
  creator: cbj
  date_created: 2025-04-16T10:15:22Z
  date_updated: 2025-04-16T10:21:45Z
  file_id: '59604'
  file_name: poster.pdf
  file_size: 2038741
  relation: main_file
file_date_updated: 2025-04-16T10:21:45Z
has_accepted_license: '1'
intvolume: '        32'
keyword:
- Electrical and Electronic Engineering
- Acoustics and Ultrasonics
- Computer Science (miscellaneous)
- Computational Mathematics
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://arxiv.org/abs/2303.03849
oa: '1'
page: 1185-1197
project:
- _id: '52'
  name: 'PC2: Computing Resources Provided by the Paderborn Center for Parallel Computing'
- _id: '508'
  grant_number: '448568305'
  name: Automatische Transkription von Gesprächssituationen
publication: IEEE/ACM Transactions on Audio, Speech, and Language Processing
publication_identifier:
  issn:
  - 2329-9290
  - 2329-9304
publication_status: published
publisher: Institute of Electrical and Electronics Engineers (IEEE)
status: public
title: 'TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker
  Embeddings'
type: journal_article
user_id: '40767'
volume: 32
year: '2024'
...
---
_id: '35602'
abstract:
- lang: eng
  text: "Continuous Speech Separation (CSS) has been proposed to address speech overlaps
    during the analysis of realistic meeting-like conversations by eliminating any
    overlaps before further processing.\r\nCSS separates a recording of arbitrarily
    many speakers into a small number of overlap-free output channels, where each
    output channel may contain speech of multiple speakers.\r\nThis is often done
    by applying a conventional separation model trained with Utterance-level Permutation
    Invariant Training (uPIT), which exclusively maps a speaker to an output channel,
    in sliding window approach called stitching.\r\nRecently, we introduced an alternative
    training scheme called Graph-PIT that teaches the separation network to directly
    produce output streams in the required format without stitching.\r\nIt can handle
    an arbitrary number of speakers as long as never more of them overlap at the same
    time than the separator has output channels.\r\nIn this contribution, we further
    investigate the Graph-PIT training scheme.\r\nWe show in extended experiments
    that models trained with Graph-PIT also work in challenging reverberant conditions.\r\nModels
    trained in this way are able to perform segment-less CSS, i.e., without stitching,
    and achieve comparable and often better separation quality than the conventional
    CSS with uPIT and stitching.\r\nWe simplify the training schedule for Graph-PIT
    with the recently proposed Source Aggregated Signal-to-Distortion Ratio (SA-SDR)
    loss.\r\nIt eliminates unfavorable properties of the previously used A-SDR loss
    and thus enables training with Graph-PIT from scratch.\r\nGraph-PIT training relaxes
    the constraints w.r.t. the allowed numbers of speakers and speaking patterns which
    allows using a larger variety of training data.\r\nFurthermore, we introduce novel
    signal-level evaluation metrics for meeting scenarios, namely the source-aggregated
    scale- and convolution-invariant Signal-to-Distortion Ratio (SA-SI-SDR and SA-CI-SDR),
    which are generalizations of the commonly used SDR-based metrics for the CSS case."
article_type: original
author:
- first_name: Thilo
  full_name: von Neumann, Thilo
  id: '49870'
  last_name: von Neumann
  orcid: https://orcid.org/0000-0002-7717-8670
- first_name: Keisuke
  full_name: Kinoshita, Keisuke
  last_name: Kinoshita
- first_name: Christoph
  full_name: Boeddeker, Christoph
  id: '40767'
  last_name: Boeddeker
- first_name: Marc
  full_name: Delcroix, Marc
  last_name: Delcroix
- first_name: Reinhold
  full_name: Haeb-Umbach, Reinhold
  id: '242'
  last_name: Haeb-Umbach
citation:
  ama: 'von Neumann T, Kinoshita K, Boeddeker C, Delcroix M, Haeb-Umbach R. Segment-Less
    Continuous Speech Separation of Meetings: Training and Evaluation Criteria. <i>IEEE/ACM
    Transactions on Audio, Speech, and Language Processing</i>. 2023;31:576-589. doi:<a
    href="https://doi.org/10.1109/taslp.2022.3228629">10.1109/taslp.2022.3228629</a>'
  apa: 'von Neumann, T., Kinoshita, K., Boeddeker, C., Delcroix, M., &#38; Haeb-Umbach,
    R. (2023). Segment-Less Continuous Speech Separation of Meetings: Training and
    Evaluation Criteria. <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>,
    <i>31</i>, 576–589. <a href="https://doi.org/10.1109/taslp.2022.3228629">https://doi.org/10.1109/taslp.2022.3228629</a>'
  bibtex: '@article{von Neumann_Kinoshita_Boeddeker_Delcroix_Haeb-Umbach_2023, title={Segment-Less
    Continuous Speech Separation of Meetings: Training and Evaluation Criteria}, volume={31},
    DOI={<a href="https://doi.org/10.1109/taslp.2022.3228629">10.1109/taslp.2022.3228629</a>},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing}, publisher={Institute
    of Electrical and Electronics Engineers (IEEE)}, author={von Neumann, Thilo and
    Kinoshita, Keisuke and Boeddeker, Christoph and Delcroix, Marc and Haeb-Umbach,
    Reinhold}, year={2023}, pages={576–589} }'
  chicago: 'Neumann, Thilo von, Keisuke Kinoshita, Christoph Boeddeker, Marc Delcroix,
    and Reinhold Haeb-Umbach. “Segment-Less Continuous Speech Separation of Meetings:
    Training and Evaluation Criteria.” <i>IEEE/ACM Transactions on Audio, Speech,
    and Language Processing</i> 31 (2023): 576–89. <a href="https://doi.org/10.1109/taslp.2022.3228629">https://doi.org/10.1109/taslp.2022.3228629</a>.'
  ieee: 'T. von Neumann, K. Kinoshita, C. Boeddeker, M. Delcroix, and R. Haeb-Umbach,
    “Segment-Less Continuous Speech Separation of Meetings: Training and Evaluation
    Criteria,” <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>,
    vol. 31, pp. 576–589, 2023, doi: <a href="https://doi.org/10.1109/taslp.2022.3228629">10.1109/taslp.2022.3228629</a>.'
  mla: 'von Neumann, Thilo, et al. “Segment-Less Continuous Speech Separation of Meetings:
    Training and Evaluation Criteria.” <i>IEEE/ACM Transactions on Audio, Speech,
    and Language Processing</i>, vol. 31, Institute of Electrical and Electronics
    Engineers (IEEE), 2023, pp. 576–89, doi:<a href="https://doi.org/10.1109/taslp.2022.3228629">10.1109/taslp.2022.3228629</a>.'
  short: T. von Neumann, K. Kinoshita, C. Boeddeker, M. Delcroix, R. Haeb-Umbach,
    IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023) 576–589.
date_created: 2023-01-09T17:24:17Z
date_updated: 2023-11-15T12:16:11Z
ddc:
- '000'
department:
- _id: '54'
doi: 10.1109/taslp.2022.3228629
file:
- access_level: open_access
  content_type: application/pdf
  creator: haebumb
  date_created: 2023-01-09T17:46:05Z
  date_updated: 2023-01-11T08:50:19Z
  file_id: '35607'
  file_name: main.pdf
  file_size: 7185077
  relation: main_file
file_date_updated: 2023-01-11T08:50:19Z
has_accepted_license: '1'
intvolume: '        31'
keyword:
- Continuous Speech Separation
- Source Separation
- Graph-PIT
- Dynamic Programming
- Permutation Invariant Training
language:
- iso: eng
oa: '1'
page: 576-589
project:
- _id: '52'
  name: 'PC2: Computing Resources Provided by the Paderborn Center for Parallel Computing'
publication: IEEE/ACM Transactions on Audio, Speech, and Language Processing
publication_identifier:
  issn:
  - 2329-9290
  - 2329-9304
publication_status: published
publisher: Institute of Electrical and Electronics Engineers (IEEE)
quality_controlled: '1'
status: public
title: 'Segment-Less Continuous Speech Separation of Meetings: Training and Evaluation
  Criteria'
type: journal_article
user_id: '49870'
volume: 31
year: '2023'
...
---
_id: '11861'
abstract:
- lang: eng
  text: 'In this contribution we present a theoretical and experimental investigation
    into the effects of reverberation and noise on features in the logarithmic mel
    power spectral domain, an intermediate stage in the computation of the mel frequency
    cepstral coefficients, prevalent in automatic speech recognition (ASR). Gaining
    insight into the complex interaction between clean speech, noise, and noisy reverberant
    speech features is essential for any ASR system to be robust against noise and
    reverberation present in distant microphone input signals. The findings are gathered
    in a probabilistic formulation of an observation model which may be used in model-based
    feature compensation schemes. The proposed observation model extends previous
    models in three major directions: First, the contribution of additive background
    noise to the observation error is explicitly taken into account. Second, an energy
    compensation constant is introduced which ensures an unbiased estimate of the
    reverberant speech features, and, third, a recursive variant of the observation
    model is developed resulting in reduced computational complexity when used in
    model-based feature compensation. The experimental section is used to evaluate
    the accuracy of the model and to describe how its parameters can be determined
    from test data.'
author:
- first_name: Volker
  full_name: Leutnant, Volker
  last_name: Leutnant
- first_name: Alexander
  full_name: Krueger, Alexander
  last_name: Krueger
- first_name: Reinhold
  full_name: Haeb-Umbach, Reinhold
  id: '242'
  last_name: Haeb-Umbach
citation:
  ama: Leutnant V, Krueger A, Haeb-Umbach R. A New Observation Model in the Logarithmic
    Mel Power Spectral Domain for the Automatic Recognition of Noisy Reverberant Speech.
    <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>. 2014;22(1):95-109.
    doi:<a href="https://doi.org/10.1109/TASLP.2013.2285480">10.1109/TASLP.2013.2285480</a>
  apa: Leutnant, V., Krueger, A., &#38; Haeb-Umbach, R. (2014). A New Observation
    Model in the Logarithmic Mel Power Spectral Domain for the Automatic Recognition
    of Noisy Reverberant Speech. <i>IEEE/ACM Transactions on Audio, Speech, and Language
    Processing</i>, <i>22</i>(1), 95–109. <a href="https://doi.org/10.1109/TASLP.2013.2285480">https://doi.org/10.1109/TASLP.2013.2285480</a>
  bibtex: '@article{Leutnant_Krueger_Haeb-Umbach_2014, title={A New Observation Model
    in the Logarithmic Mel Power Spectral Domain for the Automatic Recognition of
    Noisy Reverberant Speech}, volume={22}, DOI={<a href="https://doi.org/10.1109/TASLP.2013.2285480">10.1109/TASLP.2013.2285480</a>},
    number={1}, journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    author={Leutnant, Volker and Krueger, Alexander and Haeb-Umbach, Reinhold}, year={2014},
    pages={95–109} }'
  chicago: 'Leutnant, Volker, Alexander Krueger, and Reinhold Haeb-Umbach. “A New
    Observation Model in the Logarithmic Mel Power Spectral Domain for the Automatic
    Recognition of Noisy Reverberant Speech.” <i>IEEE/ACM Transactions on Audio, Speech,
    and Language Processing</i> 22, no. 1 (2014): 95–109. <a href="https://doi.org/10.1109/TASLP.2013.2285480">https://doi.org/10.1109/TASLP.2013.2285480</a>.'
  ieee: V. Leutnant, A. Krueger, and R. Haeb-Umbach, “A New Observation Model in the
    Logarithmic Mel Power Spectral Domain for the Automatic Recognition of Noisy Reverberant
    Speech,” <i>IEEE/ACM Transactions on Audio, Speech, and Language Processing</i>,
    vol. 22, no. 1, pp. 95–109, 2014.
  mla: Leutnant, Volker, et al. “A New Observation Model in the Logarithmic Mel Power
    Spectral Domain for the Automatic Recognition of Noisy Reverberant Speech.” <i>IEEE/ACM
    Transactions on Audio, Speech, and Language Processing</i>, vol. 22, no. 1, 2014,
    pp. 95–109, doi:<a href="https://doi.org/10.1109/TASLP.2013.2285480">10.1109/TASLP.2013.2285480</a>.
  short: V. Leutnant, A. Krueger, R. Haeb-Umbach, IEEE/ACM Transactions on Audio,
    Speech, and Language Processing 22 (2014) 95–109.
date_created: 2019-07-12T05:29:41Z
date_updated: 2022-01-06T06:51:11Z
department:
- _id: '54'
doi: 10.1109/TASLP.2013.2285480
intvolume: '        22'
issue: '1'
keyword:
- computational complexity
- reverberation
- speech recognition
- automatic speech recognition
- background noise
- clean speech
- computational complexity
- energy compensation
- logarithmic mel power spectral domain
- mel frequency cepstral coefficients
- microphone input signals
- model-based feature compensation schemes
- noisy reverberant speech automatic recognition
- noisy reverberant speech features
- reverberation
- Atmospheric modeling
- Computational modeling
- Noise
- Noise measurement
- Reverberation
- Speech
- Vectors
- Model-based feature compensation
- observation model for reverberant and noisy speech
- recursive observation model
- robust automatic speech recognition
language:
- iso: eng
page: 95-109
publication: IEEE/ACM Transactions on Audio, Speech, and Language Processing
publication_identifier:
  issn:
  - 2329-9290
status: public
title: A New Observation Model in the Logarithmic Mel Power Spectral Domain for the
  Automatic Recognition of Noisy Reverberant Speech
type: journal_article
user_id: '44006'
volume: 22
year: '2014'
...