Loose Coupling of Spectral and Spatial Models for Multi-Channel Diarization and Enhancement of Meetings in Dynamic Environments - Meise, Adrian Tobias; Cord-Landwehr, Tobias; Boeddeker, Christoph; Delcroix, Marc; Nakatani, Tomohiro; Haeb-Umbach, Reinhold (ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))
Sound capture by microphone arrays opens the possibility to exploit spatial, in addition to spectral, information for diarization and signal enhancement, two important tasks in meeting transcription. However, there is no one-to-one mapping of positions in space to speakers if speakers move. Here, we address this by proposing a novel joint spatial and spectral mixture model, whose two submodels are loosely coupled by modeling the relationship between speaker and position index probabilistically. Thus, spatial and spectral information can be jointly exploited, while at the same time allowing for speakers speaking from different positions. Experiments on the LibriCSS data set with simulated speaker position changes show great improvements over tightly coupled subsystems.
Keywords: mixture models; meeting processing; diarization; source separation

Spatio-spectral diarization of meetings by combining TDOA-based segmentation and speaker embedding-based clustering - Cord-Landwehr, Tobias; Gburrek, Tobias; Deegen, Marc; Haeb-Umbach, Reinhold (Proceedings of INTERSPEECH)
We propose a spatio-spectral, combined model-based and data-driven diarization pipeline consisting of TDOA-based segmentation followed by embedding-based clustering. The proposed system requires neither access to multi-channel training data nor prior knowledge about the number or placement of microphones. It works for both a compact microphone array and distributed microphones, with minor adjustments. Due to its superior handling of overlapping speech during segmentation, the proposed pipeline significantly outperforms the single-channel pyannote approach, both in a scenario with a compact microphone array and in a setup with distributed microphones. Additionally, we show that, unlike fully spatial diarization pipelines, the proposed system can correctly track speakers when they change positions.

On the Application of Diffusion Models for Simultaneous Denoising and Dereverberation - Meise, Adrian Tobias; Cord-Landwehr, Tobias; Haeb-Umbach, Reinhold (ITG Conference on Speech Communication)

Meeting Recognition with Continuous Speech Separation and Transcription-Supported Diarization - von Neumann, Thilo; Boeddeker, Christoph; Cord-Landwehr, Tobias; Delcroix, Marc; Haeb-Umbach, Reinhold (2024 IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW))

Once more Diarization: Improving meeting transcription systems through segment-level speaker reassignment - Boeddeker, Christoph; Cord-Landwehr, Tobias; Haeb-Umbach, Reinhold (Interspeech 2024)

Simultaneous Diarization and Separation of Meetings through the Integration of Statistical Mixture Models - Cord-Landwehr, Tobias; Boeddeker, Christoph; Haeb-Umbach, Reinhold (ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))
We propose an approach for simultaneous diarization and separation of meeting data. It consists of a complex Angular Central Gaussian Mixture Model (cACGMM) for speech source separation, and a von-Mises-Fisher Mixture Model (VMFMM) for diarization in a joint statistical framework. Through the integration, both spatial and spectral information are exploited for diarization and separation. We also develop a method for counting the number of active speakers in a segment of a meeting to support block-wise processing. While the total number of speakers in a meeting may be known, it is usually not known on a per-segment level. With the proposed speaker counting, joint diarization and source separation can be done segment-by-segment, and the permutation problem across segments is solved, thus allowing for block-online processing in the future. Experimental results on the LibriCSS meeting corpus show that the integrated approach outperforms a cascaded approach of diarization and speech enhancement in terms of WER, both on a per-segment and on a per-meeting level.
Keywords: diarization; source separation; mixture model; meeting

Geodesic Interpolation of Frame-Wise Speaker Embeddings for the Diarization of Meeting Scenarios - Cord-Landwehr, Tobias; Boeddeker, Christoph; Zorilă, Cătălin; Doddipatla, Rama; Haeb-Umbach, Reinhold (ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))

Frame-Wise and Overlap-Robust Speaker Embeddings for Meeting Diarization - Cord-Landwehr, Tobias; Boeddeker, Christoph; Zorilă, Cătălin; Doddipatla, Rama; Haeb-Umbach, Reinhold (ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))

A Teacher-Student Approach for Extracting Informative Speaker Embeddings From Speech Mixtures - Cord-Landwehr, Tobias; Boeddeker, Christoph; Zorilă, Cătălin; Doddipatla, Rama; Haeb-Umbach, Reinhold (INTERSPEECH 2023)

Multi-stage diarization refinement for the CHiME-7 DASR scenario - Boeddeker, Christoph; Cord-Landwehr, Tobias; von Neumann, Thilo; Haeb-Umbach, Reinhold (7th International Workshop on Speech Processing in Everyday Environments (CHiME 2023))

MMS-MSG: A Multi-purpose Multi-Speaker Mixture Signal Generator - Cord-Landwehr, Tobias; von Neumann, Thilo; Boeddeker, Christoph; Haeb-Umbach, Reinhold (2022 International Workshop on Acoustic Signal Enhancement (IWAENC))
The scope of speech enhancement has changed from a monolithic view of single, independent tasks, to a joint processing of complex conversational speech recordings. Training and evaluation of these single tasks requires synthetic data with access to intermediate signals that is as close as possible to the evaluation scenario. As such data often is not available, many works instead use specialized databases for the training of each system component, e.g WSJ0-mix for source separation. We present a Multi-purpose Multi-Speaker Mixture Signal Generator (MMS-MSG) for generating a variety of speech mixture signals based on any speech corpus, ranging from classical anechoic mixtures (e.g., WSJ0-mix) over reverberant mixtures (e.g., SMS-WSJ) to meeting-style data. Its highly modular and flexible structure allows for the simulation of diverse environments and dynamic mixing, while simultaneously enabling an easy extension and modification to generate new scenarios and mixture types. These meetings can be used for prototyping, evaluation, or training purposes. We provide example evaluation data and baseline results for meetings based on the WSJ corpus. Further, we demonstrate the usefulness for realistic scenarios by using MMS-MSG to provide training data for the LibriCSS database.

Monaural source separation: From anechoic to reverberant environments - Cord-Landwehr, Tobias; Boeddeker, Christoph; von Neumann, Thilo; Zorila, Catalin; Doddipatla, Rama; Haeb-Umbach, Reinhold (2022 International Workshop on Acoustic Signal Enhancement (IWAENC))
Impressive progress in neural network-based single-channel speech source separation has been made in recent years. But those improvements have been mostly reported on anechoic data, a situation that is hardly met in practice. Taking the SepFormer as a starting point, which achieves state-of-the-art performance on anechoic mixtures, we gradually modify it to optimize its performance on reverberant mixtures. Although this leads to a word error rate improvement by 7 percentage points compared to the standard SepFormer implementation, the system ends up with only marginally better performance than a PIT-BLSTM separation system, that is optimized with rather straightforward means. This is surprising and at the same time sobering, challenging the practical usefulness of many improvements reported in recent years for monaural source separation on nonreverberant data.

A Meeting Transcription System for an Ad-Hoc Acoustic Sensor Network - Gburrek, Tobias; Boeddeker, Christoph; von Neumann, Thilo; Cord-Landwehr, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold

An Initialization Scheme for Meeting Separation with Spatial Mixture Models - Boeddeker, Christoph; Cord-Landwehr, Tobias; von Neumann, Thilo; Haeb-Umbach, Reinhold (Interspeech 2022)

Contrastive Predictive Coding Supported Factorized Variational Autoencoder for Unsupervised Learning of Disentangled Speech Representations - Ebbers, Janek; Kuhlmann, Michael; Cord-Landwehr, Tobias; Haeb-Umbach, Reinhold (Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))
In this work we address disentanglement of style and content in speech signals. We propose a fully convolutional variational autoencoder employing two encoders: a content encoder and a style encoder. To foster disentanglement, we propose adversarial contrastive predictive coding. This new disentanglement method does neither need parallel data nor any supervision. We show that the proposed technique is capable of separating speaker and content traits into the two different representations and show competitive speaker-content disentanglement performance compared to other unsupervised approaches. We further demonstrate an increased robustness of the content representation against a train-test mismatch compared to spectral features, when used for phone recognition.

Towards a speaker diarization system for the CHiME 2020 dinner party transcription - Boeddeker, Christoph; Cord-Landwehr, Tobias; Heitkaemper, Jens; Zorila, Catalin; Hayakawa, Daichi; Li, Mohan; Liu, Min; Doddipatla, Rama; Haeb-Umbach, Reinhold (Proc. CHiME 2020 Workshop on Speech Processing in Everyday Environments)