Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR

T.C. von Neumann, C. Boeddeker, L. Drude, K. Kinoshita, M. Delcroix, T. Nakatani, R. Haeb-Umbach, T. von Neuann, in: Proc. Interspeech 2020, 2020, pp. 3097–3101.

Download
OA 267.89 KB
Conference Paper | English
Author
; ; ; ; ; ; ;
Abstract
Most approaches to multi-talker overlapped speech separation and recognition assume that the number of simultaneously active speakers is given, but in realistic situations, it is typically unknown. To cope with this, we extend an iterative speech extraction system with mechanisms to count the number of sources and combine it with a single-talker speech recognizer to form the first end-to-end multi-talker automatic speech recognition system for an unknown number of active speakers. Our experiments show very promising performance in counting accuracy, source separation and speech recognition on simulated clean mixtures from WSJ0-2mix and WSJ0-3mix. Among others, we set a new state-of-the-art word error rate on the WSJ0-2mix database. Furthermore, our system generalizes well to a larger number of speakers than it ever saw during training, as shown in experiments with the WSJ0-4mix database.
Publishing Year
Proceedings Title
Proc. Interspeech 2020
Page
3097-3101
LibreCat-ID

Cite this

von Neumann TC, Boeddeker C, Drude L, et al. Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR. In: Proc. Interspeech 2020. ; 2020:3097-3101. doi:10.21437/Interspeech.2020-2519
von Neumann, T. C., Boeddeker, C., Drude, L., Kinoshita, K., Delcroix, M., Nakatani, T., … von Neuann, T. (2020). Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR. In Proc. Interspeech 2020 (pp. 3097–3101). https://doi.org/10.21437/Interspeech.2020-2519
@inproceedings{von Neumann_Boeddeker_Drude_Kinoshita_Delcroix_Nakatani_Haeb-Umbach_von Neuann_2020, title={Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR}, DOI={10.21437/Interspeech.2020-2519}, booktitle={Proc. Interspeech 2020}, author={von Neumann, Thilo Christoph and Boeddeker, Christoph and Drude, Lukas and Kinoshita, Keisuke and Delcroix, Marc and Nakatani, Tomohiro and Haeb-Umbach, Reinhold and von Neuann, Thilo}, year={2020}, pages={3097–3101} }
Neumann, Thilo Christoph von, Christoph Boeddeker, Lukas Drude, Keisuke Kinoshita, Marc Delcroix, Tomohiro Nakatani, Reinhold Haeb-Umbach, and Thilo von Neuann. “Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR.” In Proc. Interspeech 2020, 3097–3101, 2020. https://doi.org/10.21437/Interspeech.2020-2519.
T. C. von Neumann et al., “Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR,” in Proc. Interspeech 2020, 2020, pp. 3097–3101.
von Neumann, Thilo Christoph, et al. “Multi-Talker ASR for an Unknown Number of Sources: Joint Training of Source Counting, Separation and ASR.” Proc. Interspeech 2020, 2020, pp. 3097–101, doi:10.21437/Interspeech.2020-2519.
All files available under the following license(s):
Creative Commons License:
CC0Creative Commons Public Domain Dedication (CC0 1.0)
Main File(s)
Access Level
OA Open Access
Last Uploaded
2020-12-16T14:14:14Z


Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar