On the Integration of Sampling Rate Synchronization and Acoustic Beamforming - Gburrek, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (European Signal Processing Conference (EUSIPCO))

LibriWASN: A Data Set for Meeting Separation, Diarization, and Recognition with Asynchronous Recording Devices - Schmalenstroeer, Joerg; Gburrek, Tobias; Haeb-Umbach, Reinhold (ITG Conference on Speech Communication)

Spatial Diarization for Meeting Transcription with Ad-Hoc Acoustic Sensor Networks - Gburrek, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (Proc. Asilomar Conference on Signals, Systems, and Computers)
We propose a diarization system, that estimates “who spoke when” based on spatial information, to be used as a front-end of a meeting transcription system running on the signals gathered from an acoustic sensor network (ASN). Although the spatial distribution of the microphones is advantageous, exploiting the spatial diversity for diarization and signal enhancement is challenging, because the microphones’ positions are typically unknown, and the recorded signals are initially unsynchronized in general. Here, we approach these issues by first blindly synchronizing the signals and then estimating time differences of arrival (TDOAs). The TDOA information is exploited to estimate the speakers’ activity, even in the presence of multiple speakers being simultaneously active. This speaker activity information serves as a guide for a spatial mixture model, on which basis the individual speaker’s signals are extracted via beamforming. Finally, the extracted signals are forwarded to a speech recognizer. Additionally, a novel initialization scheme for spatial mixture models based on the TDOA estimates is proposed. Experiments conducted on real recordings from the LibriWASN data set have shown that our proposed system is advantageous compared to a system using a spatial mixture model, which does not make use of external diarization information.
Keywords: Diarization; time difference of arrival; ad-hoc acoustic sensor network; meeting transcription

Neural Network Based Carrier Frequency Offset Estimation From Speech Transmitted Over High Frequency Channels - Heitkämper, Jens; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (Proceedings of the 30th European Signal Processing Conference (EUSIPCO))
The intelligibility of demodulated audio signals from analog high frequency transmissions, e.g., using single-sideband (SSB) modulation, can be severely degraded by channel distortions and/or a mismatch between modulation and demodulation carrier frequency. In this work a neural network (NN)-based approach for carrier frequency offset (CFO) estimation from demodulated SSB signals is proposed, whereby a task specific architecture is presented. Additionally, a simulation framework for SSB signals is introduced and utilized for training the NNs. The CFO estimator is combined with a speech enhancement network to investigate its influence on the enhancement performance. The NN-based system is compared to a recently proposed pitch tracking based approach on publicly available data from real high frequency transmissions. Experiments show that the NN exhibits good CFO estimation properties and results in significant improvements in speech intelligibility, especially when combined with a noise reduction network.

Data-driven Time Synchronization in Wireless Multimedia Networks - Afifi, Haitham; Karl, Holger; Gburrek, Tobias; Schmalenstroeer, Joerg (2022 International Wireless Communications and Mobile Computing (IWCMC))

On Synchronization of Wireless Acoustic Sensor Networks in the Presence of Time-Varying Sampling Rate Offsets and Speaker Changes - Gburrek, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))

Informed vs. Blind Beamforming in Ad-Hoc Acoustic Sensor Networks for Meeting Transcription - Gburrek, Tobias; Schmalenstroeer, Joerg; Heitkaemper, Jens; Haeb-Umbach, Reinhold (2022 International Workshop on Acoustic Signal Enhancement (IWAENC))

A Meeting Transcription System for an Ad-Hoc Acoustic Sensor Network - Gburrek, Tobias; Boeddeker, Christoph; von Neumann, Thilo; Cord-Landwehr, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold

A Database for Research on Detection and Enhancement of Speech Transmitted over HF links - Heitkaemper, Jens; Schmalenstroeer, Joerg; Ion, Valentin; Haeb-Umbach, Reinhold (Speech Communication; 14th ITG-Symposium)

Open Range Pitch Tracking for Carrier Frequency Difference Estimation from HF Transmitted Speech - Schmalenstroeer, Joerg; Heitkaemper, Jens; Ullmann, Joerg; Haeb-Umbach, Reinhold (29th European Signal Processing Conference (EUSIPCO))

Geometry calibration in wireless acoustic sensor networks utilizing DoA and distance information - Gburrek, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (EURASIP Journal on Audio, Speech, and Music Processing)
Due to the ad hoc nature of wireless acoustic sensor networks, the position of the sensor nodes is typically unknown. This contribution proposes a technique to estimate the position and orientation of the sensor nodes from the recorded speech signals. The method assumes that a node comprises a microphone array with synchronously sampled microphones rather than a single microphone, but does not require the sampling clocks of the nodes to be synchronized. From the observed audio signals, the distances between the acoustic sources and arrays, as well as the directions of arrival, are estimated. They serve as input to a non-linear least squares problem, from which both the sensor nodes’ positions and orientations, as well as the source positions, are alternatingly estimated in an iterative process. Given one set of unknowns, i.e., either the source positions or the sensor nodes’ geometry, the other set of unknowns can be computed in closed-form. The proposed approach is computationally efficient and the first one, which employs both distance and directional information for geometry calibration in a common cost function. Since both distance and direction of arrival measurements suffer from outliers, e.g., caused by strong reflections of the sound waves on the surfaces of the room, we introduce measures to deemphasize or remove unreliable measurements. Additionally, we discuss modifications of our previously proposed deep neural network-based acoustic distance estimator, to account not only for omnidirectional sources but also for directional sources. Simulation results show good positioning accuracy and compare very favorably with alternative approaches from the literature.

Iterative Geometry Calibration from Distance Estimates for Wireless Acoustic Sensor Networks - Gburrek, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP))

On Source-Microphone Distance Estimation Using Convolutional Recurrent Neural Networks - Gburrek, Tobias; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (Speech Communication; 14th ITG-Symposium)

Online Estimation of Sampling Rate Offsets in Wireless Acoustic Sensor Networks with Packet Loss - Chinaev, Aleksej; Enzner, Gerald; Gburrek, Tobias; Schmalenstroeer, Joerg (29th European Signal Processing Conference (EUSIPCO))

Statistical and Neural Network Based Speech Activity Detection in Non-Stationary Acoustic Environments - Heitkaemper, Jens; Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (INTERSPEECH 2020 Virtual Shanghai China)
Speech activity detection (SAD), which often rests on the fact that the noise is "more'' stationary than speech, is particularly challenging in non-stationary environments, because the time variance of the acoustic scene makes it difficult to discriminate speech from noise. We propose two approaches to SAD, where one is based on statistical signal processing, while the other utilizes neural networks. The former employs sophisticated signal processing to track the noise and speech energies and is meant to support the case for a resource efficient, unsupervised signal processing approach. The latter introduces a recurrent network layer that operates on short segments of the input speech to do temporal smoothing in the presence of non-stationary noise. The systems are tested on the Fearless Steps challenge database, which consists of the transmission data from the Apollo-11 space mission. The statistical SAD achieves comparable detection performance to earlier proposed neural network based SADs, while the neural network based approach leads to a decision cost function of 1.07% on the evaluation set of the 2020 Fearless Steps Challenge, which sets a new state of the art.
Keywords: voice activity detection; speech activity detection; neural network; statistical speech processing

Deep Neural Network based Distance Estimation for Geometry Calibration in Acoustic Sensor Network - Gburrek, Tobias; Schmalenstroeer, Joerg; Brendel, Andreas; Kellermann, Walter; Haeb-Umbach, Reinhold (European Signal Processing Conference (EUSIPCO))
We present an approach to deep neural network based (DNN-based) distance estimation in reverberant rooms for supporting geometry calibration tasks in wireless acoustic sensor networks. Signal diffuseness information from acoustic signals is aggregated via the coherent-to-diffuse power ratio to obtain a distance-related feature, which is mapped to a source-to-microphone distance estimate by means of a DNN. This information is then combined with direction-of-arrival estimates from compact microphone arrays to infer the geometry of the sensor network. Unlike many other approaches to geometry calibration, the proposed scheme does only require that the sampling clocks of the sensor nodes are roughly synchronized. In simulations we show that the proposed DNN-based distance estimator generalizes to unseen acoustic environments and that precise estimates of the sensor node positions are obtained.

Fast and Accurate Audio Resampling for Acoustic Sensor Networks by Polyphase-Farrow Filters with FFT Realization - Schmalenstroeer, Joerg; Chinaev, Aleksej; Enzner, Gerald (Speech Communication; 13th ITG-Symposium)
Arbitrary sampling rate conversion has already received considerable attention in the past, but still lacks an equivalent representation of the effective time-dilation process in the block frequency domain. Good sampling rate converters in the time domain have been known, for instance, in terms of time-varying 'Sinc' or fixed 'Farrow' polynomial filters. The former can deliver nearly exact conversion at high complexity, while the latter has pronounced computational efficiency with limited accuracy. Only recently, it was shown that a composite 'polyphase Farrow' form with high resampling precision can be implemented with quasi-fixed filters that operate at the input sampling rate. We therefore propose to capitalize from that fixed-filter architecture in that we translate the polyphase-Farrow filters into an equivalent FFT-based overlap-save form. Experimental evaluation and comparison with other state-of-the art frequency-domain approaches then proves currently the best price-performance ratio of the proposed algorithm. It is thus an ideal candidate for the new framework of acoustic sensor networks that critically rests upon fast and accurate alignment of autonomous sampling processes.

Front-End Processing for the CHiME-5 Dinner Party Scenario - Boeddeker, Christoph; Heitkaemper, Jens; Schmalenstroeer, Joerg; Drude, Lukas; Heymann, Jahn; Haeb-Umbach, Reinhold (Proc. CHiME 2018 Workshop on Speech Processing in Everyday Environments, Hyderabad, India)
This contribution presents a speech enhancement system for the CHiME-5 Dinner Party Scenario. The front-end employs multi-channel linear time-variant filtering and achieves its gains without the use of a neural network. We present an adaptation of blind source separation techniques to the CHiME-5 database which we call Guided Source Separation (GSS). Using the baseline acoustic and language model, the combination of Weighted Prediction Error based dereverberation, guided source separation, and beamforming reduces the WER by 10:54% (relative) for the single array track and by 21:12% (relative) on the multiple array track.

MARVELO - A Framework for Signal Processing in Wireless Acoustic Sensor Networks - Afifi, Haitham; Schmalenstroeer, Joerg; Ullmann, Joerg; Haeb-Umbach, Reinhold; Karl, Holger (Speech Communication; 13th ITG-Symposium)
Signal processing in WASNs is based on a software framework for hosting the algorithms as well as on a set of wireless connected devices representing the hardware. Each of the nodes contributes memory, processing power, communication bandwidth and some sensor information for the tasks to be solved on the network. In this paper we present our MARVELO framework for distributed signal processing. It is intended for transforming existing centralized implementations into distributed versions. To this end, the software only needs a block-oriented implementation, which MARVELO picks-up and distributes on the network. Additionally, our sensor node hardware and the audio interfaces responsible for multi-channel recordings are presented.

Efficient Sampling Rate Offset Compensation - An Overlap-Save Based Approach - Schmalenstroeer, Joerg; Haeb-Umbach, Reinhold (26th European Signal Processing Conference (EUSIPCO 2018))
Distributed sensor data acquisition usually encompasses data sampling by the individual devices, where each of them has its own oscillator driving the local sampling process, resulting in slightly different sampling rates at the individual sensor nodes. Nevertheless, for certain downstream signal processing tasks it is important to compensate even for small sampling rate offsets. Aligning the sampling rates of oscillators which differ only by a few parts-per-million, is, however, challenging and quite different from traditional multirate signal processing tasks. In this paper we propose to transfer a precise but computationally demanding time domain approach, inspired by the Nyquist-Shannon sampling theorem, to an efficient frequency domain implementation. To this end a buffer control is employed which compensates for sampling offsets which are multiples of the sampling period, while a digital filter, realized by the wellknown Overlap-Save method, handles the fractional part of the sampling phase offset. With experiments on artificially misaligned data we investigate the parametrization, the efficiency, and the induced distortions of the proposed resampling method. It is shown that a favorable compromise between residual distortion and computational complexity is achieved, compared to other sampling rate offset compensation techniques.