article yes Christopher Grimm author Tai Fei author Ernst Warsitz author Ridha Farhoud author Tobias Breddermann author Reinhold Haeb-Umbach author 242 54 department We present an approach to automatically generate semantic labels for real recordings of automotive range-Doppler (RD) radar spectra. Such labels are required when training a neural network for object recognition from radar data. The automatic labeling approach rests on the simultaneous recording of camera and lidar data in addition to the radar spectrum. By warping radar spectra into the camera image, state-of-the-art object recognition algorithms can be applied to label relevant objects, such as cars, in the camera image. The warping operation is designed to be fully differentiable, which allows backpropagating the gradient computed on the camera image through the warping operation to the neural network operating on the radar data. As the warping operation relies on accurate scene flow estimation, we further propose a novel scene flow estimation algorithm which exploits information from camera, lidar and radar sensors. The proposed scene flow estimation approach is compared against a state-of-the-art scene flow algorithm, and it outperforms it by approximately 30% w.r.t. mean average error. The feasibility of the overall framework for automatic label generation for RD spectra is verified by evaluating the performance of neural networks trained with the proposed framework for Direction-of-Arrival estimation. https://ris.uni-paderborn.de/download/33451/33460/T-VT_AcceptedVersion.pdf application/pdfno https://creativecommons.org/publicdomain/zero/1.0/ 2022 eng 10.1109/TVT.2022.3182411 7199435-9449 Grimm, Christopher, Tai Fei, Ernst Warsitz, Ridha Farhoud, Tobias Breddermann, and Reinhold Haeb-Umbach. “Warping of Radar Data Into Camera Image for Cross-Modal Supervision in Automotive Applications.” <i>IEEE Transactions on Vehicular Technology</i> 71, no. 9 (2022): 9435–49. <a href="https://doi.org/10.1109/TVT.2022.3182411">https://doi.org/10.1109/TVT.2022.3182411</a>. C. Grimm, T. Fei, E. Warsitz, R. Farhoud, T. Breddermann, R. Haeb-Umbach, IEEE Transactions on Vehicular Technology 71 (2022) 9435–9449. Grimm, C., Fei, T., Warsitz, E., Farhoud, R., Breddermann, T., &#38; Haeb-Umbach, R. (2022). Warping of Radar Data Into Camera Image for Cross-Modal Supervision in Automotive Applications. <i>IEEE Transactions on Vehicular Technology</i>, <i>71</i>(9), 9435–9449. <a href="https://doi.org/10.1109/TVT.2022.3182411">https://doi.org/10.1109/TVT.2022.3182411</a> C. Grimm, T. Fei, E. Warsitz, R. Farhoud, T. Breddermann, and R. Haeb-Umbach, “Warping of Radar Data Into Camera Image for Cross-Modal Supervision in Automotive Applications,” <i>IEEE Transactions on Vehicular Technology</i>, vol. 71, no. 9, pp. 9435–9449, 2022, doi: <a href="https://doi.org/10.1109/TVT.2022.3182411">10.1109/TVT.2022.3182411</a>. Grimm C, Fei T, Warsitz E, Farhoud R, Breddermann T, Haeb-Umbach R. Warping of Radar Data Into Camera Image for Cross-Modal Supervision in Automotive Applications. <i>IEEE Transactions on Vehicular Technology</i>. 2022;71(9):9435-9449. doi:<a href="https://doi.org/10.1109/TVT.2022.3182411">10.1109/TVT.2022.3182411</a> @article{Grimm_Fei_Warsitz_Farhoud_Breddermann_Haeb-Umbach_2022, title={Warping of Radar Data Into Camera Image for Cross-Modal Supervision in Automotive Applications}, volume={71}, DOI={<a href="https://doi.org/10.1109/TVT.2022.3182411">10.1109/TVT.2022.3182411</a>}, number={9}, journal={IEEE Transactions on Vehicular Technology}, author={Grimm, Christopher and Fei, Tai and Warsitz, Ernst and Farhoud, Ridha and Breddermann, Tobias and Haeb-Umbach, Reinhold}, year={2022}, pages={9435–9449} } Grimm, Christopher, et al. “Warping of Radar Data Into Camera Image for Cross-Modal Supervision in Automotive Applications.” <i>IEEE Transactions on Vehicular Technology</i>, vol. 71, no. 9, 2022, pp. 9435–49, doi:<a href="https://doi.org/10.1109/TVT.2022.3182411">10.1109/TVT.2022.3182411</a>. 334512022-09-21T07:26:19Z2023-11-20T16:37:16Z