--- _id: '30793' author: - first_name: Adrian full_name: Redder, Adrian id: '52265' last_name: Redder orcid: https://orcid.org/0000-0001-7391-4688 - first_name: Arunselvan full_name: Ramaswamy, Arunselvan id: '66937' last_name: Ramaswamy orcid: https://orcid.org/ 0000-0001-7547-8111 - first_name: Holger full_name: Karl, Holger id: '126' last_name: Karl citation: ama: 'Redder A, Ramaswamy A, Karl H. Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication. In: Proceedings of the 14th International Conference on Agents and Artificial Intelligence. SCITEPRESS - Science and Technology Publications; 2022. doi:10.5220/0010845400003116' apa: Redder, A., Ramaswamy, A., & Karl, H. (2022). Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication. Proceedings of the 14th International Conference on Agents and Artificial Intelligence. https://doi.org/10.5220/0010845400003116 bibtex: '@inproceedings{Redder_Ramaswamy_Karl_2022, title={Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication}, DOI={10.5220/0010845400003116}, booktitle={Proceedings of the 14th International Conference on Agents and Artificial Intelligence}, publisher={SCITEPRESS - Science and Technology Publications}, author={Redder, Adrian and Ramaswamy, Arunselvan and Karl, Holger}, year={2022} }' chicago: Redder, Adrian, Arunselvan Ramaswamy, and Holger Karl. “Multi-Agent Policy Gradient Algorithms for Cyber-Physical Systems with Lossy Communication.” In Proceedings of the 14th International Conference on Agents and Artificial Intelligence. SCITEPRESS - Science and Technology Publications, 2022. https://doi.org/10.5220/0010845400003116. ieee: 'A. Redder, A. Ramaswamy, and H. Karl, “Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication,” 2022, doi: 10.5220/0010845400003116.' mla: Redder, Adrian, et al. “Multi-Agent Policy Gradient Algorithms for Cyber-Physical Systems with Lossy Communication.” Proceedings of the 14th International Conference on Agents and Artificial Intelligence, SCITEPRESS - Science and Technology Publications, 2022, doi:10.5220/0010845400003116. short: 'A. Redder, A. Ramaswamy, H. Karl, in: Proceedings of the 14th International Conference on Agents and Artificial Intelligence, SCITEPRESS - Science and Technology Publications, 2022.' date_created: 2022-04-06T07:18:36Z date_updated: 2022-11-18T09:32:14Z ddc: - '006' department: - _id: '75' doi: 10.5220/0010845400003116 file: - access_level: closed content_type: application/pdf creator: aredder date_created: 2022-08-31T07:10:13Z date_updated: 2022-08-31T07:10:13Z file_id: '33237' file_name: ICCART2022.pdf file_size: 298926 relation: main_file success: 1 file_date_updated: 2022-08-31T07:10:13Z has_accepted_license: '1' language: - iso: eng project: - _id: '16' name: 'SFB 901 - C4: SFB 901 - Subproject C4' - _id: '24' name: 'NICCI-CN: Netzgewahre Regelung & regelungsgewahre Netze' - _id: '1' name: 'SFB 901: SFB 901' - _id: '4' name: 'SFB 901 - C: SFB 901 - Project Area C' publication: Proceedings of the 14th International Conference on Agents and Artificial Intelligence publication_status: published publisher: SCITEPRESS - Science and Technology Publications status: public title: Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy Communication type: conference user_id: '477' year: '2022' ... --- _id: '15741' abstract: - lang: eng text: "\r\nIn many cyber–physical systems, we encounter the problem of remote state estimation of geo- graphically distributed and remote physical processes. This paper studies the scheduling of sensor transmissions to estimate the states of multiple remote, dynamic processes. Information from the different sensors has to be transmitted to a central gateway over a wireless network for monitoring purposes, where typically fewer wireless channels are available than there are processes to be monitored. For effective estimation at the gateway, the sensors need to be scheduled appropriately, i.e., at each time instant one needs to decide which sensors have network access and which ones do not. To address this scheduling problem, we formulate an associated Markov decision process (MDP). This MDP is then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm that is at once scalable and model-free. We compare our scheduling algorithm to popular scheduling algorithms such as round-robin and reduced-waiting-time, among others. Our algorithm is shown to significantly outperform these algorithms for many example scenario" article_number: '108759' author: - first_name: Alex S. full_name: Leong, Alex S. last_name: Leong - first_name: Arunselvan full_name: Ramaswamy, Arunselvan id: '66937' last_name: Ramaswamy orcid: https://orcid.org/ 0000-0001-7547-8111 - first_name: Daniel E. full_name: Quevedo, Daniel E. last_name: Quevedo - first_name: Holger full_name: Karl, Holger id: '126' last_name: Karl - first_name: Ling full_name: Shi, Ling last_name: Shi citation: ama: Leong AS, Ramaswamy A, Quevedo DE, Karl H, Shi L. Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems. Automatica. 2019. doi:10.1016/j.automatica.2019.108759 apa: Leong, A. S., Ramaswamy, A., Quevedo, D. E., Karl, H., & Shi, L. (2019). Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems. Automatica. https://doi.org/10.1016/j.automatica.2019.108759 bibtex: '@article{Leong_Ramaswamy_Quevedo_Karl_Shi_2019, title={Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems}, DOI={10.1016/j.automatica.2019.108759}, number={108759}, journal={Automatica}, author={Leong, Alex S. and Ramaswamy, Arunselvan and Quevedo, Daniel E. and Karl, Holger and Shi, Ling}, year={2019} }' chicago: Leong, Alex S., Arunselvan Ramaswamy, Daniel E. Quevedo, Holger Karl, and Ling Shi. “Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber–Physical Systems.” Automatica, 2019. https://doi.org/10.1016/j.automatica.2019.108759. ieee: A. S. Leong, A. Ramaswamy, D. E. Quevedo, H. Karl, and L. Shi, “Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems,” Automatica, 2019. mla: Leong, Alex S., et al. “Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber–Physical Systems.” Automatica, 108759, 2019, doi:10.1016/j.automatica.2019.108759. short: A.S. Leong, A. Ramaswamy, D.E. Quevedo, H. Karl, L. Shi, Automatica (2019). date_created: 2020-01-31T15:55:27Z date_updated: 2022-01-06T06:52:32Z ddc: - '000' department: - _id: '7' - _id: '34' - _id: '3' - _id: '75' - _id: '57' doi: 10.1016/j.automatica.2019.108759 file: - access_level: closed content_type: application/pdf creator: hkarl date_created: 2020-01-31T15:57:50Z date_updated: 2020-01-31T15:57:50Z file_id: '15743' file_name: leoram20a.pdf file_size: '675382' relation: main_file success: 1 file_date_updated: 2020-01-31T15:57:50Z has_accepted_license: '1' language: - iso: eng project: - _id: '24' name: Netzgewahre Regelung & regelungsgewahre Netze publication: Automatica publication_identifier: issn: - 0005-1098 publication_status: published quality_controlled: '1' status: public title: Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems type: journal_article user_id: '126' year: '2019' ... --- _id: '2476' author: - first_name: Binyam full_name: Shiferaw Heyi, Binyam last_name: Shiferaw Heyi - first_name: Holger full_name: Karl, Holger id: '126' last_name: Karl citation: ama: 'Shiferaw Heyi B, Karl H. Modelling Time-Limited Capacity of a Wireless Channel as aMarkov Reward Process. In: Proc. of IEEE Wireless Communications and Networking Conference (WCNC); 2018.' apa: Shiferaw Heyi, B., & Karl, H. (2018). Modelling Time-Limited Capacity of a Wireless Channel as aMarkov Reward Process. Proc. of IEEE Wireless Communications and Networking Conference (WCNC). bibtex: '@inproceedings{Shiferaw Heyi_Karl_2018, title={Modelling Time-Limited Capacity of a Wireless Channel as aMarkov Reward Process}, publisher={Proc. of IEEE Wireless Communications and Networking Conference (WCNC)}, author={Shiferaw Heyi, Binyam and Karl, Holger}, year={2018} }' chicago: Shiferaw Heyi, Binyam, and Holger Karl. “Modelling Time-Limited Capacity of a Wireless Channel as AMarkov Reward Process.” Proc. of IEEE Wireless Communications and Networking Conference (WCNC), 2018. ieee: B. Shiferaw Heyi and H. Karl, “Modelling Time-Limited Capacity of a Wireless Channel as aMarkov Reward Process,” 2018. mla: Shiferaw Heyi, Binyam, and Holger Karl. Modelling Time-Limited Capacity of a Wireless Channel as AMarkov Reward Process. Proc. of IEEE Wireless Communications and Networking Conference (WCNC), 2018. short: 'B. Shiferaw Heyi, H. Karl, in: Proc. of IEEE Wireless Communications and Networking Conference (WCNC), 2018.' date_created: 2018-04-24T08:09:00Z date_updated: 2022-01-06T06:56:34Z ddc: - '000' department: - _id: '75' file: - access_level: closed content_type: application/pdf creator: tabu date_created: 2018-04-24T08:08:50Z date_updated: 2018-04-24T08:08:50Z file_id: '2477' file_name: p2861-heyi.pdf file_size: 428839 relation: main_file success: 1 file_date_updated: 2018-04-24T08:08:50Z has_accepted_license: '1' project: - _id: '24' name: Netzgewahre Regelung & regelungsgewahre Netze publisher: Proc. of IEEE Wireless Communications and Networking Conference (WCNC) status: public title: Modelling Time-Limited Capacity of a Wireless Channel as aMarkov Reward Process type: conference user_id: '15572' year: '2018' ... --- _id: '3217' author: - first_name: Burak full_name: Demirel, Burak last_name: Demirel - first_name: Arunselvan full_name: Ramaswamy, Arunselvan last_name: Ramaswamy - first_name: Daniel full_name: Quevedo, Daniel last_name: Quevedo - first_name: Holger full_name: Karl, Holger id: '126' last_name: Karl citation: ama: 'Demirel B, Ramaswamy A, Quevedo D, Karl H. DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling. In: ; 2018. doi:10.1109/LCSYS.2018.2847721' apa: 'Demirel, B., Ramaswamy, A., Quevedo, D., & Karl, H. (2018). DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling. https://doi.org/10.1109/LCSYS.2018.2847721' bibtex: '@inproceedings{Demirel_Ramaswamy_Quevedo_Karl_2018, title={DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling}, DOI={10.1109/LCSYS.2018.2847721}, author={Demirel, Burak and Ramaswamy, Arunselvan and Quevedo, Daniel and Karl, Holger}, year={2018} }' chicago: 'Demirel, Burak, Arunselvan Ramaswamy, Daniel Quevedo, and Holger Karl. “DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling,” 2018. https://doi.org/10.1109/LCSYS.2018.2847721.' ieee: 'B. Demirel, A. Ramaswamy, D. Quevedo, and H. Karl, “DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling,” 2018.' mla: 'Demirel, Burak, et al. DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling. 2018, doi:10.1109/LCSYS.2018.2847721.' short: 'B. Demirel, A. Ramaswamy, D. Quevedo, H. Karl, in: 2018.' date_created: 2018-06-13T10:26:23Z date_updated: 2022-01-06T06:59:05Z ddc: - '000' department: - _id: '75' doi: 10.1109/LCSYS.2018.2847721 file: - access_level: closed content_type: application/pdf creator: tabu date_created: 2018-06-13T10:47:57Z date_updated: 2018-06-13T10:47:57Z file_id: '3218' file_name: 1803.02998.pdf file_size: 354166 relation: main_file success: 1 file_date_updated: 2018-06-13T10:47:57Z has_accepted_license: '1' main_file_link: - url: https://arxiv.org/pdf/1803.02998.pdf project: - _id: '24' name: Netzgewahre Regelung & regelungsgewahre Netze status: public title: 'DeepCAS: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling' type: conference user_id: '126' year: '2018' ...