---
_id: '15741'
abstract:
- lang: eng
  text: "\r\nIn many cyber–physical systems, we encounter the problem of remote state
    estimation of geo- graphically distributed and remote physical processes. This
    paper studies the scheduling of sensor transmissions to estimate the states of
    multiple remote, dynamic processes. Information from the different sensors has
    to be transmitted to a central gateway over a wireless network for monitoring
    purposes, where typically fewer wireless channels are available than there are
    processes to be monitored. For effective estimation at the gateway, the sensors
    need to be scheduled appropriately, i.e., at each time instant one needs to decide
    which sensors have network access and which ones do not. To address this scheduling
    problem, we formulate an associated Markov decision process (MDP). This MDP is
    then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm
    that is at once scalable and model-free. We compare our scheduling algorithm to
    popular scheduling algorithms such as round-robin and reduced-waiting-time, among
    others. Our algorithm is shown to significantly outperform these algorithms for
    many example scenario"
article_number: '108759'
author:
- first_name: Alex S.
  full_name: Leong, Alex S.
  last_name: Leong
- first_name: Arunselvan
  full_name: Ramaswamy, Arunselvan
  id: '66937'
  last_name: Ramaswamy
  orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Daniel E.
  full_name: Quevedo, Daniel E.
  last_name: Quevedo
- first_name: Holger
  full_name: Karl, Holger
  id: '126'
  last_name: Karl
- first_name: Ling
  full_name: Shi, Ling
  last_name: Shi
citation:
  ama: Leong AS, Ramaswamy A, Quevedo DE, Karl H, Shi L. Deep reinforcement learning
    for wireless sensor scheduling in cyber–physical systems. <i>Automatica</i>. 2019.
    doi:<a href="https://doi.org/10.1016/j.automatica.2019.108759">10.1016/j.automatica.2019.108759</a>
  apa: Leong, A. S., Ramaswamy, A., Quevedo, D. E., Karl, H., &#38; Shi, L. (2019).
    Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems.
    <i>Automatica</i>. <a href="https://doi.org/10.1016/j.automatica.2019.108759">https://doi.org/10.1016/j.automatica.2019.108759</a>
  bibtex: '@article{Leong_Ramaswamy_Quevedo_Karl_Shi_2019, title={Deep reinforcement
    learning for wireless sensor scheduling in cyber–physical systems}, DOI={<a href="https://doi.org/10.1016/j.automatica.2019.108759">10.1016/j.automatica.2019.108759</a>},
    number={108759}, journal={Automatica}, author={Leong, Alex S. and Ramaswamy, Arunselvan
    and Quevedo, Daniel E. and Karl, Holger and Shi, Ling}, year={2019} }'
  chicago: Leong, Alex S., Arunselvan Ramaswamy, Daniel E. Quevedo, Holger Karl, and
    Ling Shi. “Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber–Physical
    Systems.” <i>Automatica</i>, 2019. <a href="https://doi.org/10.1016/j.automatica.2019.108759">https://doi.org/10.1016/j.automatica.2019.108759</a>.
  ieee: A. S. Leong, A. Ramaswamy, D. E. Quevedo, H. Karl, and L. Shi, “Deep reinforcement
    learning for wireless sensor scheduling in cyber–physical systems,” <i>Automatica</i>,
    2019.
  mla: Leong, Alex S., et al. “Deep Reinforcement Learning for Wireless Sensor Scheduling
    in Cyber–Physical Systems.” <i>Automatica</i>, 108759, 2019, doi:<a href="https://doi.org/10.1016/j.automatica.2019.108759">10.1016/j.automatica.2019.108759</a>.
  short: A.S. Leong, A. Ramaswamy, D.E. Quevedo, H. Karl, L. Shi, Automatica (2019).
date_created: 2020-01-31T15:55:27Z
date_updated: 2022-01-06T06:52:32Z
ddc:
- '000'
department:
- _id: '7'
- _id: '34'
- _id: '3'
- _id: '75'
- _id: '57'
doi: 10.1016/j.automatica.2019.108759
file:
- access_level: closed
  content_type: application/pdf
  creator: hkarl
  date_created: 2020-01-31T15:57:50Z
  date_updated: 2020-01-31T15:57:50Z
  file_id: '15743'
  file_name: leoram20a.pdf
  file_size: '675382'
  relation: main_file
  success: 1
file_date_updated: 2020-01-31T15:57:50Z
has_accepted_license: '1'
language:
- iso: eng
project:
- _id: '24'
  name: Netzgewahre Regelung & regelungsgewahre Netze
publication: Automatica
publication_identifier:
  issn:
  - 0005-1098
publication_status: published
quality_controlled: '1'
status: public
title: Deep reinforcement learning for wireless sensor scheduling in cyber–physical
  systems
type: journal_article
user_id: '126'
year: '2019'
...