---
_id: '32811'
abstract:
- lang: eng
text: 'The decentralized nature of multi-agent systems requires continuous data
exchange to achieve global objectives. In such scenarios, Age of Information (AoI)
has become an important metric of the freshness of exchanged data due to the error-proneness
and delays of communication systems. Communication systems usually possess dependencies:
the process describing the success or failure of communication is highly correlated
when these attempts are ``close'''' in some domain (e.g. in time, frequency, space
or code as in wireless communication) and is, in general, non-stationary. To study
AoI in such scenarios, we consider an abstract event-based AoI process $\Delta(n)$,
expressing time since the last update: If, at time $n$, a monitoring node receives
a status update from a source node (event $A(n-1)$ occurs), then $\Delta(n)$ is
reset to one; otherwise, $\Delta(n)$ grows linearly in time. This AoI process
can thus be viewed as a special random walk with resets. The event process $A(n)$
may be nonstationary and we merely assume that its temporal dependencies decay
sufficiently, described by $\alpha$-mixing. We calculate moment bounds for the
resulting AoI process as a function of the mixing rate of $A(n)$. Furthermore,
we prove that the AoI process $\Delta(n)$ is itself $\alpha$-mixing from which
we conclude a strong law of large numbers for $\Delta(n)$. These results are new,
since AoI processes have not been studied so far in this general strongly mixing
setting. This opens up future work on renewal processes with non-independent interarrival
times.'
author:
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: 'Redder A, Ramaswamy A, Karl H. Age of Information Process under Strongly Mixing
Communication -- Moment Bound, Mixing Rate and Strong Law. In: Proceedings
of the 58th Allerton Conference on Communication, Control, and Computing.
; 2022.'
apa: Redder, A., Ramaswamy, A., & Karl, H. (2022). Age of Information Process
under Strongly Mixing Communication -- Moment Bound, Mixing Rate and Strong Law.
Proceedings of the 58th Allerton Conference on Communication, Control, and
Computing. 58th Allerton Conference on Communication, Control, and Computing.
bibtex: '@inproceedings{Redder_Ramaswamy_Karl_2022, title={Age of Information Process
under Strongly Mixing Communication -- Moment Bound, Mixing Rate and Strong Law},
booktitle={Proceedings of the 58th Allerton Conference on Communication, Control,
and Computing}, author={Redder, Adrian and Ramaswamy, Arunselvan and Karl, Holger},
year={2022} }'
chicago: Redder, Adrian, Arunselvan Ramaswamy, and Holger Karl. “Age of Information
Process under Strongly Mixing Communication -- Moment Bound, Mixing Rate and Strong
Law.” In Proceedings of the 58th Allerton Conference on Communication, Control,
and Computing, 2022.
ieee: A. Redder, A. Ramaswamy, and H. Karl, “Age of Information Process under Strongly
Mixing Communication -- Moment Bound, Mixing Rate and Strong Law,” presented at
the 58th Allerton Conference on Communication, Control, and Computing, 2022.
mla: Redder, Adrian, et al. “Age of Information Process under Strongly Mixing Communication
-- Moment Bound, Mixing Rate and Strong Law.” Proceedings of the 58th Allerton
Conference on Communication, Control, and Computing, 2022.
short: 'A. Redder, A. Ramaswamy, H. Karl, in: Proceedings of the 58th Allerton Conference
on Communication, Control, and Computing, 2022.'
conference:
name: 58th Allerton Conference on Communication, Control, and Computing
date_created: 2022-08-15T09:59:17Z
date_updated: 2022-11-18T09:31:19Z
ddc:
- '000'
department:
- _id: '75'
has_accepted_license: '1'
language:
- iso: eng
project:
- _id: '16'
name: 'SFB 901 - C4: SFB 901 - Subproject C4'
- _id: '1'
name: 'SFB 901: SFB 901'
- _id: '4'
name: 'SFB 901 - C: SFB 901 - Project Area C'
publication: Proceedings of the 58th Allerton Conference on Communication, Control,
and Computing
status: public
title: Age of Information Process under Strongly Mixing Communication -- Moment Bound,
Mixing Rate and Strong Law
type: conference
user_id: '477'
year: '2022'
...
---
_id: '30793'
author:
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: 'Redder A, Ramaswamy A, Karl H. Multi-agent Policy Gradient Algorithms for
Cyber-physical Systems with Lossy Communication. In: Proceedings of the 14th
International Conference on Agents and Artificial Intelligence. SCITEPRESS
- Science and Technology Publications; 2022. doi:10.5220/0010845400003116'
apa: Redder, A., Ramaswamy, A., & Karl, H. (2022). Multi-agent Policy Gradient
Algorithms for Cyber-physical Systems with Lossy Communication. Proceedings
of the 14th International Conference on Agents and Artificial Intelligence.
https://doi.org/10.5220/0010845400003116
bibtex: '@inproceedings{Redder_Ramaswamy_Karl_2022, title={Multi-agent Policy Gradient
Algorithms for Cyber-physical Systems with Lossy Communication}, DOI={10.5220/0010845400003116},
booktitle={Proceedings of the 14th International Conference on Agents and Artificial
Intelligence}, publisher={SCITEPRESS - Science and Technology Publications}, author={Redder,
Adrian and Ramaswamy, Arunselvan and Karl, Holger}, year={2022} }'
chicago: Redder, Adrian, Arunselvan Ramaswamy, and Holger Karl. “Multi-Agent Policy
Gradient Algorithms for Cyber-Physical Systems with Lossy Communication.” In Proceedings
of the 14th International Conference on Agents and Artificial Intelligence.
SCITEPRESS - Science and Technology Publications, 2022. https://doi.org/10.5220/0010845400003116.
ieee: 'A. Redder, A. Ramaswamy, and H. Karl, “Multi-agent Policy Gradient Algorithms
for Cyber-physical Systems with Lossy Communication,” 2022, doi: 10.5220/0010845400003116.'
mla: Redder, Adrian, et al. “Multi-Agent Policy Gradient Algorithms for Cyber-Physical
Systems with Lossy Communication.” Proceedings of the 14th International Conference
on Agents and Artificial Intelligence, SCITEPRESS - Science and Technology
Publications, 2022, doi:10.5220/0010845400003116.
short: 'A. Redder, A. Ramaswamy, H. Karl, in: Proceedings of the 14th International
Conference on Agents and Artificial Intelligence, SCITEPRESS - Science and Technology
Publications, 2022.'
date_created: 2022-04-06T07:18:36Z
date_updated: 2022-11-18T09:32:14Z
ddc:
- '006'
department:
- _id: '75'
doi: 10.5220/0010845400003116
file:
- access_level: closed
content_type: application/pdf
creator: aredder
date_created: 2022-08-31T07:10:13Z
date_updated: 2022-08-31T07:10:13Z
file_id: '33237'
file_name: ICCART2022.pdf
file_size: 298926
relation: main_file
success: 1
file_date_updated: 2022-08-31T07:10:13Z
has_accepted_license: '1'
language:
- iso: eng
project:
- _id: '16'
name: 'SFB 901 - C4: SFB 901 - Subproject C4'
- _id: '24'
name: 'NICCI-CN: Netzgewahre Regelung & regelungsgewahre Netze'
- _id: '1'
name: 'SFB 901: SFB 901'
- _id: '4'
name: 'SFB 901 - C: SFB 901 - Project Area C'
publication: Proceedings of the 14th International Conference on Agents and Artificial
Intelligence
publication_status: published
publisher: SCITEPRESS - Science and Technology Publications
status: public
title: Multi-agent Policy Gradient Algorithms for Cyber-physical Systems with Lossy
Communication
type: conference
user_id: '477'
year: '2022'
...
---
_id: '30790'
abstract:
- lang: eng
text: "Iterative distributed optimization algorithms involve multiple agents that\r\ncommunicate
with each other, over time, in order to minimize/maximize a global\r\nobjective.
In the presence of unreliable communication networks, the\r\nAge-of-Information
(AoI), which measures the freshness of data received, may be\r\nlarge and hence
hinder algorithmic convergence. In this paper, we study the\r\nconvergence of
general distributed gradient-based optimization algorithms in\r\nthe presence
of communication that neither happens periodically nor at\r\nstochastically independent
points in time. We show that convergence is\r\nguaranteed provided the random
variables associated with the AoI processes are\r\nstochastically dominated by
a random variable with finite first moment. This\r\nimproves on previous requirements
of boundedness of more than the first moment.\r\nWe then introduce stochastically
strongly connected (SSC) networks, a new\r\nstochastic form of strong connectedness
for time-varying networks. We show: If\r\nfor any $p \\ge0$ the processes that
describe the success of communication\r\nbetween agents in a SSC network are $\\alpha$-mixing
with $n^{p-1}\\alpha(n)$\r\nsummable, then the associated AoI processes are stochastically
dominated by a\r\nrandom variable with finite $p$-th moment. In combination with
our first\r\ncontribution, this implies that distributed stochastic gradient descend\r\nconverges
in the presence of AoI, if $\\alpha(n)$ is summable."
author:
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: Redder A, Ramaswamy A, Karl H. Distributed gradient-based optimization in the
presence of dependent aperiodic communication. arXiv:220111343. Published
online 2022.
apa: Redder, A., Ramaswamy, A., & Karl, H. (2022). Distributed gradient-based
optimization in the presence of dependent aperiodic communication. In arXiv:2201.11343.
bibtex: '@article{Redder_Ramaswamy_Karl_2022, title={Distributed gradient-based
optimization in the presence of dependent aperiodic communication}, journal={arXiv:2201.11343},
author={Redder, Adrian and Ramaswamy, Arunselvan and Karl, Holger}, year={2022}
}'
chicago: Redder, Adrian, Arunselvan Ramaswamy, and Holger Karl. “Distributed Gradient-Based
Optimization in the Presence of Dependent Aperiodic Communication.” ArXiv:2201.11343,
2022.
ieee: A. Redder, A. Ramaswamy, and H. Karl, “Distributed gradient-based optimization
in the presence of dependent aperiodic communication,” arXiv:2201.11343.
2022.
mla: Redder, Adrian, et al. “Distributed Gradient-Based Optimization in the Presence
of Dependent Aperiodic Communication.” ArXiv:2201.11343, 2022.
short: A. Redder, A. Ramaswamy, H. Karl, ArXiv:2201.11343 (2022).
date_created: 2022-04-06T06:53:38Z
date_updated: 2022-11-18T09:33:01Z
department:
- _id: '75'
external_id:
arxiv:
- '2201.11343'
language:
- iso: eng
project:
- _id: '16'
name: 'SFB 901 - C4: SFB 901 - Subproject C4'
- _id: '1'
name: 'SFB 901: SFB 901'
- _id: '4'
name: 'SFB 901 - C: SFB 901 - Project Area C'
publication: arXiv:2201.11343
status: public
title: Distributed gradient-based optimization in the presence of dependent aperiodic
communication
type: preprint
user_id: '477'
year: '2022'
...
---
_id: '30791'
abstract:
- lang: eng
text: "We present sufficient conditions that ensure convergence of the multi-agent\r\nDeep
Deterministic Policy Gradient (DDPG) algorithm. It is an example of one of\r\nthe
most popular paradigms of Deep Reinforcement Learning (DeepRL) for tackling\r\ncontinuous
action spaces: the actor-critic paradigm. In the setting considered\r\nherein,
each agent observes a part of the global state space in order to take\r\nlocal
actions, for which it receives local rewards. For every agent, DDPG\r\ntrains
a local actor (policy) and a local critic (Q-function). The analysis\r\nshows
that multi-agent DDPG using neural networks to approximate the local\r\npolicies
and critics converge to limits with the following properties: The\r\ncritic limits
minimize the average squared Bellman loss; the actor limits\r\nparameterize a
policy that maximizes the local critic's approximation of\r\n$Q_i^*$, where $i$
is the agent index. The averaging is with respect to a\r\nprobability distribution
over the global state-action space. It captures the\r\nasymptotics of all local
training processes. Finally, we extend the analysis to\r\na fully decentralized
setting where agents communicate over a wireless network\r\nprone to delays and
losses; a typical scenario in, e.g., robotic applications."
author:
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: Redder A, Ramaswamy A, Karl H. Asymptotic Convergence of Deep Multi-Agent Actor-Critic
Algorithms. arXiv:220100570. Published online 2022.
apa: Redder, A., Ramaswamy, A., & Karl, H. (2022). Asymptotic Convergence of
Deep Multi-Agent Actor-Critic Algorithms. In arXiv:2201.00570.
bibtex: '@article{Redder_Ramaswamy_Karl_2022, title={Asymptotic Convergence of Deep
Multi-Agent Actor-Critic Algorithms}, journal={arXiv:2201.00570}, author={Redder,
Adrian and Ramaswamy, Arunselvan and Karl, Holger}, year={2022} }'
chicago: Redder, Adrian, Arunselvan Ramaswamy, and Holger Karl. “Asymptotic Convergence
of Deep Multi-Agent Actor-Critic Algorithms.” ArXiv:2201.00570, 2022.
ieee: A. Redder, A. Ramaswamy, and H. Karl, “Asymptotic Convergence of Deep Multi-Agent
Actor-Critic Algorithms,” arXiv:2201.00570. 2022.
mla: Redder, Adrian, et al. “Asymptotic Convergence of Deep Multi-Agent Actor-Critic
Algorithms.” ArXiv:2201.00570, 2022.
short: A. Redder, A. Ramaswamy, H. Karl, ArXiv:2201.00570 (2022).
date_created: 2022-04-06T06:53:52Z
date_updated: 2022-11-18T09:33:42Z
department:
- _id: '75'
external_id:
arxiv:
- '2201.00570'
language:
- iso: eng
project:
- _id: '16'
name: 'SFB 901 - C4: SFB 901 - Subproject C4'
- _id: '1'
name: 'SFB 901: SFB 901'
- _id: '4'
name: 'SFB 901 - C: SFB 901 - Project Area C'
publication: arXiv:2201.00570
status: public
title: Asymptotic Convergence of Deep Multi-Agent Actor-Critic Algorithms
type: preprint
user_id: '477'
year: '2022'
...
---
_id: '32854'
author:
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: Redder A, Ramaswamy A, Karl H. Practical Network Conditions for the Convergence
of Distributed Optimization. IFAC-PapersOnLine. 2022;55(13):133–138.
apa: Redder, A., Ramaswamy, A., & Karl, H. (2022). Practical Network Conditions
for the Convergence of Distributed Optimization. IFAC-PapersOnLine, 55(13),
133–138.
bibtex: '@article{Redder_Ramaswamy_Karl_2022, title={Practical Network Conditions
for the Convergence of Distributed Optimization}, volume={55}, number={13}, journal={IFAC-PapersOnLine},
publisher={Elsevier}, author={Redder, Adrian and Ramaswamy, Arunselvan and Karl,
Holger}, year={2022}, pages={133–138} }'
chicago: 'Redder, Adrian, Arunselvan Ramaswamy, and Holger Karl. “Practical Network
Conditions for the Convergence of Distributed Optimization.” IFAC-PapersOnLine
55, no. 13 (2022): 133–138.'
ieee: A. Redder, A. Ramaswamy, and H. Karl, “Practical Network Conditions for the
Convergence of Distributed Optimization,” IFAC-PapersOnLine, vol. 55, no.
13, pp. 133–138, 2022.
mla: Redder, Adrian, et al. “Practical Network Conditions for the Convergence of
Distributed Optimization.” IFAC-PapersOnLine, vol. 55, no. 13, Elsevier,
2022, pp. 133–138.
short: A. Redder, A. Ramaswamy, H. Karl, IFAC-PapersOnLine 55 (2022) 133–138.
conference:
name: IFAC Conference on Networked Systems
date_created: 2022-08-16T09:12:55Z
date_updated: 2022-11-18T10:05:14Z
ddc:
- '006'
department:
- _id: '75'
file:
- access_level: closed
content_type: application/pdf
creator: aredder
date_created: 2022-08-31T07:06:30Z
date_updated: 2022-08-31T07:06:30Z
file_id: '33236'
file_name: NecSys2022____Practical_Conditions_for_Conv.pdf
file_size: 298395
relation: main_file
success: 1
file_date_updated: 2022-08-31T07:06:30Z
has_accepted_license: '1'
intvolume: ' 55'
issue: '13'
language:
- iso: eng
page: 133–138
project:
- _id: '16'
name: 'SFB 901 - C4: SFB 901 - Subproject C4'
- _id: '1'
name: 'SFB 901: SFB 901'
- _id: '4'
name: 'SFB 901 - C: SFB 901 - Project Area C'
publication: IFAC-PapersOnLine
publisher: Elsevier
status: public
title: Practical Network Conditions for the Convergence of Distributed Optimization
type: journal_article
user_id: '477'
volume: 55
year: '2022'
...
---
_id: '24142'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Shalabh
full_name: Bhatnagar, Shalabh
last_name: Bhatnagar
citation:
ama: Ramaswamy A, Bhatnagar S. Analyzing Approximate Value Iteration Algorithms.
Mathematics of Operations Research (to appear). Published online 2021.
apa: Ramaswamy, A., & Bhatnagar, S. (2021). Analyzing Approximate Value Iteration
Algorithms. Mathematics of Operations Research (to Appear).
bibtex: '@article{Ramaswamy_Bhatnagar_2021, title={Analyzing Approximate Value Iteration
Algorithms}, journal={Mathematics of Operations Research (to appear)}, author={Ramaswamy,
Arunselvan and Bhatnagar, Shalabh}, year={2021} }'
chicago: Ramaswamy, Arunselvan, and Shalabh Bhatnagar. “Analyzing Approximate Value
Iteration Algorithms.” Mathematics of Operations Research (to Appear),
2021.
ieee: A. Ramaswamy and S. Bhatnagar, “Analyzing Approximate Value Iteration Algorithms,”
Mathematics of Operations Research (to appear), 2021.
mla: Ramaswamy, Arunselvan, and Shalabh Bhatnagar. “Analyzing Approximate Value
Iteration Algorithms.” Mathematics of Operations Research (to Appear),
2021.
short: A. Ramaswamy, S. Bhatnagar, Mathematics of Operations Research (to Appear)
(2021).
date_created: 2021-09-10T09:52:53Z
date_updated: 2022-01-06T06:56:08Z
language:
- iso: eng
publication: Mathematics of Operations Research (to appear)
status: public
title: Analyzing Approximate Value Iteration Algorithms
type: journal_article
user_id: '66937'
year: '2021'
...
---
_id: '24143'
author:
- first_name: Jan Peter
full_name: Drees, Jan Peter
last_name: Drees
- first_name: Pritha
full_name: Gupta, Pritha
id: '54803'
last_name: Gupta
- first_name: Eyke
full_name: Hüllermeier, Eyke
id: '48129'
last_name: Hüllermeier
- first_name: Tibor
full_name: Jager, Tibor
last_name: Jager
- first_name: Alexander
full_name: Konze, Alexander
last_name: Konze
- first_name: Claudia
full_name: Priesterjahn, Claudia
last_name: Priesterjahn
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Juraj
full_name: Somorovsky, Juraj
id: '83504'
last_name: Somorovsky
orcid: 0000-0002-3593-7720
citation:
ama: 'Drees JP, Gupta P, Hüllermeier E, et al. Automated Detection of Side Channels
in Cryptographic Protocols: DROWN the ROBOTs! 14th ACM Workshop on Artificial
Intelligence and Security. Published online 2021.'
apa: 'Drees, J. P., Gupta, P., Hüllermeier, E., Jager, T., Konze, A., Priesterjahn,
C., Ramaswamy, A., & Somorovsky, J. (2021). Automated Detection of Side Channels
in Cryptographic Protocols: DROWN the ROBOTs! 14th ACM Workshop on Artificial
Intelligence and Security.'
bibtex: '@article{Drees_Gupta_Hüllermeier_Jager_Konze_Priesterjahn_Ramaswamy_Somorovsky_2021,
title={Automated Detection of Side Channels in Cryptographic Protocols: DROWN
the ROBOTs!}, journal={14th ACM Workshop on Artificial Intelligence and Security},
author={Drees, Jan Peter and Gupta, Pritha and Hüllermeier, Eyke and Jager, Tibor
and Konze, Alexander and Priesterjahn, Claudia and Ramaswamy, Arunselvan and Somorovsky,
Juraj}, year={2021} }'
chicago: 'Drees, Jan Peter, Pritha Gupta, Eyke Hüllermeier, Tibor Jager, Alexander
Konze, Claudia Priesterjahn, Arunselvan Ramaswamy, and Juraj Somorovsky. “Automated
Detection of Side Channels in Cryptographic Protocols: DROWN the ROBOTs!” 14th
ACM Workshop on Artificial Intelligence and Security, 2021.'
ieee: 'J. P. Drees et al., “Automated Detection of Side Channels in Cryptographic
Protocols: DROWN the ROBOTs!,” 14th ACM Workshop on Artificial Intelligence
and Security, 2021.'
mla: 'Drees, Jan Peter, et al. “Automated Detection of Side Channels in Cryptographic
Protocols: DROWN the ROBOTs!” 14th ACM Workshop on Artificial Intelligence
and Security, 2021.'
short: J.P. Drees, P. Gupta, E. Hüllermeier, T. Jager, A. Konze, C. Priesterjahn,
A. Ramaswamy, J. Somorovsky, 14th ACM Workshop on Artificial Intelligence and
Security (2021).
date_created: 2021-09-10T09:56:27Z
date_updated: 2022-01-06T06:56:08Z
department:
- _id: '632'
language:
- iso: eng
publication: 14th ACM Workshop on Artificial Intelligence and Security
status: public
title: 'Automated Detection of Side Channels in Cryptographic Protocols: DROWN the
ROBOTs!'
type: journal_article
user_id: '83504'
year: '2021'
...
---
_id: '24148'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Eyke
full_name: Hüllermeier, Eyke
id: '48129'
last_name: Hüllermeier
citation:
ama: 'Ramaswamy A, Hüllermeier E. Deep Q-Learning: Theoretical Insights from an
Asymptotic Analysis. IEEE Transactions on Artificial Intelligence (to appear).
Published online 2021.'
apa: 'Ramaswamy, A., & Hüllermeier, E. (2021). Deep Q-Learning: Theoretical
Insights from an Asymptotic Analysis. IEEE Transactions on Artificial Intelligence
(to Appear).'
bibtex: '@article{Ramaswamy_Hüllermeier_2021, title={Deep Q-Learning: Theoretical
Insights from an Asymptotic Analysis}, journal={IEEE Transactions on Artificial
Intelligence (to appear)}, author={Ramaswamy, Arunselvan and Hüllermeier, Eyke},
year={2021} }'
chicago: 'Ramaswamy, Arunselvan, and Eyke Hüllermeier. “Deep Q-Learning: Theoretical
Insights from an Asymptotic Analysis.” IEEE Transactions on Artificial Intelligence
(to Appear), 2021.'
ieee: 'A. Ramaswamy and E. Hüllermeier, “Deep Q-Learning: Theoretical Insights from
an Asymptotic Analysis,” IEEE Transactions on Artificial Intelligence (to appear),
2021.'
mla: 'Ramaswamy, Arunselvan, and Eyke Hüllermeier. “Deep Q-Learning: Theoretical
Insights from an Asymptotic Analysis.” IEEE Transactions on Artificial Intelligence
(to Appear), 2021.'
short: A. Ramaswamy, E. Hüllermeier, IEEE Transactions on Artificial Intelligence
(to Appear) (2021).
date_created: 2021-09-10T10:03:25Z
date_updated: 2022-01-06T06:56:08Z
language:
- iso: eng
publication: IEEE Transactions on Artificial Intelligence (to appear)
status: public
title: 'Deep Q-Learning: Theoretical Insights from an Asymptotic Analysis'
type: journal_article
user_id: '66937'
year: '2021'
...
---
_id: '21478'
abstract:
- lang: eng
text: 'In this work we use autonomous vehicles to improve the performance of Wireless
Sensor Networks (WSNs). In contrast to other autonomous vehicle applications,
WSNs have two metrics for performance evaluation. First, quality of information
(QoI) which is used to measure the quality of sensed data (e.g., measurement uncertainties
or signal strength). Second, quality of service (QoS) which is used to measure
the network''s performance for data forwarding (e.g., delay and packet losses).
As a use case, we consider wireless acoustic sensor networks, where a group of
speakers move inside a room and there are autonomous vehicles installed with microphones
for streaming the audio data. We formulate the problem as a Markov decision problem
(MDP) and solve it using Deep-Q-Networks (DQN). Additionally, we compare the performance
of DQN solution to two different real-world implementations: speakers holding/passing
microphones and microphones being preinstalled in fixed positions. We show that
the performance of autonomous vehicles in terms of QoI and QoS is better than
the real-world implementation in some scenarios. Moreover, we study the impact
of the vehicles speed on the learning process of the DQN solution and show how
low speeds degrade the performance. Finally, we compare the DQN solution to a
heuristic one and provide theoretical analysis of the performance with respect
to dynamic WSNs.'
author:
- first_name: Haitham
full_name: Afifi, Haitham
id: '65718'
last_name: Afifi
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: 'Afifi H, Ramaswamy A, Karl H. Reinforcement Learning for Autonomous Vehicle
Movements in Wireless Sensor Networks. In: 2021 IEEE International Conference
on Communications (ICC): IoT and Sensor Networks Symposium (IEEE ICC’21 - IoTSN
Symposium). Montreal, Canada; 2021.'
apa: 'Afifi, H., Ramaswamy, A., & Karl, H. (2021). Reinforcement Learning for
Autonomous Vehicle Movements in Wireless Sensor Networks. In 2021 IEEE International
Conference on Communications (ICC): IoT and Sensor Networks Symposium (IEEE ICC’21
- IoTSN Symposium). Montreal, Canada.'
bibtex: '@inproceedings{Afifi_Ramaswamy_Karl_2021, place={Montreal, Canada}, title={Reinforcement
Learning for Autonomous Vehicle Movements in Wireless Sensor Networks}, booktitle={2021
IEEE International Conference on Communications (ICC): IoT and Sensor Networks
Symposium (IEEE ICC’21 - IoTSN Symposium)}, author={Afifi, Haitham and Ramaswamy,
Arunselvan and Karl, Holger}, year={2021} }'
chicago: 'Afifi, Haitham, Arunselvan Ramaswamy, and Holger Karl. “Reinforcement
Learning for Autonomous Vehicle Movements in Wireless Sensor Networks.” In 2021
IEEE International Conference on Communications (ICC): IoT and Sensor Networks
Symposium (IEEE ICC’21 - IoTSN Symposium). Montreal, Canada, 2021.'
ieee: 'H. Afifi, A. Ramaswamy, and H. Karl, “Reinforcement Learning for Autonomous
Vehicle Movements in Wireless Sensor Networks,” in 2021 IEEE International
Conference on Communications (ICC): IoT and Sensor Networks Symposium (IEEE ICC’21
- IoTSN Symposium), 2021.'
mla: 'Afifi, Haitham, et al. “Reinforcement Learning for Autonomous Vehicle Movements
in Wireless Sensor Networks.” 2021 IEEE International Conference on Communications
(ICC): IoT and Sensor Networks Symposium (IEEE ICC’21 - IoTSN Symposium),
2021.'
short: 'H. Afifi, A. Ramaswamy, H. Karl, in: 2021 IEEE International Conference
on Communications (ICC): IoT and Sensor Networks Symposium (IEEE ICC’21 - IoTSN
Symposium), Montreal, Canada, 2021.'
date_created: 2021-03-12T16:02:04Z
date_updated: 2022-01-06T06:55:00Z
language:
- iso: eng
place: Montreal, Canada
project:
- _id: '27'
name: Akustische Sensornetzwerke - Teilprojekt "Verteilte akustische Signalverarbeitung
über funkbasierte Sensornetzwerke
publication: '2021 IEEE International Conference on Communications (ICC): IoT and
Sensor Networks Symposium (IEEE ICC''21 - IoTSN Symposium)'
status: public
title: Reinforcement Learning for Autonomous Vehicle Movements in Wireless Sensor
Networks
type: conference
user_id: '65718'
year: '2021'
...
---
_id: '21479'
abstract:
- lang: eng
text: Two of the most important metrics when developing Wireless Sensor Networks
(WSNs) applications are the Quality of Information (QoI) and Quality of Service
(QoS). The former is used to specify the quality of the collected data by the
sensors (e.g., measurements error or signal's intensity), while the latter defines
the network's performance and availability (e.g., packet losses and latency).
In this paper, we consider an example of wireless acoustic sensor networks, where
we select a subset of microphones for two different objectives. First, we maximize
the recording quality under QoS constraints. Second, we apply a trade-off between
QoI and QoS. We formulate the problem as a constrained Markov Decision Problem
(MDP) and solve it using reinforcement learning (RL). We compare the RL solution
to a baseline model and show that in case of QoS-guarantee objective, the RL solution
has an optimality gap up to 1\%. Meanwhile, the RL solution is better than the
baseline with improvements up to 23\%, when using the trade-off objective.
author:
- first_name: Haitham
full_name: Afifi, Haitham
id: '65718'
last_name: Afifi
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
citation:
ama: 'Afifi H, Ramaswamy A, Karl H. A Reinforcement Learning QoI/QoS-Aware Approach
in Acoustic Sensor Networks. In: 2021 IEEE 18th Annual Consumer Communications
\& Networking Conference (CCNC) (CCNC 2021). ; 2021.'
apa: Afifi, H., Ramaswamy, A., & Karl, H. (2021). A Reinforcement Learning QoI/QoS-Aware
Approach in Acoustic Sensor Networks. In 2021 IEEE 18th Annual Consumer Communications
\& Networking Conference (CCNC) (CCNC 2021).
bibtex: '@inproceedings{Afifi_Ramaswamy_Karl_2021, title={A Reinforcement Learning
QoI/QoS-Aware Approach in Acoustic Sensor Networks}, booktitle={2021 IEEE 18th
Annual Consumer Communications \& Networking Conference (CCNC) (CCNC 2021)},
author={Afifi, Haitham and Ramaswamy, Arunselvan and Karl, Holger}, year={2021}
}'
chicago: Afifi, Haitham, Arunselvan Ramaswamy, and Holger Karl. “A Reinforcement
Learning QoI/QoS-Aware Approach in Acoustic Sensor Networks.” In 2021 IEEE
18th Annual Consumer Communications \& Networking Conference (CCNC) (CCNC
2021), 2021.
ieee: H. Afifi, A. Ramaswamy, and H. Karl, “A Reinforcement Learning QoI/QoS-Aware
Approach in Acoustic Sensor Networks,” in 2021 IEEE 18th Annual Consumer Communications
\& Networking Conference (CCNC) (CCNC 2021), 2021.
mla: Afifi, Haitham, et al. “A Reinforcement Learning QoI/QoS-Aware Approach in
Acoustic Sensor Networks.” 2021 IEEE 18th Annual Consumer Communications \&
Networking Conference (CCNC) (CCNC 2021), 2021.
short: 'H. Afifi, A. Ramaswamy, H. Karl, in: 2021 IEEE 18th Annual Consumer Communications
\& Networking Conference (CCNC) (CCNC 2021), 2021.'
date_created: 2021-03-12T16:03:53Z
date_updated: 2022-01-06T06:55:00Z
keyword:
- reinforcement learning
- wireless sensor networks
- resource allocation
- acoustic sensor networks
language:
- iso: eng
project:
- _id: '27'
name: Akustische Sensornetzwerke - Teilprojekt "Verteilte akustische Signalverarbeitung
über funkbasierte Sensornetzwerke
publication: 2021 IEEE 18th Annual Consumer Communications \& Networking Conference
(CCNC) (CCNC 2021)
status: public
title: A Reinforcement Learning QoI/QoS-Aware Approach in Acoustic Sensor Networks
type: conference
user_id: '65718'
year: '2021'
...
---
_id: '24140'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Daniel E.
full_name: Quevedo, Daniel E.
last_name: Quevedo
citation:
ama: Ramaswamy A, Redder A, Quevedo DE. Distributed optimization over time-varying
networks with stochastic information delays. IEEE Transactions on Automatic
Control. Published online 2021:1-1. doi:10.1109/TAC.2021.3108492
apa: Ramaswamy, A., Redder, A., & Quevedo, D. E. (2021). Distributed optimization
over time-varying networks with stochastic information delays. IEEE Transactions
on Automatic Control, 1–1. https://doi.org/10.1109/TAC.2021.3108492
bibtex: '@article{Ramaswamy_Redder_Quevedo_2021, title={Distributed optimization
over time-varying networks with stochastic information delays}, DOI={10.1109/TAC.2021.3108492},
journal={IEEE Transactions on Automatic Control}, author={Ramaswamy, Arunselvan
and Redder, Adrian and Quevedo, Daniel E.}, year={2021}, pages={1–1} }'
chicago: Ramaswamy, Arunselvan, Adrian Redder, and Daniel E. Quevedo. “Distributed
Optimization over Time-Varying Networks with Stochastic Information Delays.” IEEE
Transactions on Automatic Control, 2021, 1–1. https://doi.org/10.1109/TAC.2021.3108492.
ieee: 'A. Ramaswamy, A. Redder, and D. E. Quevedo, “Distributed optimization over
time-varying networks with stochastic information delays,” IEEE Transactions
on Automatic Control, pp. 1–1, 2021, doi: 10.1109/TAC.2021.3108492.'
mla: Ramaswamy, Arunselvan, et al. “Distributed Optimization over Time-Varying Networks
with Stochastic Information Delays.” IEEE Transactions on Automatic Control,
2021, pp. 1–1, doi:10.1109/TAC.2021.3108492.
short: A. Ramaswamy, A. Redder, D.E. Quevedo, IEEE Transactions on Automatic Control
(2021) 1–1.
date_created: 2021-09-10T09:48:55Z
date_updated: 2022-08-16T09:16:43Z
doi: 10.1109/TAC.2021.3108492
language:
- iso: eng
page: 1-1
project:
- _id: '24'
name: 'NICCI-CN: Netzgewahre Regelung & regelungsgewahre Netze'
publication: IEEE Transactions on Automatic Control
status: public
title: Distributed optimization over time-varying networks with stochastic information
delays
type: journal_article
user_id: '52265'
year: '2021'
...
---
_id: '24145'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
citation:
ama: 'Ramaswamy A. DSPG: Decentralized Simultaneous Perturbations Gradient Descent
Scheme. In: 2020 28th Euromicro International Conference on Parallel, Distributed
and Network-Based Processing (PDP). ; 2020:54-62.'
apa: 'Ramaswamy, A. (2020). DSPG: Decentralized Simultaneous Perturbations Gradient
Descent Scheme. 2020 28th Euromicro International Conference on Parallel, Distributed
and Network-Based Processing (PDP), 54–62.'
bibtex: '@inproceedings{Ramaswamy_2020, title={DSPG: Decentralized Simultaneous
Perturbations Gradient Descent Scheme}, booktitle={2020 28th Euromicro International
Conference on Parallel, Distributed and Network-Based Processing (PDP)}, author={Ramaswamy,
Arunselvan}, year={2020}, pages={54–62} }'
chicago: 'Ramaswamy, Arunselvan. “DSPG: Decentralized Simultaneous Perturbations
Gradient Descent Scheme.” In 2020 28th Euromicro International Conference on
Parallel, Distributed and Network-Based Processing (PDP), 54–62, 2020.'
ieee: 'A. Ramaswamy, “DSPG: Decentralized Simultaneous Perturbations Gradient Descent
Scheme,” in 2020 28th Euromicro International Conference on Parallel, Distributed
and Network-Based Processing (PDP), 2020, pp. 54–62.'
mla: 'Ramaswamy, Arunselvan. “DSPG: Decentralized Simultaneous Perturbations Gradient
Descent Scheme.” 2020 28th Euromicro International Conference on Parallel,
Distributed and Network-Based Processing (PDP), 2020, pp. 54–62.'
short: 'A. Ramaswamy, in: 2020 28th Euromicro International Conference on Parallel,
Distributed and Network-Based Processing (PDP), 2020, pp. 54–62.'
date_created: 2021-09-10T09:58:09Z
date_updated: 2022-01-06T06:56:08Z
language:
- iso: eng
page: 54-62
publication: 2020 28th Euromicro International Conference on Parallel, Distributed
and Network-Based Processing (PDP)
status: public
title: 'DSPG: Decentralized Simultaneous Perturbations Gradient Descent Scheme'
type: conference
user_id: '66937'
year: '2020'
...
---
_id: '24146'
author:
- first_name: Stefan Helmut
full_name: Heid, Stefan Helmut
id: '39640'
last_name: Heid
orcid: 0000-0002-9461-7372
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Eyke
full_name: Hüllermeier, Eyke
id: '48129'
last_name: Hüllermeier
citation:
ama: 'Heid SH, Ramaswamy A, Hüllermeier E. Constrained Multi-Agent Optimization
with Unbounded Information Delay. In: Proceedings-30. Workshop Computational
Intelligence: Berlin, 26.-27. November 2020. Vol 26. ; 2020:247.'
apa: 'Heid, S. H., Ramaswamy, A., & Hüllermeier, E. (2020). Constrained Multi-Agent
Optimization with Unbounded Information Delay. Proceedings-30. Workshop Computational
Intelligence: Berlin, 26.-27. November 2020, 26, 247.'
bibtex: '@inproceedings{Heid_Ramaswamy_Hüllermeier_2020, title={Constrained Multi-Agent
Optimization with Unbounded Information Delay}, volume={26}, booktitle={Proceedings-30.
Workshop Computational Intelligence: Berlin, 26.-27. November 2020}, author={Heid,
Stefan Helmut and Ramaswamy, Arunselvan and Hüllermeier, Eyke}, year={2020}, pages={247}
}'
chicago: 'Heid, Stefan Helmut, Arunselvan Ramaswamy, and Eyke Hüllermeier. “Constrained
Multi-Agent Optimization with Unbounded Information Delay.” In Proceedings-30.
Workshop Computational Intelligence: Berlin, 26.-27. November 2020, 26:247,
2020.'
ieee: 'S. H. Heid, A. Ramaswamy, and E. Hüllermeier, “Constrained Multi-Agent Optimization
with Unbounded Information Delay,” in Proceedings-30. Workshop Computational
Intelligence: Berlin, 26.-27. November 2020, 2020, vol. 26, p. 247.'
mla: 'Heid, Stefan Helmut, et al. “Constrained Multi-Agent Optimization with Unbounded
Information Delay.” Proceedings-30. Workshop Computational Intelligence: Berlin,
26.-27. November 2020, vol. 26, 2020, p. 247.'
short: 'S.H. Heid, A. Ramaswamy, E. Hüllermeier, in: Proceedings-30. Workshop Computational
Intelligence: Berlin, 26.-27. November 2020, 2020, p. 247.'
date_created: 2021-09-10T09:59:16Z
date_updated: 2022-01-06T06:56:08Z
intvolume: ' 26'
language:
- iso: eng
page: '247'
publication: 'Proceedings-30. Workshop Computational Intelligence: Berlin, 26.-27.
November 2020'
status: public
title: Constrained Multi-Agent Optimization with Unbounded Information Delay
type: conference
user_id: '66937'
volume: 26
year: '2020'
...
---
_id: '24147'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Shalabh
full_name: Bhatnagar, Shalabh
last_name: Bhatnagar
- first_name: Daniel E
full_name: Quevedo, Daniel E
last_name: Quevedo
citation:
ama: Ramaswamy A, Bhatnagar S, Quevedo DE. Asynchronous stochastic approximations
with asymptotically biased errors and deep multi-agent learning. IEEE Transactions
on Automatic Control. Published online 2020.
apa: Ramaswamy, A., Bhatnagar, S., & Quevedo, D. E. (2020). Asynchronous stochastic
approximations with asymptotically biased errors and deep multi-agent learning.
IEEE Transactions on Automatic Control.
bibtex: '@article{Ramaswamy_Bhatnagar_Quevedo_2020, title={Asynchronous stochastic
approximations with asymptotically biased errors and deep multi-agent learning},
journal={IEEE Transactions on Automatic Control}, publisher={IEEE}, author={Ramaswamy,
Arunselvan and Bhatnagar, Shalabh and Quevedo, Daniel E}, year={2020} }'
chicago: Ramaswamy, Arunselvan, Shalabh Bhatnagar, and Daniel E Quevedo. “Asynchronous
Stochastic Approximations with Asymptotically Biased Errors and Deep Multi-Agent
Learning.” IEEE Transactions on Automatic Control, 2020.
ieee: A. Ramaswamy, S. Bhatnagar, and D. E. Quevedo, “Asynchronous stochastic approximations
with asymptotically biased errors and deep multi-agent learning,” IEEE Transactions
on Automatic Control, 2020.
mla: Ramaswamy, Arunselvan, et al. “Asynchronous Stochastic Approximations with
Asymptotically Biased Errors and Deep Multi-Agent Learning.” IEEE Transactions
on Automatic Control, IEEE, 2020.
short: A. Ramaswamy, S. Bhatnagar, D.E. Quevedo, IEEE Transactions on Automatic
Control (2020).
date_created: 2021-09-10T10:01:18Z
date_updated: 2022-01-06T06:56:08Z
language:
- iso: eng
publication: IEEE Transactions on Automatic Control
publisher: IEEE
status: public
title: Asynchronous stochastic approximations with asymptotically biased errors and
deep multi-agent learning
type: journal_article
user_id: '66937'
year: '2020'
...
---
_id: '24141'
author:
- first_name: Jonas
full_name: König, Jonas
last_name: König
- first_name: Simon
full_name: Malberg, Simon
last_name: Malberg
- first_name: Martin
full_name: Martens, Martin
last_name: Martens
- first_name: Sebastian
full_name: Niehaus, Sebastian
last_name: Niehaus
- first_name: Artus
full_name: Krohn-Grimberghe, Artus
last_name: Krohn-Grimberghe
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
citation:
ama: 'König J, Malberg S, Martens M, Niehaus S, Krohn-Grimberghe A, Ramaswamy A.
Multi-stage reinforcement learning for object detection. In: Science and Information
Conference. ; 2019:178-191.'
apa: König, J., Malberg, S., Martens, M., Niehaus, S., Krohn-Grimberghe, A., &
Ramaswamy, A. (2019). Multi-stage reinforcement learning for object detection.
Science and Information Conference, 178–191.
bibtex: '@inproceedings{König_Malberg_Martens_Niehaus_Krohn-Grimberghe_Ramaswamy_2019,
title={Multi-stage reinforcement learning for object detection}, booktitle={Science
and Information Conference}, author={König, Jonas and Malberg, Simon and Martens,
Martin and Niehaus, Sebastian and Krohn-Grimberghe, Artus and Ramaswamy, Arunselvan},
year={2019}, pages={178–191} }'
chicago: König, Jonas, Simon Malberg, Martin Martens, Sebastian Niehaus, Artus Krohn-Grimberghe,
and Arunselvan Ramaswamy. “Multi-Stage Reinforcement Learning for Object Detection.”
In Science and Information Conference, 178–91, 2019.
ieee: J. König, S. Malberg, M. Martens, S. Niehaus, A. Krohn-Grimberghe, and A.
Ramaswamy, “Multi-stage reinforcement learning for object detection,” in Science
and Information Conference, 2019, pp. 178–191.
mla: König, Jonas, et al. “Multi-Stage Reinforcement Learning for Object Detection.”
Science and Information Conference, 2019, pp. 178–91.
short: 'J. König, S. Malberg, M. Martens, S. Niehaus, A. Krohn-Grimberghe, A. Ramaswamy,
in: Science and Information Conference, 2019, pp. 178–191.'
date_created: 2021-09-10T09:50:53Z
date_updated: 2022-01-06T06:56:08Z
language:
- iso: eng
page: 178-191
publication: Science and Information Conference
status: public
title: Multi-stage reinforcement learning for object detection
type: conference
user_id: '66937'
year: '2019'
...
---
_id: '15741'
abstract:
- lang: eng
text: "\r\nIn many cyber–physical systems, we encounter the problem of remote state
estimation of geo- graphically distributed and remote physical processes. This
paper studies the scheduling of sensor transmissions to estimate the states of
multiple remote, dynamic processes. Information from the different sensors has
to be transmitted to a central gateway over a wireless network for monitoring
purposes, where typically fewer wireless channels are available than there are
processes to be monitored. For effective estimation at the gateway, the sensors
need to be scheduled appropriately, i.e., at each time instant one needs to decide
which sensors have network access and which ones do not. To address this scheduling
problem, we formulate an associated Markov decision process (MDP). This MDP is
then solved using a Deep Q-Network, a recent deep reinforcement learning algorithm
that is at once scalable and model-free. We compare our scheduling algorithm to
popular scheduling algorithms such as round-robin and reduced-waiting-time, among
others. Our algorithm is shown to significantly outperform these algorithms for
many example scenario"
article_number: '108759'
author:
- first_name: Alex S.
full_name: Leong, Alex S.
last_name: Leong
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Daniel E.
full_name: Quevedo, Daniel E.
last_name: Quevedo
- first_name: Holger
full_name: Karl, Holger
id: '126'
last_name: Karl
- first_name: Ling
full_name: Shi, Ling
last_name: Shi
citation:
ama: Leong AS, Ramaswamy A, Quevedo DE, Karl H, Shi L. Deep reinforcement learning
for wireless sensor scheduling in cyber–physical systems. Automatica. 2019.
doi:10.1016/j.automatica.2019.108759
apa: Leong, A. S., Ramaswamy, A., Quevedo, D. E., Karl, H., & Shi, L. (2019).
Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems.
Automatica. https://doi.org/10.1016/j.automatica.2019.108759
bibtex: '@article{Leong_Ramaswamy_Quevedo_Karl_Shi_2019, title={Deep reinforcement
learning for wireless sensor scheduling in cyber–physical systems}, DOI={10.1016/j.automatica.2019.108759},
number={108759}, journal={Automatica}, author={Leong, Alex S. and Ramaswamy, Arunselvan
and Quevedo, Daniel E. and Karl, Holger and Shi, Ling}, year={2019} }'
chicago: Leong, Alex S., Arunselvan Ramaswamy, Daniel E. Quevedo, Holger Karl, and
Ling Shi. “Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber–Physical
Systems.” Automatica, 2019. https://doi.org/10.1016/j.automatica.2019.108759.
ieee: A. S. Leong, A. Ramaswamy, D. E. Quevedo, H. Karl, and L. Shi, “Deep reinforcement
learning for wireless sensor scheduling in cyber–physical systems,” Automatica,
2019.
mla: Leong, Alex S., et al. “Deep Reinforcement Learning for Wireless Sensor Scheduling
in Cyber–Physical Systems.” Automatica, 108759, 2019, doi:10.1016/j.automatica.2019.108759.
short: A.S. Leong, A. Ramaswamy, D.E. Quevedo, H. Karl, L. Shi, Automatica (2019).
date_created: 2020-01-31T15:55:27Z
date_updated: 2022-01-06T06:52:32Z
ddc:
- '000'
department:
- _id: '7'
- _id: '34'
- _id: '3'
- _id: '75'
- _id: '57'
doi: 10.1016/j.automatica.2019.108759
file:
- access_level: closed
content_type: application/pdf
creator: hkarl
date_created: 2020-01-31T15:57:50Z
date_updated: 2020-01-31T15:57:50Z
file_id: '15743'
file_name: leoram20a.pdf
file_size: '675382'
relation: main_file
success: 1
file_date_updated: 2020-01-31T15:57:50Z
has_accepted_license: '1'
language:
- iso: eng
project:
- _id: '24'
name: Netzgewahre Regelung & regelungsgewahre Netze
publication: Automatica
publication_identifier:
issn:
- 0005-1098
publication_status: published
quality_controlled: '1'
status: public
title: Deep reinforcement learning for wireless sensor scheduling in cyber–physical
systems
type: journal_article
user_id: '126'
year: '2019'
...
---
_id: '13443'
abstract:
- lang: eng
text: "This work considers the problem of control and resource allocation in networked\r\nsystems.
To this end, we present DIRA a Deep reinforcement learning based Iterative Resource\r\nAllocation
algorithm, which is scalable and control-aware. Our algorithm is tailored towards\r\nlarge-scale
problems where control and scheduling need to act jointly to optimize performance.\r\nDIRA
can be used to schedule general time-domain optimization based controllers. In
the present\r\nwork, we focus on control designs based on suitably adapted linear
quadratic regulators. We\r\napply our algorithm to networked systems with correlated
fading communication channels. Our\r\nsimulations show that DIRA scales well to
large scheduling problems."
author:
- first_name: Adrian
full_name: Redder, Adrian
id: '52265'
last_name: Redder
orcid: https://orcid.org/0000-0001-7391-4688
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Daniel
full_name: Quevedo, Daniel
last_name: Quevedo
citation:
ama: 'Redder A, Ramaswamy A, Quevedo D. Deep reinforcement learning for scheduling
in large-scale networked control systems. In: Proceedings of the 8th IFAC Workshop
on Distributed Estimation and Control in Networked Systems. ; 2019.'
apa: Redder, A., Ramaswamy, A., & Quevedo, D. (2019). Deep reinforcement learning
for scheduling in large-scale networked control systems. In Proceedings of
the 8th IFAC Workshop on Distributed Estimation and Control in Networked Systems.
Chicago, USA.
bibtex: '@inproceedings{Redder_Ramaswamy_Quevedo_2019, title={Deep reinforcement
learning for scheduling in large-scale networked control systems}, booktitle={Proceedings
of the 8th IFAC Workshop on Distributed Estimation and Control in Networked Systems},
author={Redder, Adrian and Ramaswamy, Arunselvan and Quevedo, Daniel}, year={2019}
}'
chicago: Redder, Adrian, Arunselvan Ramaswamy, and Daniel Quevedo. “Deep Reinforcement
Learning for Scheduling in Large-Scale Networked Control Systems.” In Proceedings
of the 8th IFAC Workshop on Distributed Estimation and Control in Networked Systems,
2019.
ieee: A. Redder, A. Ramaswamy, and D. Quevedo, “Deep reinforcement learning for
scheduling in large-scale networked control systems,” in Proceedings of the
8th IFAC Workshop on Distributed Estimation and Control in Networked Systems,
Chicago, USA, 2019.
mla: Redder, Adrian, et al. “Deep Reinforcement Learning for Scheduling in Large-Scale
Networked Control Systems.” Proceedings of the 8th IFAC Workshop on Distributed
Estimation and Control in Networked Systems, 2019.
short: 'A. Redder, A. Ramaswamy, D. Quevedo, in: Proceedings of the 8th IFAC Workshop
on Distributed Estimation and Control in Networked Systems, 2019.'
conference:
end_date: 2019-09-17
location: Chicago, USA
name: 8th IFAC Workshop on Distributed Estimation and Control in Networked Systems
- NECSYS 2019
start_date: 2019-09-16
date_created: 2019-09-23T16:00:58Z
date_updated: 2022-01-06T06:51:36Z
ddc:
- '620'
file:
- access_level: local
content_type: application/pdf
creator: aredder
date_created: 2019-09-23T15:48:33Z
date_updated: 2019-09-23T16:21:16Z
file_id: '13444'
file_name: ifacconf.pdf
file_size: 371429
relation: main_file
file_date_updated: 2019-09-23T16:21:16Z
has_accepted_license: '1'
keyword:
- Networked control systems
- deep reinforcement learning
- large-scale systems
- resource scheduling
- stochastic control
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1905.05992
oa: '1'
project:
- _id: '52'
name: Computing Resources Provided by the Paderborn Center for Parallel Computing
publication: Proceedings of the 8th IFAC Workshop on Distributed Estimation and Control
in Networked Systems
publication_status: published
status: public
title: Deep reinforcement learning for scheduling in large-scale networked control
systems
type: conference
user_id: '52265'
year: '2019'
...
---
_id: '24150'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Shalabh
full_name: Bhatnagar, Shalabh
last_name: Bhatnagar
citation:
ama: Ramaswamy A, Bhatnagar S. Stability of stochastic approximations with “controlled
markov” noise and temporal difference learning. IEEE Transactions on Automatic
Control. 2018;64(6):2614-2620.
apa: Ramaswamy, A., & Bhatnagar, S. (2018). Stability of stochastic approximations
with “controlled markov” noise and temporal difference learning. IEEE Transactions
on Automatic Control, 64(6), 2614–2620.
bibtex: '@article{Ramaswamy_Bhatnagar_2018, title={Stability of stochastic approximations
with “controlled markov” noise and temporal difference learning}, volume={64},
number={6}, journal={IEEE Transactions on Automatic Control}, publisher={IEEE},
author={Ramaswamy, Arunselvan and Bhatnagar, Shalabh}, year={2018}, pages={2614–2620}
}'
chicago: 'Ramaswamy, Arunselvan, and Shalabh Bhatnagar. “Stability of Stochastic
Approximations with ‘Controlled Markov’ Noise and Temporal Difference Learning.”
IEEE Transactions on Automatic Control 64, no. 6 (2018): 2614–20.'
ieee: A. Ramaswamy and S. Bhatnagar, “Stability of stochastic approximations with
‘controlled markov’ noise and temporal difference learning,” IEEE Transactions
on Automatic Control, vol. 64, no. 6, pp. 2614–2620, 2018.
mla: Ramaswamy, Arunselvan, and Shalabh Bhatnagar. “Stability of Stochastic Approximations
with ‘Controlled Markov’ Noise and Temporal Difference Learning.” IEEE Transactions
on Automatic Control, vol. 64, no. 6, IEEE, 2018, pp. 2614–20.
short: A. Ramaswamy, S. Bhatnagar, IEEE Transactions on Automatic Control 64 (2018)
2614–2620.
date_created: 2021-09-10T10:17:54Z
date_updated: 2022-01-06T06:56:08Z
department:
- _id: '355'
intvolume: ' 64'
issue: '6'
language:
- iso: eng
page: 2614-2620
publication: IEEE Transactions on Automatic Control
publisher: IEEE
status: public
title: Stability of stochastic approximations with “controlled markov” noise and temporal
difference learning
type: journal_article
user_id: '66937'
volume: 64
year: '2018'
...
---
_id: '24151'
author:
- first_name: Burak
full_name: Demirel, Burak
last_name: Demirel
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Daniel E
full_name: Quevedo, Daniel E
last_name: Quevedo
- first_name: Holger
full_name: Karl, Holger
last_name: Karl
citation:
ama: 'Demirel B, Ramaswamy A, Quevedo DE, Karl H. Deepcas: A deep reinforcement
learning algorithm for control-aware scheduling. IEEE Control Systems Letters.
2018;2(4):737-742.'
apa: 'Demirel, B., Ramaswamy, A., Quevedo, D. E., & Karl, H. (2018). Deepcas:
A deep reinforcement learning algorithm for control-aware scheduling. IEEE
Control Systems Letters, 2(4), 737–742.'
bibtex: '@article{Demirel_Ramaswamy_Quevedo_Karl_2018, title={Deepcas: A deep reinforcement
learning algorithm for control-aware scheduling}, volume={2}, number={4}, journal={IEEE
Control Systems Letters}, publisher={IEEE}, author={Demirel, Burak and Ramaswamy,
Arunselvan and Quevedo, Daniel E and Karl, Holger}, year={2018}, pages={737–742}
}'
chicago: 'Demirel, Burak, Arunselvan Ramaswamy, Daniel E Quevedo, and Holger Karl.
“Deepcas: A Deep Reinforcement Learning Algorithm for Control-Aware Scheduling.”
IEEE Control Systems Letters 2, no. 4 (2018): 737–42.'
ieee: 'B. Demirel, A. Ramaswamy, D. E. Quevedo, and H. Karl, “Deepcas: A deep reinforcement
learning algorithm for control-aware scheduling,” IEEE Control Systems Letters,
vol. 2, no. 4, pp. 737–742, 2018.'
mla: 'Demirel, Burak, et al. “Deepcas: A Deep Reinforcement Learning Algorithm for
Control-Aware Scheduling.” IEEE Control Systems Letters, vol. 2, no. 4,
IEEE, 2018, pp. 737–42.'
short: B. Demirel, A. Ramaswamy, D.E. Quevedo, H. Karl, IEEE Control Systems Letters
2 (2018) 737–742.
date_created: 2021-09-10T10:19:07Z
date_updated: 2022-01-06T06:56:08Z
department:
- _id: '355'
intvolume: ' 2'
issue: '4'
language:
- iso: eng
page: 737-742
publication: IEEE Control Systems Letters
publisher: IEEE
status: public
title: 'Deepcas: A deep reinforcement learning algorithm for control-aware scheduling'
type: journal_article
user_id: '66937'
volume: 2
year: '2018'
...
---
_id: '24152'
author:
- first_name: Arunselvan
full_name: Ramaswamy, Arunselvan
id: '66937'
last_name: Ramaswamy
orcid: https://orcid.org/ 0000-0001-7547-8111
- first_name: Shalabh
full_name: Bhatnagar, Shalabh
last_name: Bhatnagar
citation:
ama: Ramaswamy A, Bhatnagar S. Analysis of gradient descent methods with nondiminishing
bounded errors. IEEE Transactions on Automatic Control. 2017;63(5):1465-1471.
apa: Ramaswamy, A., & Bhatnagar, S. (2017). Analysis of gradient descent methods
with nondiminishing bounded errors. IEEE Transactions on Automatic Control,
63(5), 1465–1471.
bibtex: '@article{Ramaswamy_Bhatnagar_2017, title={Analysis of gradient descent
methods with nondiminishing bounded errors}, volume={63}, number={5}, journal={IEEE
Transactions on Automatic Control}, publisher={IEEE}, author={Ramaswamy, Arunselvan
and Bhatnagar, Shalabh}, year={2017}, pages={1465–1471} }'
chicago: 'Ramaswamy, Arunselvan, and Shalabh Bhatnagar. “Analysis of Gradient Descent
Methods with Nondiminishing Bounded Errors.” IEEE Transactions on Automatic
Control 63, no. 5 (2017): 1465–71.'
ieee: A. Ramaswamy and S. Bhatnagar, “Analysis of gradient descent methods with
nondiminishing bounded errors,” IEEE Transactions on Automatic Control,
vol. 63, no. 5, pp. 1465–1471, 2017.
mla: Ramaswamy, Arunselvan, and Shalabh Bhatnagar. “Analysis of Gradient Descent
Methods with Nondiminishing Bounded Errors.” IEEE Transactions on Automatic
Control, vol. 63, no. 5, IEEE, 2017, pp. 1465–71.
short: A. Ramaswamy, S. Bhatnagar, IEEE Transactions on Automatic Control 63 (2017)
1465–1471.
date_created: 2021-09-10T10:19:40Z
date_updated: 2022-01-06T06:56:08Z
department:
- _id: '355'
extern: '1'
intvolume: ' 63'
issue: '5'
language:
- iso: eng
page: 1465-1471
publication: IEEE Transactions on Automatic Control
publisher: IEEE
status: public
title: Analysis of gradient descent methods with nondiminishing bounded errors
type: journal_article
user_id: '66937'
volume: 63
year: '2017'
...