Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning

S.B. Schneider, R. Khalili, A. Manzoor, H. Qarawlus, R. Schellenberg, H. Karl, A. Hecker, Transactions on Network and Service Management (2021).

Download
OA 4.17 MB
Journal Article | English
Author
; ; ; ; ; ;
Department
Abstract
Modern services consist of interconnected components,e.g., microservices in a service mesh or machine learning functions in a pipeline. These services can scale and run across multiple network nodes on demand. To process incoming traffic, service components have to be instantiated and traffic assigned to these instances, taking capacities, changing demands, and Quality of Service (QoS) requirements into account. This challenge is usually solved with custom approaches designed by experts. While this typically works well for the considered scenario, the models often rely on unrealistic assumptions or on knowledge that is not available in practice (e.g., a priori knowledge). We propose DeepCoord, a novel deep reinforcement learning approach that learns how to best coordinate services and is geared towards realistic assumptions. It interacts with the network and relies on available, possibly delayed monitoring information. Rather than defining a complex model or an algorithm on how to achieve an objective, our model-free approach adapts to various objectives and traffic patterns. An agent is trained offline without expert knowledge and then applied online with minimal overhead. Compared to a state-of-the-art heuristic, DeepCoord significantly improves flow throughput (up to 76%) and overall network utility (more than 2x) on realworld network topologies and traffic traces. It also supports optimizing multiple, possibly competing objectives, learns to respect QoS requirements, generalizes to scenarios with unseen, stochastic traffic, and scales to large real-world networks. For reproducibility and reuse, our code is publicly available.
Publishing Year
Journal Title
Transactions on Network and Service Management
LibreCat-ID

Cite this

Schneider SB, Khalili R, Manzoor A, et al. Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning. Transactions on Network and Service Management. 2021. doi:10.1109/TNSM.2021.3076503
Schneider, S. B., Khalili, R., Manzoor, A., Qarawlus, H., Schellenberg, R., Karl, H., & Hecker, A. (2021). Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning. Transactions on Network and Service Management. https://doi.org/10.1109/TNSM.2021.3076503
@article{Schneider_Khalili_Manzoor_Qarawlus_Schellenberg_Karl_Hecker_2021, title={Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning}, DOI={10.1109/TNSM.2021.3076503}, journal={Transactions on Network and Service Management}, publisher={IEEE}, author={Schneider, Stefan Balthasar and Khalili, Ramin and Manzoor, Adnan and Qarawlus, Haydar and Schellenberg, Rafael and Karl, Holger and Hecker, Artur}, year={2021} }
Schneider, Stefan Balthasar, Ramin Khalili, Adnan Manzoor, Haydar Qarawlus, Rafael Schellenberg, Holger Karl, and Artur Hecker. “Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning.” Transactions on Network and Service Management, 2021. https://doi.org/10.1109/TNSM.2021.3076503.
S. B. Schneider et al., “Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning,” Transactions on Network and Service Management, 2021.
Schneider, Stefan Balthasar, et al. “Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning.” Transactions on Network and Service Management, IEEE, 2021, doi:10.1109/TNSM.2021.3076503.
Main File(s)
File Name
Description
Author version of the accepted paper
Access Level
OA Open Access
Last Uploaded
2021-04-27T08:01:26Z


Export

Marked Publications

Open Data LibreCat

Search this title in

Google Scholar