{"abstract":[{"text":"Modern services consist of interconnected components,e.g., microservices in a service mesh or machine learning functions in a pipeline. These services can scale and run across multiple network nodes on demand. To process incoming traffic, service components have to be instantiated and traffic assigned to these instances, taking capacities, changing demands, and Quality of Service (QoS) requirements into account. This challenge is usually solved with custom approaches designed by experts. While this typically works well for the considered scenario, the models often rely on unrealistic assumptions or on knowledge that is not available in practice (e.g., a priori knowledge).\r\n\r\nWe propose DeepCoord, a novel deep reinforcement learning approach that learns how to best coordinate services and is geared towards realistic assumptions. It interacts with the network and relies on available, possibly delayed monitoring information. Rather than defining a complex model or an algorithm on how to achieve an objective, our model-free approach adapts to various objectives and traffic patterns. An agent is trained offline without expert knowledge and then applied online with minimal overhead. Compared to a state-of-the-art heuristic, DeepCoord significantly improves flow throughput (up to 76%) and overall network utility (more than 2x) on realworld network topologies and traffic traces. It also supports optimizing multiple, possibly competing objectives, learns to respect QoS requirements, generalizes to scenarios with unseen, stochastic traffic, and scales to large real-world networks. For reproducibility and reuse, our code is publicly available.","lang":"eng"}],"title":"Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning","date_updated":"2022-01-06T06:55:15Z","file":[{"creator":"stschn","file_size":4172270,"relation":"main_file","description":"Author version of the accepted paper","date_updated":"2021-04-27T08:01:26Z","file_name":"ris-accepted-version.pdf","access_level":"open_access","file_id":"21809","content_type":"application/pdf","date_created":"2021-04-27T08:01:26Z"}],"department":[{"_id":"75"}],"has_accepted_license":"1","project":[{"_id":"1","name":"SFB 901"},{"name":"SFB 901 - Project Area C","_id":"4"},{"name":"SFB 901 - Subproject C4","_id":"16"}],"author":[{"first_name":"Stefan Balthasar","id":"35343","full_name":"Schneider, Stefan Balthasar","last_name":"Schneider","orcid":"0000-0001-8210-4011"},{"first_name":"Ramin","last_name":"Khalili","full_name":"Khalili, Ramin"},{"first_name":"Adnan","last_name":"Manzoor","full_name":"Manzoor, Adnan"},{"full_name":"Qarawlus, Haydar","last_name":"Qarawlus","first_name":"Haydar"},{"last_name":"Schellenberg","full_name":"Schellenberg, Rafael","first_name":"Rafael"},{"full_name":"Karl, Holger","last_name":"Karl","first_name":"Holger","id":"126"},{"last_name":"Hecker","full_name":"Hecker, Artur","first_name":"Artur"}],"file_date_updated":"2021-04-27T08:01:26Z","doi":"10.1109/TNSM.2021.3076503","date_created":"2021-04-27T08:04:16Z","publication":"Transactions on Network and Service Management","language":[{"iso":"eng"}],"ddc":["000"],"citation":{"mla":"Schneider, Stefan Balthasar, et al. “Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning.” <i>Transactions on Network and Service Management</i>, IEEE, 2021, doi:<a href=\"https://doi.org/10.1109/TNSM.2021.3076503\">10.1109/TNSM.2021.3076503</a>.","chicago":"Schneider, Stefan Balthasar, Ramin Khalili, Adnan Manzoor, Haydar Qarawlus, Rafael Schellenberg, Holger Karl, and Artur Hecker. “Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning.” <i>Transactions on Network and Service Management</i>, 2021. <a href=\"https://doi.org/10.1109/TNSM.2021.3076503\">https://doi.org/10.1109/TNSM.2021.3076503</a>.","apa":"Schneider, S. B., Khalili, R., Manzoor, A., Qarawlus, H., Schellenberg, R., Karl, H., &#38; Hecker, A. (2021). Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning. <i>Transactions on Network and Service Management</i>. <a href=\"https://doi.org/10.1109/TNSM.2021.3076503\">https://doi.org/10.1109/TNSM.2021.3076503</a>","short":"S.B. Schneider, R. Khalili, A. Manzoor, H. Qarawlus, R. Schellenberg, H. Karl, A. Hecker, Transactions on Network and Service Management (2021).","ama":"Schneider SB, Khalili R, Manzoor A, et al. Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning. <i>Transactions on Network and Service Management</i>. 2021. doi:<a href=\"https://doi.org/10.1109/TNSM.2021.3076503\">10.1109/TNSM.2021.3076503</a>","bibtex":"@article{Schneider_Khalili_Manzoor_Qarawlus_Schellenberg_Karl_Hecker_2021, title={Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning}, DOI={<a href=\"https://doi.org/10.1109/TNSM.2021.3076503\">10.1109/TNSM.2021.3076503</a>}, journal={Transactions on Network and Service Management}, publisher={IEEE}, author={Schneider, Stefan Balthasar and Khalili, Ramin and Manzoor, Adnan and Qarawlus, Haydar and Schellenberg, Rafael and Karl, Holger and Hecker, Artur}, year={2021} }","ieee":"S. B. Schneider <i>et al.</i>, “Self-Learning Multi-Objective Service Coordination Using Deep Reinforcement Learning,” <i>Transactions on Network and Service Management</i>, 2021."},"type":"journal_article","oa":"1","keyword":["network management","service management","coordination","reinforcement learning","self-learning","self-adaptation","multi-objective"],"publisher":"IEEE","user_id":"35343","year":"2021","status":"public","_id":"21808","article_type":"original"}