TY - GEN AB - Iterative distributed optimization algorithms involve multiple agents that communicate with each other, over time, in order to minimize/maximize a global objective. In the presence of unreliable communication networks, the Age-of-Information (AoI), which measures the freshness of data received, may be large and hence hinder algorithmic convergence. In this paper, we study the convergence of general distributed gradient-based optimization algorithms in the presence of communication that neither happens periodically nor at stochastically independent points in time. We show that convergence is guaranteed provided the random variables associated with the AoI processes are stochastically dominated by a random variable with finite first moment. This improves on previous requirements of boundedness of more than the first moment. We then introduce stochastically strongly connected (SSC) networks, a new stochastic form of strong connectedness for time-varying networks. We show: If for any $p \ge0$ the processes that describe the success of communication between agents in a SSC network are $\alpha$-mixing with $n^{p-1}\alpha(n)$ summable, then the associated AoI processes are stochastically dominated by a random variable with finite $p$-th moment. In combination with our first contribution, this implies that distributed stochastic gradient descend converges in the presence of AoI, if $\alpha(n)$ is summable. AU - Redder, Adrian AU - Ramaswamy, Arunselvan AU - Karl, Holger ID - 30790 T2 - arXiv:2201.11343 TI - Distributed gradient-based optimization in the presence of dependent aperiodic communication ER - TY - GEN AB - We present sufficient conditions that ensure convergence of the multi-agent Deep Deterministic Policy Gradient (DDPG) algorithm. It is an example of one of the most popular paradigms of Deep Reinforcement Learning (DeepRL) for tackling continuous action spaces: the actor-critic paradigm. In the setting considered herein, each agent observes a part of the global state space in order to take local actions, for which it receives local rewards. For every agent, DDPG trains a local actor (policy) and a local critic (Q-function). The analysis shows that multi-agent DDPG using neural networks to approximate the local policies and critics converge to limits with the following properties: The critic limits minimize the average squared Bellman loss; the actor limits parameterize a policy that maximizes the local critic's approximation of $Q_i^*$, where $i$ is the agent index. The averaging is with respect to a probability distribution over the global state-action space. It captures the asymptotics of all local training processes. Finally, we extend the analysis to a fully decentralized setting where agents communicate over a wireless network prone to delays and losses; a typical scenario in, e.g., robotic applications. AU - Redder, Adrian AU - Ramaswamy, Arunselvan AU - Karl, Holger ID - 30791 T2 - arXiv:2201.00570 TI - Asymptotic Convergence of Deep Multi-Agent Actor-Critic Algorithms ER - TY - GEN AB - Macrodiversity is a key technique to increase the capacity of mobile networks. It can be realized using coordinated multipoint (CoMP), simultaneously connecting users to multiple overlapping cells. Selecting which users to serve by how many and which cells is NP-hard but needs to happen continuously in real time as users move and channel state changes. Existing approaches often require strict assumptions about or perfect knowledge of the underlying radio system, its resource allocation scheme, or user movements, none of which is readily available in practice. Instead, we propose three novel self-learning and self-adapting approaches using model-free deep reinforcement learning (DRL): DeepCoMP, DD-CoMP, and D3-CoMP. DeepCoMP leverages central observations and control of all users to select cells almost optimally. DD-CoMP and D3-CoMP use multi-agent DRL, which allows distributed, robust, and highly scalable coordination. All three approaches learn from experience and self-adapt to varying scenarios, reaching 2x higher Quality of Experience than other approaches. They have very few built-in assumptions and do not need prior system knowledge, making them more robust to change and better applicable in practice than existing approaches. AU - Schneider, Stefan Balthasar AU - Karl, Holger AU - Khalili, Ramin AU - Hecker, Artur ID - 33854 KW - mobility management KW - coordinated multipoint KW - CoMP KW - cell selection KW - resource management KW - reinforcement learning KW - multi agent KW - MARL KW - self-learning KW - self-adaptation KW - QoE TI - DeepCoMP: Coordinated Multipoint Using Multi-Agent Deep Reinforcement Learning ER - TY - GEN AB - Network and service coordination is important to provide modern services consisting of multiple interconnected components, e.g., in 5G, network function virtualization (NFV), or cloud and edge computing. In this paper, I outline my dissertation research, which proposes six approaches to automate such network and service coordination. All approaches dynamically react to the current demand and optimize coordination for high service quality and low costs. The approaches range from centralized to distributed methods and from conventional heuristic algorithms and mixed-integer linear programs to machine learning approaches using supervised and reinforcement learning. I briefly discuss their main ideas and advantages over other state-of-the-art approaches and compare strengths and weaknesses. AU - Schneider, Stefan Balthasar ID - 35889 KW - nfv KW - coordination KW - machine learning KW - reinforcement learning KW - phd KW - digest TI - Conventional and Machine Learning Approaches for Network and Service Coordination ER - TY - GEN AB - Understanding the behavior of distributed cloud service components in different load situations is important for efficient and automatic management and orchestration of these services. For this purpose and for practical research in distributed cloud computing in general, there is need for benchmarks and experimental data. In this paper, we describe our experiments for characterizing the relationship between resource demands of application components and the expected performance of applica- tions. We present initial results for predicting the interdependence between resource demands and performance characteristics using support vector regression and polynomial regression models. The data gathered from our experiments is publicly available. AU - Dräxler, Sevil AU - Peuster, Manuel AU - Illian, Marvin AU - Karl, Holger ID - 2483 TI - Towards Predicting Resource Demands and Performance of Distributed Cloud Services ER - TY - GEN AU - Rosa, Raphael Vicente AU - Rothenberg, Christian Esteve AU - Peuster, Manuel AU - Karl, Holger ID - 6485 TI - Methodology for VNF Benchmarking Automation ER - TY - GEN AU - Dräxler, Sevil AU - Karl, Holger ID - 749 T2 - CoRR TI - Specification of Complex Structures in Distributed Service Function Chaining Using a YANG Data Model ER - TY - GEN AU - Dräxler, Martin AU - Karl, Holger ID - 750 T2 - CoRR TI - Dynamic Backhaul Network Configuration in SDN-based Cloud RANs ER - TY - GEN AU - Mehraghdam, Sevil AU - Keller, Matthias AU - Karl, Holger ID - 766 T2 - CoRR TI - Specifying and Placing Chains of Virtual Network Functions ER - TY - GEN AU - Wette, Philip AU - Karl, Holger ID - 767 T2 - CoRR TI - DCT²Gen: A Versatile TCP Traffic Generator for Data Centers ER - TY - GEN AU - Schwabe, Arne AU - Karl, Holger ID - 768 T2 - CoRR TI - Adding Geographical Embedding to AS Topology Generation ER - TY - CHAP AU - Blanckenstein, Johannes AU - Garcia-Jimenez, Javier AU - Klaue, Jirka AU - Karl, Holger ID - 1800 SN - 1876-1100 T2 - Lecture Notes in Electrical Engineering TI - A Scalable Redundant TDMA Protocol for High-Density WSNs Inside an Aircraft ER - TY - CHAP AU - de la Oliva, Antonio AU - Morelli, Arianna AU - Mancuso, Vincenzo AU - Draexler, Martin AU - Hentschel, Tim AU - Melia, Telemaco AU - Seite, Pierrick AU - Cicconetti, Claudio ID - 1804 SN - 1867-8211 T2 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering TI - Denser Networks for the Future Internet, the CROWD Approach ER - TY - GEN AU - Dräxler, Martin AU - Blobel, Johannes AU - Dreimann, Philipp AU - Valentin, Stefan AU - Karl, Holger ID - 781 T2 - CoRR TI - Anticipatory Buffer Control and Quality Selection for Wireless Video Streaming ER - TY - GEN AU - Khan, Rana Azeem M. AU - Karl, Holger ID - 2504 TI - Simulating Cooperative Diversity Protocols for Multi-hop Wireless and Sensor Networks ER - TY - GEN AU - Dannewitz, Christian AU - Karl, Holger AU - Yadav, Aditya ID - 2505 TI - Report on Locality in DNS Requests – Evaluation and Impact on Future Internet Architectures ER - TY - GEN AB - Preemptive Routing and Wavelength Assignment (RWA) algorithms preempt established lightpaths in case not enough resources are available to setup a new lightpath in a Wavelength Division Multiplexing (WDM) network. The selection of lightpaths to be preempted relies on internal decisions of the RWA algorithm. Thus, if dedicated properties of the network topology are required by the applications running on the network, these requirements have to be known by the RWA algorithm. Otherwise it might happen that by preempting a particular lightpath these requirements are violated. If, however, these requirements include parameters only known at the nodes running the application, the RWA algorithm cannot evaluate the requirements. For this reason a RWA algorithm is needed which involves its users in the preemption decisions. We present a family of preemptive RWA algorithms for WDM networks. These algorithms have two distinguishing features: a) they can handle dynamic traffic by on-the-fly reconfiguration, and b) users can give feedback for reconfiguration decisions and thus influence the preemption decision of the RWA algorithm, leading to networks which adapt directly to application needs. This is different from traffic engineering where the network is (slowly) adapted to observed traffic patterns. Our algorithms handle various WDM network configurations including networks consisting of heterogeneous WDM hardware. To this end, we are using the layered graph approach together with a newly developed graph model that is used to determine conflicting lightpaths. AU - Wette, Philip AU - Karl, Holger ID - 603 TI - Introducing feedback to preemptive routing and wavelength assignment algorithms for dynamic traffic scenarios ER - TY - CHAP AU - Dannewitz, Christian AU - al, et ID - 2662 T2 - Architecture and Design for the Future Internet TI - How to manage and Search/Retrieve Information Objects ER - TY - CHAP AU - Dannewitz, Christian AU - al, et ID - 2663 T2 - Architecture and Design for the Future Internet TI - Integrating Generic Paths and NetInf ER - TY - CHAP AU - Kanter, Theo AU - Kardeby, Victor AU - Forsström, Stefan AU - Walters, Jamie ID - 1838 SN - 1867-8211 T2 - Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering TI - Scenarios, Research Issues, and Architecture for Ubiquitous Sensing ER - TY - CHAP AU - Savorić, Michael AU - Karl, Holger AU - Wolisz, Adam ID - 1915 SN - 1388-3437 T2 - Providing Quality of Service in Heterogeneous Environments, Proceedings of the 18th International Teletraffic Congress - ITC-18 TI - Selected properties of a joint congestion controller for TCP connections ER - TY - CHAP AU - Biermann, Thorsten AU - Schwabe, Arne AU - Karl, Holger ID - 1830 SN - 0302-9743 T2 - NETWORKING 2009 TI - Creating Butterflies in the Core – A Network Coding Extension for MPLS/RSVP-TE ER - TY - GEN AU - Biermann, Thorsten AU - Dannewitz, Christian AU - Karl, Holger ID - 2513 TI - Extended Results on an Adaptive Resource/Performance Trade-Off for Resolving Complex Queries in P2P Networks ER - TY - CHAP AU - Woldegebreal, Dereje H. AU - Karl, Holger ID - 1841 SN - 9783540776895 T2 - Lecture Notes in Computer Science TI - Network-Coding-Based Cooperative Transmission in Wireless Sensor Networks: Diversity-Multiplexing Tradeoff and Coverage Area Extension ER - TY - GEN AU - Eitzen, Falk AU - Valentin, Stefan AU - Gossens, Kai AU - Karl, Holger AU - Rolfes, Oliver ID - 2515 TI - Experimental evaluation of IEEE 802.11a-based WLANs for medium range communication ER - TY - GEN AU - Valentin, Stefan AU - von Malm, Holger AU - Karl, Holger ID - 2518 TI - Evaluating the GNU Software Radio platform for wireless testbeds ER - TY - BOOK AU - Karl, Holger AU - Willig, Andreas ID - 841 SN - 978-0-470-09510-2 TI - Protocols and architectures for wireless sensor networks ER - TY - CHAP AU - Karl, Holger ID - 848 T2 - The Industrial Information Technology Handbook TI - Ad Hoc Networks ER - TY - CHAP AU - Karl, Holger ID - 858 T2 - Location-Based Services TI - Data Transmission in Mobile Communication Systems ER - TY - GEN AU - Handziski, Vlado AU - Köpke, Andreas AU - Karl, Holger AU - Wolisz, Adam ID - 2533 TI - A common wireless sensor network architecture? ER - TY - GEN AU - Karl, Holger AU - Mengesha, Seble AU - Hollos, Daniel ID - 2538 TI - Relaying in Wireless Access Networks ER -