Emulation-based Smoke Testing of NFV Orchestrators in Large Multi-PoP Environments

Management and orchestration~(MANO) systems are the key components of future large-scale NFV environments. They will manage resources of hundreds or even thousands of NFV infrastructure installations, so called points of presence~(PoP). Such scenarios need to be automatically tested during the development phase of a MANO system. This task becomes very challenging because large-scale NFV testbeds are hard to maintain, too expensive, or simply not available. In this paper, we present a multi-PoP NFV infrastructure emulation platform that enables automated, large-scale testing of MANO stacks. We show that our platform can easily emulate hundreds of PoPs on a single physical machine and reduces the setup time of a test PoP by a factor of 232x compared to a DevStack-based test PoP installation. Further, we present a case study in which we test ETSI's Open Source MANO~(OSM) against our proposed system to gain insights about OSM's behaviour in large-scale NFV deployments.