TY - GEN AB - We push the boundaries of electronic structure-based \textit{ab-initio} molecular dynamics (AIMD) beyond 100 million atoms. This scale is otherwise barely reachable with classical force-field methods or novel neural network and machine learning potentials. We achieve this breakthrough by combining innovations in linear-scaling AIMD, efficient and approximate sparse linear algebra, low and mixed-precision floating-point computation on GPUs, and a compensation scheme for the errors introduced by numerical approximations. The core of our work is the non-orthogonalized local submatrix method (NOLSM), which scales very favorably to massively parallel computing systems and translates large sparse matrix operations into highly parallel, dense matrix operations that are ideally suited to hardware accelerators. We demonstrate that the NOLSM method, which is at the center point of each AIMD step, is able to achieve a sustained performance of 324 PFLOP/s in mixed FP16/FP32 precision corresponding to an efficiency of 67.7% when running on 1536 NVIDIA A100 GPUs. AU - Schade, Robert AU - Kenter, Tobias AU - Elgabarty, Hossam AU - Lass, Michael AU - Schütt, Ole AU - Lazzaro, Alfio AU - Pabst, Hans AU - Mohr, Stephan AU - Hutter, Jürg AU - Kühne, Thomas D. AU - Plessl, Christian ID - 32244 T2 - arXiv:2104.08245 TI - Towards Electronic Structure-Based Ab-Initio Molecular Dynamics Simulations with Hundreds of Millions of Atoms ER - TY - CONF AU - Meyer, Marius ID - 27365 T2 - Proceedings of the 11th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies TI - Towards Performance Characterization of FPGAs in Context of HPC using OpenCL Benchmarks ER - TY - CONF AB - Electronic structure calculations based on density-functional theory (DFT) represent a significant part of today's HPC workloads and pose high demands on high-performance computing resources. To perform these quantum-mechanical DFT calculations on complex large-scale systems, so-called linear scaling methods instead of conventional cubic scaling methods are required. In this work, we take up the idea of the submatrix method and apply it to the DFT computations in the software package CP2K. For that purpose, we transform the underlying numeric operations on distributed, large, sparse matrices into computations on local, much smaller and nearly dense matrices. This allows us to exploit the full floating-point performance of modern CPUs and to make use of dedicated accelerator hardware, where performance has been limited by memory bandwidth before. We demonstrate both functionality and performance of our implementation and show how it can be accelerated with GPUs and FPGAs. AU - Lass, Michael AU - Schade, Robert AU - Kühne, Thomas AU - Plessl, Christian ID - 16898 T2 - Proc. International Conference for High Performance Computing, Networking, Storage and Analysis (SC) TI - A Submatrix-Based Method for Approximate Matrix Function Evaluation in the Quantum Chemistry Code CP2K ER - TY - CONF AB - FPGAs have found increasing adoption in data center applications since a new generation of high-level tools have become available which noticeably reduce development time for FPGA accelerators and still provide high-quality results. There is, however, no high-level benchmark suite available, which specifically enables a comparison of FPGA architectures, programming tools, and libraries for HPC applications. To fill this gap, we have developed an OpenCL-based open-source implementation of the HPCC benchmark suite for Xilinx and Intel FPGAs. This benchmark can serve to analyze the current capabilities of FPGA devices, cards, and development tool flows, track progress over time, and point out specific difficulties for FPGA acceleration in the HPC domain. Additionally, the benchmark documents proven performance optimization patterns. We will continue optimizing and porting the benchmark for new generations of FPGAs and design tools and encourage active participation to create a valuable tool for the community. To fill this gap, we have developed an OpenCL-based open-source implementation of the HPCC benchmark suite for Xilinx and Intel FPGAs. This benchmark can serve to analyze the current capabilities of FPGA devices, cards, and development tool flows, track progress over time, and point out specific difficulties for FPGA acceleration in the HPC domain. Additionally, the benchmark documents proven performance optimization patterns. We will continue optimizing and porting the benchmark for new generations of FPGAs and design tools and encourage active participation to create a valuable tool for the community. AU - Meyer, Marius AU - Kenter, Tobias AU - Plessl, Christian ID - 21632 KW - FPGA KW - OpenCL KW - High Level Synthesis KW - HPC benchmarking SN - 9781665415927 T2 - 2020 IEEE/ACM International Workshop on Heterogeneous High-performance Reconfigurable Computing (H2RC) TI - Evaluating FPGA Accelerator Performance with a Parameterized OpenCL Adaptation of Selected Benchmarks of the HPCChallenge Benchmark Suite ER - TY - GEN AB - We consider a resource-aware variant of the classical multi-armed bandit problem: In each round, the learner selects an arm and determines a resource limit. It then observes a corresponding (random) reward, provided the (random) amount of consumed resources remains below the limit. Otherwise, the observation is censored, i.e., no reward is obtained. For this problem setting, we introduce a measure of regret, which incorporates the actual amount of allocated resources of each learning round as well as the optimality of realizable rewards. Thus, to minimize regret, the learner needs to set a resource limit and choose an arm in such a way that the chance to realize a high reward within the predefined resource limit is high, while the resource limit itself should be kept as low as possible. We derive the theoretical lower bound on the cumulative regret and propose a learning algorithm having a regret upper bound that matches the lower bound. In a simulation study, we show that our learning algorithm outperforms straightforward extensions of standard multi-armed bandit algorithms. AU - Bengs, Viktor AU - Hüllermeier, Eyke ID - 32242 T2 - arXiv:2011.00813 TI - Multi-Armed Bandits with Censored Consumption of Resources ER - TY - CONF AB - Stratix 10 FPGA cards have a good potential for the acceleration of HPC workloads since the Stratix 10 product line introduces devices with a large number of DSP and memory blocks. The high level synthesis of OpenCL codes can play a fundamental role for FPGAs in HPC, because it allows to implement different designs with lower development effort compared to hand optimized HDL. However, Stratix 10 cards are still hard to fully exploit using the Intel FPGA SDK for OpenCL. The implementation of designs with thousands of concurrent arithmetic operations often suffers from place and route problems that limit the maximum frequency or entirely prevent a successful synthesis. In order to overcome these issues for the implementation of the matrix multiplication, we formulate Cannon's matrix multiplication algorithm with regard to its efficient synthesis within the FPGA logic. We obtain a two-level block algorithm, where the lower level sub-matrices are multiplied using our Cannon's algorithm implementation. Following this design approach with multiple compute units, we are able to get maximum frequencies close to and above 300 MHz with high utilization of DSP and memory blocks. This allows for performance results above 1 TeraFLOPS. AU - Gorlani, Paolo AU - Kenter, Tobias AU - Plessl, Christian ID - 15478 T2 - Proceedings of the International Conference on Field-Programmable Technology (FPT) TI - OpenCL Implementation of Cannon's Matrix Multiplication Algorithm on Intel Stratix 10 FPGAs ER - TY - CONF AB - This paper describes a data structure and a heuristic to plan and map arbitrary resources in complex combinations while applying time dependent constraints. The approach is used in the planning based workload manager OpenCCS at the Paderborn Center for Parallel Computing (PC\(^2\)) to operate heterogeneous clusters with up to 10000 cores. We also show performance results derived from four years of operation. AU - Keller, Axel ED - Klusáček, D. ED - Cirne, W. ED - Desai, N. ID - 22 KW - Scheduling Planning Mapping Workload management SN - 978-3-319-77398-8 T2 - Proc. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP) TI - A Data Structure for Planning Based Workload Management of Heterogeneous HPC Systems VL - 10773 ER - TY - CONF AB - We present the submatrix method, a highly parallelizable method for the approximate calculation of inverse p-th roots of large sparse symmetric matrices which are required in different scientific applications. Following the idea of Approximate Computing, we allow imprecision in the final result in order to utilize the sparsity of the input matrix and to allow massively parallel execution. For an n x n matrix, the proposed algorithm allows to distribute the calculations over n nodes with only little communication overhead. The result matrix exhibits the same sparsity pattern as the input matrix, allowing for efficient reuse of allocated data structures. We evaluate the algorithm with respect to the error that it introduces into calculated results, as well as its performance and scalability. We demonstrate that the error is relatively limited for well-conditioned matrices and that results are still valuable for error-resilient applications like preconditioning even for ill-conditioned matrices. We discuss the execution time and scaling of the algorithm on a theoretical level and present a distributed implementation of the algorithm using MPI and OpenMP. We demonstrate the scalability of this implementation by running it on a high-performance compute cluster comprised of 1024 CPU cores, showing a speedup of 665x compared to single-threaded execution. AU - Lass, Michael AU - Mohr, Stephan AU - Wiebeler, Hendrik AU - Kühne, Thomas AU - Plessl, Christian ID - 1590 KW - approximate computing KW - linear algebra KW - matrix inversion KW - matrix p-th roots KW - numeric algorithm KW - parallel computing SN - 978-1-4503-5891-0/18/07 T2 - Proc. Platform for Advanced Scientific Computing (PASC) Conference TI - A Massively Parallel Algorithm for the Approximate Calculation of Inverse p-th Roots of Large Sparse Matrices ER - TY - CONF AU - Riebler, Heinrich AU - Vaz, Gavin Francis AU - Kenter, Tobias AU - Plessl, Christian ID - 1204 KW - htrop SN - 9781450349826 T2 - Proc. ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP) TI - Automated Code Acceleration Targeting Heterogeneous OpenCL Devices ER - TY - CONF AB - The exploration of FPGAs as accelerators for scientific simulations has so far mostly been focused on small kernels of methods working on regular data structures, for example in the form of stencil computations for finite difference methods. In computational sciences, often more advanced methods are employed that promise better stability, convergence, locality and scaling. Unstructured meshes are shown to be more effective and more accurate, compared to regular grids, in representing computation domains of various shapes. Using unstructured meshes, the discontinuous Galerkin method preserves the ability to perform explicit local update operations for simulations in the time domain. In this work, we investigate FPGAs as target platform for an implementation of the nodal discontinuous Galerkin method to find time-domain solutions of Maxwell's equations in an unstructured mesh. When maximizing data reuse and fitting constant coefficients into suitably partitioned on-chip memory, high computational intensity allows us to implement and feed wide data paths with hundreds of floating point operators. By decoupling off-chip memory accesses from the computations, high memory bandwidth can be sustained, even for the irregular access pattern required by parts of the application. Using the Intel/Altera OpenCL SDK for FPGAs, we present different implementation variants for different polynomial orders of the method. In different phases of the algorithm, either computational or bandwidth limits of the Arria 10 platform are almost reached, thus outperforming a highly multithreaded CPU implementation by around 2x. AU - Kenter, Tobias AU - Mahale, Gopinath AU - Alhaddad, Samer AU - Grynko, Yevgen AU - Schmitt, Christian AU - Afzal, Ayesha AU - Hannig, Frank AU - Förstner, Jens AU - Plessl, Christian ID - 1588 KW - tet_topic_hpc T2 - Proc. Int. Symp. on Field-Programmable Custom Computing Machines (FCCM) TI - OpenCL-based FPGA Design to Accelerate the Nodal Discontinuous Galerkin Method for Unstructured Meshes ER - TY - CONF AB - Compared to classical HDL designs, generating FPGA with high-level synthesis from an OpenCL specification promises easier exploration of different design alternatives and, through ready-to-use infrastructure and common abstractions for host and memory interfaces, easier portability between different FPGA families. In this work, we evaluate the extent of this promise. To this end, we present a parameterized FDTD implementation for photonic microcavity simulations. Our design can trade-off different forms of parallelism and works for two independent OpenCL-based FPGA design flows. Hence, we can target FPGAs from different vendors and different FPGA families. We describe how we used pre-processor macros to achieve this flexibility and to work around different shortcomings of the current tools. Choosing the right design configurations, we are able to present two extremely competitive solutions for very different FPGA targets, reaching up to 172 GFLOPS sustained performance. With the portability and flexibility demonstrated, code developers not only avoid vendor lock-in, but can even make best use of real trade-offs between different architectures. AU - Kenter, Tobias AU - Förstner, Jens AU - Plessl, Christian ID - 1592 KW - tet_topic_hpc T2 - Proc. Int. Conf. on Field Programmable Logic and Applications (FPL) TI - Flexible FPGA design for FDTD using OpenCL ER - TY - CONF AB - Version Control Systems (VCS) are a valuable tool for software development and document management. Both client/server and distributed (Peer-to-Peer) models exist, with the latter (e.g., Git and Mercurial) becoming increasingly popular. Their distributed nature introduces complications, especially concerning security: it is hard to control the dissemination of contents stored in distributed VCS as they rely on replication of complete repositories to any involved user. We overcome this issue by designing and implementing a concept for cryptography-enforced access control which is transparent to the user. Use of field-tested schemes (end-to-end encryption, digital signatures) allows for strong security, while adoption of convergent encryption and content-defined chunking retains storage efficiency. The concept is seamlessly integrated into Mercurial---respecting its distributed storage concept---to ensure practical usability and compatibility to existing deployments. AU - Lass, Michael AU - Leibenger, Dominik AU - Sorge, Christoph ID - 19 KW - access control KW - distributed version control systems KW - mercurial KW - peer-to-peer KW - convergent encryption KW - confidentiality KW - authenticity SN - 978-1-5090-2054-6 T2 - Proc. 41st Conference on Local Computer Networks (LCN) TI - Confidentiality and Authenticity for Distributed Version Control Systems - A Mercurial Extension ER - TY - CONF AU - Kenter, Tobias AU - Plessl, Christian ID - 24 T2 - Proc. Workshop on Heterogeneous High-performance Reconfigurable Computing (H2RC) TI - Microdisk Cavity FDTD Simulation on FPGA using OpenCL ER - TY - CONF AU - Dellnitz, Michael AU - Eckstein, Julian AU - Flaßkamp, Kathrin AU - Friedel, Patrick AU - Horenkamp, Christian AU - Köhler, Ulrich AU - Ober-Blöbaum, Sina AU - Peitz, Sebastian AU - Tiemeyer, Sebastian ID - 34 SN - 2212-0173 T2 - Progress in Industrial Mathematics at ECMI TI - Multiobjective Optimal Control Methods for the Development of an Intelligent Cruise Control VL - 22 ER - TY - CONF AU - Kenter, Tobias AU - Vaz, Gavin Francis AU - Riebler, Heinrich AU - Plessl, Christian ID - 171 T2 - Workshop on Reconfigurable Computing (WRC) TI - Opportunities for deferring application partitioning and accelerator synthesis to runtime (extended abstract) ER - TY - CONF AB - The use of heterogeneous computing resources, such as Graphic Processing Units or other specialized coprocessors, has become widespread in recent years because of their per- formance and energy efficiency advantages. Approaches for managing and scheduling tasks to heterogeneous resources are still subject to research. Although queuing systems have recently been extended to support accelerator resources, a general solution that manages heterogeneous resources at the operating system- level to exploit a global view of the system state is still missing.In this paper we present a user space scheduler that enables task scheduling and migration on heterogeneous processing resources in Linux. Using run queues for available resources we perform scheduling decisions based on the system state and on task characterization from earlier measurements. With a pro- gramming pattern that supports the integration of checkpoints into applications, we preempt tasks and migrate them between three very different compute resources. Considering static and dynamic workload scenarios, we show that this approach can gain up to 17% performance, on average 7%, by effectively avoiding idle resources. We demonstrate that a work-conserving strategy without migration is no suitable alternative. AU - Lösch, Achim AU - Beisel, Tobias AU - Kenter, Tobias AU - Plessl, Christian AU - Platzner, Marco ID - 168 T2 - Proceedings of the 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE) TI - Performance-centric scheduling with task migration for a heterogeneous compute node in the data center ER - TY - CONF AU - Lass, Michael AU - Kühne, Thomas AU - Plessl, Christian ID - 25 T2 - Workshop on Approximate Computing (AC) TI - Using Approximate Computing in Scientific Codes ER - TY - CONF AU - Riebler, Heinrich AU - Vaz, Gavin Francis AU - Plessl, Christian AU - Trainiti, Ettore M. G. AU - Durelli, Gianluca C. AU - Bolchini, Cristiana ID - 31 T2 - Proc. HiPEAC Workshop on Reonfigurable Computing (WRC) TI - Using Just-in-Time Code Generation for Transparent Resource Management in Heterogeneous Systems ER - TY - CONF AB - Hardware accelerators are becoming popular in academia and industry. To move one step further from the state-of-the-art multicore plus accelerator approaches, we present in this paper our innovative SAVEHSA architecture. It comprises of a heterogeneous hardware platform with three different high-end accelerators attached over PCIe (GPGPU, FPGA and Intel MIC). Such systems can process parallel workloads very efficiently whilst being more energy efficient than regular CPU systems. To leverage the heterogeneity, the workload has to be distributed among the computing units in a way that each unit is well-suited for the assigned task and executable code must be available. To tackle this problem we present two software components; the first can perform resource allocation at runtime while respecting system and application goals (in terms of throughput, energy, latency, etc.) and the second is able to analyze an application and generate executable code for an accelerator at runtime. We demonstrate the first proof-of-concept implementation of our framework on the heterogeneous platform, discuss different runtime policies and measure the introduced overheads. AU - Riebler, Heinrich AU - Vaz, Gavin Francis AU - Plessl, Christian AU - Trainiti, Ettore M. G. AU - Durelli, Gianluca C. AU - Del Sozzo, Emanuele AU - Santambrogio, Marco D. AU - Bolchini, Christina ID - 138 T2 - Proceedings of International Forum on Research and Technologies for Society and Industry (RTSI) TI - Using Just-in-Time Code Generation for Transparent Resource Management in Heterogeneous Systems ER - TY - CONF AB - This paper introduces Binary Acceleration At Runtime(BAAR), an easy-to-use on-the-fly binary acceleration mechanismwhich aims to tackle the problem of enabling existentsoftware to automatically utilize accelerators at runtime. BAARis based on the LLVM Compiler Infrastructure and has aclient-server architecture. The client runs the program to beaccelerated in an environment which allows program analysisand profiling. Program parts which are identified as suitable forthe available accelerator are exported and sent to the server.The server optimizes these program parts for the acceleratorand provides RPC execution for the client. The client transformsits program to utilize accelerated execution on the server foroffloaded program parts. We evaluate our work with a proofof-concept implementation of BAAR that uses an Intel XeonPhi 5110P as the acceleration target and performs automaticoffloading, parallelization and vectorization of suitable programparts. The practicality of BAAR for real-world examples is shownbased on a study of stencil codes. Our results show a speedup ofup to 4 without any developer-provided hints and 5.77 withhints over the same code compiled with the Intel Compiler atoptimization level O2 and running on an Intel Xeon E5-2670machine. Based on our insights gained during implementationand evaluation we outline future directions of research, e.g.,offloading more fine-granular program parts than functions, amore sophisticated communication mechanism or introducing onstack-replacement. AU - Damschen, Marvin AU - Plessl, Christian ID - 303 T2 - Proceedings of the 5th International Workshop on Adaptive Self-tuning Computing Systems (ADAPT) TI - Easy-to-Use On-The-Fly Binary Program Acceleration on Many-Cores ER -