Taming <scp>next‐generation HPC</scp> systems: <scp>Run‐time</scp> system and algorithmic advancements

This special issue of Concurrency and Computation: Practice and Experience contains revised and extended versions of selected papers presented at the 13th International Conference on Parallel Processing and Applied Mathematics, PPAM 2019, which was held on September 8–11, 2019 in Bialystok, Poland. PPAM 2019 was organized by the Department of Computer and Information Science of the Czestochowa University of Technology together with the Bialystok University of Technology, under the patronage of the Committee of Informatics of the Polish Academy of Sciences, in technical cooperation with the IEEE Computer Society and IEEE Computational Intelligence Society. PPAM is a biennial series of international conferences dedicated to exchanging ideas between researchers involved in parallel and distributed computing, including theory and applications, as well as applied and computational mathematics. Twelve previous events have been held in different universities in Poland since 1994, when the first PPAM took place in Czestochowa. Thus, the event in Bialystok was an opportunity to celebrate the 25th anniversary of PPAM. The focus of PPAM 2019 was on models, algorithms, and software tools that facilitate efficient and convenient use of modern parallel and distributed computing systems, as well as on large-scale modern applications, including advances in machine learning and artificial intelligence. This meeting gathered more than 170 participants from 26 countries. The accepted papers were presented at the regular tracks of the PPAM 2019 conference and during the workshops. With each submission evaluated by at least three reviewers, a strict reviewing process resulted in the acceptance of 91 contributed papers for publication in the conference proceedings, while approximately 43% of the submissions were rejected. The Program Committee selected 41 papers for presentation in the regular conference track, resulting in an acceptance rate of about 46%. Based on the review results, 10 papers (11% of submissions) were selected for a special journal issue. Besides quality, another important criterion for selection was each paper's contribution to the thematic consistency of the issue. The focus of this special issue is on algorithmic advancements in matching the software properties to parallel architecture, including GPU accelerators and clusters. These advancements are crucial for successfully parallelizing such complex applications as simulating geophysical flows, solving ordinary differential equations (ODEs), structural analysis of nuclear reactor containment buildings, solving generalized eigenvalue problems, modeling of material science phenomena, and others. A complementary topic of this issue is advances in run-time systems since increasing levels of parallelism in multi- and many-core chips and the emerging heterogeneity of computational resources coupled with energy, resilience, and data movement constraints radically increase the importance of efficient run-time scheduling and execution control. After the conference, the Program Committee invited the authors of selected papers to submit revised and extended versions of their works. These new versions were reviewed independently again by at least three reviewers. Finally, nine contributions were accepted for publication. They are summarized below. Paper [1] focuses on the accurate assembly of the system matrix, which is an essential step in any code that solves partial differential equations on a mesh. This step can become costly in multigrid codes requiring cascades of matrices that depend upon each other, or dynamic adaptive mesh refinement. To reduce the time to solution, the authors propose that these constructions can be performed concurrently with the multigrid cycles. Furthermore, they desynchronize the assembly from the solution process. This non-trivial increase in the concurrency level improves the scalability. As assembly routines are notoriously memory- and bandwidth-demanding, the final algorithmic enhancement uses a hierarchical, lossy compression scheme that brings the memory footprint down aggressively even when the system matrix entries carry little information or are not yet available with high accuracy. An efficient algorithm for the parallel solution of indefinite saddle point systems with iterative solvers based on the Golub–Kahan bidiagonalization is presented in Reference [2]. Such systems arise in many application fields, for example, in structural mechanics. A scalability study of the generalized solver shows improved performance for the two-dimensional (2D) Stokes equations compared to previous works. Furthermore, the authors investigate the performance of different parallel inner solvers in the outer Golub–Kahan iteration for a three-dimensional (3D) Stokes problem. When the number of cores is increasing for a fixed problem size, the solver exhibits good speedups of up to 50% with the 1024 cores. For the tests in which the problem size grows while the workload in each core stays constant, the performance of the solver scales almost linearly with the increase in the number of cores. Paper [3] proposes a locality optimization technique for the parallel solution on GPUs of large systems of ODEs by explicit one-step methods. This technique is based on tiling across the stages of a one-step method and is enabled by a special structure of the class of ODE systems—with the limited access distance. The paper focuses on increasing the range of access distances for which the tiling technique can provide a speedup by joining the memory resources and computational power of multiple workgroups for computations in one tile. Consequently, a much wider range of applications can benefit from tiling across the stages, in particular, for stencils on 2D and 3D grids. The experiments show the novel technique's speedup compared with traditional single-workgroup tiling for two different Runge–Kutta methods on NVIDIA Kepler and Volta architectures. The StarNEig library for solving dense nonsymmetric standard and generalized eigenvalue problems is presented in paper [4]. This task-based library is built on top of the StarPU runtime system and targets both shared and distributed memory machines. Some components of the library have support for GPU acceleration. The library currently applies to real matrices with real and complex eigenvalues, and all calculations are performed using real arithmetic. Support for complex matrices is planned for a future release. The library's design choices and capabilities are disclosed and compared to existing software such as LAPACK and ScaLAPACK. StarNEig implements a ScaLAPACK compatibility layer, which should assist new users in the transition to StarNEig. The authors demonstrate the performance of the library with a sample of computational experiments. Paper [5] is a part of the global tendency to use high performance computing (HPC) systems for modeling the phase-field phenomena. This work aims to improve the performance of a parallel application for the solidification modeling with the dynamic intensity of computations in successive time steps when calculations are performed in selected nodes of the grid. A two-step method is proposed to optimize the application for multi- and many-core architectures. In the first step, loop fusion is used to reduce the number of conditional operators, while the second step includes an algorithm for the dynamic workload prediction and load balancing across cores. Two versions of the algorithm are proposed—one with the one-dimensional map and the other with the 2D map for predicting the computation domain within the grid. These optimizations allow increasing the application performance significantly for all tested configurations of computing resources, including a single and two Intel Xeon Platinum 8180 CPUs, and a single KNL accelerator. Paper [6] presents how advanced parallel graph partitioning algorithms can be designed using global application states monitoring of distributed programs. The proposed algorithms are implemented inside a novel distributed program design framework PEGASUS DA. Two strategies for designing the control of parallel/distributed partitioning algorithms are investigated. In the first one, the parallel algorithm control runs on top of the popular graph partitioning METIS tool. The second strategy is based on genetic programming. The use of the framework allows easy design and testing of different graph optimization strategies. Experiments with benchmark graphs illustrate the presented partitioning methods. These experiments comparatively assess the graph partitioning quality for which a clear improvement is observed, and the benefits of the proposed approach for programmers are identified. Profiling and tuning of parallel applications is an essential part of high performance computing. Analysis and elimination of application hotspots can be performed using many available tools, which also provide resource consumption measurements for instrumented parts of the code. Since each tool can bring different insights into an application's behavior, it is valuable to analyze and optimize an application using a variety of them. Paper [7] presents the C/C++ API that simplifies manual instrumentation for the most common open-source HPC performance analysis tools. Simultaneously, profiling libraries provide different instrumentation methods, with the binary patching being the universal mechanism that highly improves the user-friendliness of a tool. The authors analyze the most commonly used binary patching tools and present a workflow for using them to implement binary instrumentation for any profiler or autotuner. Paper [8] introduces DiPOSH, a multi-network, distributed implementation of the OpenSHMEM standard. The core idea behind DiPOSH is to have an API-to-network software stack as slim as possible to minimize the software overhead. Following the heritage of its non-distributed parent POSH, DiPOSH's communication engine is organized around the processes' shared heaps, and remote communications are moving data directly from and to these shared heaps. This paper presents DiPOSH's architecture and several communication drivers, including one that takes advantage of a helper process for inter-process communications. This architecture allows exploring different options for implementing the communication drivers, from using high-level, portable, optimized libraries to low-level, close to the hardware communication routines. The authors present the perspectives opened by this additional component in terms of communication scheduling between and on the nodes. Multicore NUMA systems present memory hierarchies and communication networks that influence the performance of parallel codes. Understanding the effect of particular hardware configurations on various codes is of paramount importance. In paper [9], monitoring information from hardware counters at runtime is used to characterize the behavior of each thread of multithreaded processes running in a NUMA environment. The authors propose a runtime tool, executed in user space, that uses this information to guide two different thread migration strategies for improving execution efficiency by increasing locality and affinity without requiring any modification in the codes. The benefits of this tool are validated using NAS Parallel OpenMP benchmarks with various locality and affinity scenarios. In more than 95% of them, the novel tool outperforms those of the operating system (OS) and produces up to 38% improvement in execution time over the OS. The guest editors of this special issue wish to thank the reviewers for the careful evaluation of the papers, and useful suggestions that helped the authors to improve their contributions. In addition, we would like to sincerely thank Professor Geoffrey Fox (former Editor-in-Chief of the Concurrency and Computation: Practice and Experience) and David W. Walker (current Editor-in-Chief of the journal) for opportunity to edit this special issue and for their guidance during this process.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.