EPiGRAM-HS focuses in designing and delivering a programming environment to support the execution of large scale applications. Below are the pilot applications that are used to test our programming environment.
It is our goal that HPC application developers can adopt our environment as easily as possible, by providing them with interfaces that hide the complexity of the heterogeneous systems.
The IFS software (Integrated Forecast System) is a major European resource representing many years of investment by the ECMWF’s Member States. IFS has been developed for over more than 30 years and has been run on various hardware architectures, such as shared and distributed memory machines, vector and scalar multi-core processors.
Given the size and complexity of the IFS code base, the work in EPiGRAM-HS will be carried out on a selection of ESCAPE dwarves which stand to benefit the most from runtime developments in the project. In particular, implementations of dwarves related to spectral transforms and to advection will be carried out using GPI. This will allow investigations of both automatic memory placement developments and runtime advancements for the use of distributed reconfigurable hardware.
IPR: Noncommercial Creative Commons license
Image: Nek5000 simulation of a nuclear reactor.
Computational Fluid Dynamics, KTH
Nek5000 is a highly scalable open-source code for the simulation of incom- pressible flows in moderately complex geometries . The code is maintained at Argonne National Laboratory (ANL) and is used at several universities across Europe, including KTH and ETH. Nek5000 discretization scheme is based on the spectral-element method (SEM), which combines the high-order accuracy from spectral methods with the geometric flexibility of finite-element methods. Such scheme is inherently more accurate than the present industry standard of using low-order finite volume codes.
Our objective is to carry out the full port of the Nek5000 application to large-scale heterogeneous systems with GPUs by redesigning the gather-scatter operator to use the accelerator-centric extension of MPI. The experience gained in porting a legacy Fortran application to large- scales systems with GPUs will allow us to derive best-practices and guide for porting large-scale/production-level applications.
IPR: BSD License
Image: Volume rendering of the velocity magnitude of the turbulent flow in the Taylor-Green vortex
High-order spectral element flow simulation, KTH
Neko is a portable framework for high-order spectral element flow simulations. Written in modern Fortran, Neko adopts an object-oriented approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, CUDA and HIP enabled accelerators to SX-Aurora vector processors. Neko has its roots in the spectral element code Nek5000 from UChicago/ANL, from where many of the namings, code structure and numerical methods are adopted.
Image: iPIC3D simulation of a magnetosphere.
Space Weather Prediction, KTH
iPIC3D is an open-source C++ and MPI Particle-in-Cell code for the simulation of space plasma. The iPIC3D code is maintained at KTH and it is a component of the Space Weather Modeling Framework (SWMF) that is maintained at University of Michigan and used to predict the impact of space weather events in space. The numerical discretization of the Maxwell’s equa- tions and particle equations of motion is based on the semi-implicit method that allows simulations with large time step and grid spacing still retaining the numerical stability. iPIC3D is used at many Institutions in Europe (KTH, KU Leuven, University of Pisa, University of Nice) and it of importance for preparing ESA and NASA missions. The code is entirely written in C++ and consists of approximately 30,000 lines of code. Within the EPiGRAM project, we extended the scalability of the code up to one million cores, developed a GASPI communication, and experimented porting to GPU systems with OpenACC.
Our objective is to fully port iPIC3D to large- scale GPU heterogeneous systems using accelerator MPI extensions and au- tomatic data placement strategies for coupling with other component of the SWMF framework. In addition, we intend to use support for FPGAs and the deep-learning DSL to enable data analysis of the iPIC3D simulation results.
IPR: Apache License
Image: Cancer detection in lung scan.
Lung Cancer Detection + Malware Classification
StreamBrain is a high-performance DSL for brain-like neural networks. StreamBrain supports multiple backends such as FPGAs, GPUs and CPUs on heterogeneous HPC systems while providing a convenient Keras-like interface to users.
Our objective is to validate the components of EPiGRAM-HS by deploying our software created using high-level machine learning libraries, adopted for specific hybrid hardware. We expect to use different TensorFlow capabilities, in particular the Dataset API for high-performance tuning of the input pipeline. Prototype implementations are expected to be adopted to GPI-Space and DSL, and ported to FPGAs.
An example of our approach can be found here,or on "Characterizing Deep-Learning I/O Workloads in TensorFlow."
IPR: 2-Clause License BSD
Tarantella is an open-source, distributed Deep Learning framework built on top of TensorFlow 2, providing scalable Deep Neural Network training on CPU and GPU compute clusters.
GPI-Space is a task-based workflow management system for parallel applications. It allows the developers to build domain-specific workflows using their own parallelization patterns, data management policies and I/O routines, while relying on the runtime system for the workflow management. The GPI-Space ecosystem "auto-manages" the application runs with dynamic scheduling, in-built distributed memory transfer and distributed task execution.