Pilot Applications

EPiGRAM-HS focuses in designing and delivering a programming environment to support the execution of large scale applications. Below are the pilot applications that are used to test our programming environment.

It is our goal that HPC application developers can adopt our environment as easily as possible, by providing them with interfaces that hide the complexity of the heterogeneous systems.

Image source: https://www.ecmwf.int.

Weather ForecastECMWF

The IFS software (Integrated Forecast System) is a major European resource representing many years of investment by the ECMWF’s Member States. IFS has been developed for over more than 30 years and has been run on various hardware architectures, such as shared and distributed memory machines, vector and scalar multi-core processors.

EPiGRAM-HS Contribution

Given the size and complexity of the IFS code base, the work in EPiGRAM-HS will be carried out on a selection of ESCAPE dwarves which stand to benefit the most from runtime developments in the project. In particular, implementations of dwarves related to spectral transforms and to advection will be carried out using GPI. This will allow investigations of both automatic memory placement developments and runtime advancements for the use of distributed reconfigurable hardware.

IPR: Noncommercial Creative Commons license

Image: Nek5000 simulation of a nuclear reactor.

Computational Fluid Dynamics, KTH

Nek5000 is a highly scalable open-source code for the simulation of incom- pressible flows in moderately complex geometries [13]. The code is maintained at Argonne National Laboratory (ANL) and is used at several universities across Europe, including KTH and ETH. Nek5000 discretization scheme is based on the spectral-element method (SEM), which combines the high-order accuracy from spectral methods with the geometric flexibility of finite-element methods. Such scheme is inherently more accurate than the present industry standard of using low-order finite volume codes.

EPiGRAM-HS Contribution

Our objective is to carry out the full port of the Nek5000 application to large-scale heterogeneous systems with GPUs by redesigning the gather-scatter operator to use the accelerator-centric extension of MPI. The experience gained in porting a legacy Fortran application to large- scales systems with GPUs will allow us to derive best-practices and guide for porting large-scale/production-level applications.

IPR: BSD License

Image: Volume rendering of the velocity magnitude of the turbulent flow in the Taylor-Green vortex

High-order spectral element flow simulation, KTH

Neko is a portable framework for high-order spectral element flow simulations. Written in modern Fortran, Neko adopts an object-oriented approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, CUDA and HIP enabled accelerators to SX-Aurora vector processors. Neko has its roots in the spectral element code Nek5000 from UChicago/ANL, from where many of the namings, code structure and numerical methods are adopted.

Neko is currently maintained and developed at KTH Royal Institute of Technology.

IPR: 3-clause BSD

Image: iPIC3D simulation of a magnetosphere.

Space Weather Prediction, KTH

iPIC3D is an open-source C++ and MPI Particle-in-Cell code for the simulation of space plasma. The iPIC3D code is maintained at KTH and it is a component of the Space Weather Modeling Framework (SWMF) that is maintained at University of Michigan and used to predict the impact of space weather events in space. The numerical discretization of the Maxwell’s equa- tions and particle equations of motion is based on the semi-implicit method that allows simulations with large time step and grid spacing still retaining the numerical stability. iPIC3D is used at many Institutions in Europe (KTH, KU Leuven, University of Pisa, University of Nice) and it of importance for preparing ESA and NASA missions. The code is entirely written in C++ and consists of approximately 30,000 lines of code. Within the EPiGRAM project, we extended the scalability of the code up to one million cores, developed a GASPI communication, and experimented porting to GPU systems with OpenACC.

EPiGRAM-HS Contribution

Our objective is to fully port iPIC3D to large- scale GPU heterogeneous systems using accelerator MPI extensions and au- tomatic data placement strategies for coupling with other component of the SWMF framework. In addition, we intend to use support for FPGAs and the deep-learning DSL to enable data analysis of the iPIC3D simulation results.

IPR: Apache License

Image: Cancer detection in lung scan.

Lung Cancer Detection + Malware Classification


StreamBrain is a high-performance DSL for brain-like neural networks. StreamBrain supports multiple backends such as FPGAs, GPUs and CPUs on heterogeneous HPC systems while providing a convenient Keras-like interface to users.

EPiGRAM-HS contribution

Our objective is to validate the components of EPiGRAM-HS by deploying our software created using high-level machine learning libraries, adopted for specific hybrid hardware. We expect to use different TensorFlow capabilities, in particular the Dataset API for high-performance tuning of the input pipeline. Prototype implementations are expected to be adopted to GPI-Space and DSL, and ported to FPGAs.

An example of our approach can be found here,or on "Characterizing Deep-Learning I/O Workloads in TensorFlow."

IPR: 2-Clause License BSD


Tarantella is an open-source, distributed Deep Learning framework built on top of TensorFlow 2, providing scalable Deep Neural Network training on CPU and GPU compute clusters.

EPiGRAM-HS contribution


GPI-Space is a task-based workflow management system for parallel applications. It allows the developers to build domain-specific workflows using their own parallelization patterns, data management policies and I/O routines, while relying on the runtime system for the workflow management. The GPI-Space ecosystem "auto-manages" the application runs with dynamic scheduling, in-built distributed memory transfer and distributed task execution.

EPiGRAM-HS contribution

IPR: GPLv3 - Free for academic use. Dual licensing option for commercial use. More information here.