EPiGRAM-HS focuses in designing and delivering a programming environment to support the execution of large scale applications. Below are the pilot applications that are used to test our programming environment.
It is our goal that HPC application developers can adopt our environment as easily as possible, by providing them with interfaces that hide the complexity of the heterogeneous systems.
The IFS software (Integrated Forecast System) is a major European resource representing many years of investment by the ECMWF’s Member States. IFS has been developed for over more than 30 years and has been run on various hardware architectures, such as shared and distributed memory machines, vector and scalar multi-core processors.
Given the size and complexity of the IFS code base, the work in EPiGRAM-HS will be carried out on a selection of ESCAPE dwarves which stand to benefit the most from runtime developments in the project. In particular, implementations of dwarves related to spectral transforms and to advection will be carried out using GPI. This will allow investigations of both automatic memory placement developments and runtime advancements for the use of distributed reconfigurable hardware.
Image: Nek5000 simulation of a nuclear reactor.
Computational Fluid Dynamics, KTH
Nek5000 is a highly scalable open-source code for the simulation of incom- pressible flows in moderately complex geometries . The code is maintained at Argonne National Laboratory (ANL) and is used at several universities across Europe, including KTH and ETH. Nek5000 discretization scheme is based on the spectral-element method (SEM), which combines the high-order accuracy from spectral methods with the geometric flexibility of finite-element methods. Such scheme is inherently more accurate than the present industry standard of using low-order finite volume codes.
Our objective is to carry out the full port of the Nek5000 application to large-scale heterogeneous systems with GPUs by redesigning the gather-scatter operator to use the accelerator-centric extension of MPI. The experience gained in porting a legacy Fortran application to large- scales systems with GPUs will allow us to derive best-practices and guide for porting large-scale/production-level applications.
Image: iPIC3D simulation of a magnetosphere.
Space Weather Prediction, KTH
iPIC3D is an open-source C++ and MPI Particle-in-Cell code for the simulation of space plasma. The iPIC3D code is maintained at KTH and it is a component of the Space Weather Modeling Framework (SWMF) that is maintained at University of Michigan and used to predict the impact of space weather events in space. The numerical discretization of the Maxwell’s equa- tions and particle equations of motion is based on the semi-implicit method that allows simulations with large time step and grid spacing still retaining the numerical stability. iPIC3D is used at many Institutions in Europe (KTH, KU Leuven, University of Pisa, University of Nice) and it of importance for preparing ESA and NASA missions. The code is entirely written in C++ and consists of approximately 30,000 lines of code. Within the EPiGRAM project, we extended the scalability of the code up to one million cores, developed a GASPI communication, and experimented porting to GPU systems with OpenACC.
Our objective is to fully port iPIC3D to large- scale GPU heterogeneous systems using accelerator MPI extensions and au- tomatic data placement strategies for coupling with other component of the SWMF framework. In addition, we intend to use support for FPGAs and the deep-learning DSL to enable data analysis of the iPIC3D simulation results.
Image: Cancer detection in lung scan.
Lung Cancer Detection + Malware Classification
Can we improve lung cancer detection? Our first use-case will be to determine when lesions in the lungs are cancerous, based on analysis of high-resolution lung scans provided by the National Cancer Institute. The training dataset is approximately 124 GB. This computer-aided detection problem constitutes the 2017 Data Science Bowl challenge, a milestone in support of the Cancer Moonshot initiative.
Microsoft Malware Classification Challenge. Our second use case is detection and classification of malware hidden in files. For the classification challenge, Microsoft provided the data science community with a large annotated malware dataset of approximately 17 GB. The classification problem is followed by the more challenging and more relevant nowadays detection problem, which we also might tackle, to be on the edge of the present-day techniques.
Our objective is to validate the components of EPiGRAM-HS by deploying our software created using high-level machine learning libraries such as TensorFlow, adopted for specific hybrid hardware. We expect to use different TensorFlow capabilities, in particular the Dataset API for high-performance tuning of the input pipeline. Prototype implementations are expected to be adopted to GPI-Space and DSL, and ported to FPGAs.
An example of our approach can be found here,or on "Characterizing Deep-Learning I/O Workloads in TensorFlow."