EVEREST SDK: A System Development Kit for Extreme-Scale Data Analytics on High-Performance and Heterogeneous Platforms

October 27, 2023 - 11:30 - 12:30
Room 4F
Technology, Platforms, and impact
In person
Share this

The objective of the special session is to introduce the EVEREST System Development Kit (SDK) to the Big Data community and the solution the project provides. EVEREST harnesses the computational power of future heterogeneous distributed systems for extreme-data applications, moving towards more efficient big-data computing and knowledge extraction from vast amounts of data. The EVEREST SDK will help designers to optimize high-level descriptions for heterogeneous platforms with minimal human intervention. Further description (if needed): EVEREST project aims at proposing a design environment to target high-performance, distributed and FPGA based architectures. The EVEREST design environment aims at simplifying the description, optimization, and implementation of extreme-scale Big Data applications (with multiple data sources) onto heterogeneous and distributed architectures having different computational paradigms and requirements. EVEREST targets next-generation architectures for big-data computing and extreme-scale analytics with a combination of domain-specific extensions, compiler-level transformations, hardware optimizations, and adaptive strategies to accelerate distributed, AI-based applications by means of FPGA-based architectures. With this approach, the EVEREST programming environment forms the basis for exploiting the computational power of future heterogeneous distributed systems for extreme-data applications, moving towards a more efficient big-data computing and knowledge extraction from vast amounts of data. Today, there are many factors that act against reaching the efficient usage of heterogeneous architectures with reconfigurable fabric. One of the major issues is the programmability effort required by such systems. Solutions currently offered are lacking in supporting the entire software stack, relying on specific solutions that are hard to integrate. Domain-specific information is usually not propagated through the entire compilation toolchain, limiting the optimizations and the results that can be achieved. In addition to all this, hardware acceleration has a steep learning curve. The combination of heterogeneous and distributed platforms and companion design methods proposed in EVEREST will offer the possibility to approach larger research problems by non-expert application developers, opening new market opportunities for both service providers and technology developers. The EVEREST SDK includes compilation and runtime solutions. At the compilation time, the different input flows are unified into a single high-level representation (based on the MLIR framework) which is then connected to a hardware generation flow based on high-level synthesis. This approach allows us to leverage commercial tools for hardware generation from high-level languages. At runtime, the EVEREST SDK combines virtualization and autotuning solutions for adapt the computation to resource or application changes. The EVEREST approach was validated on industry-relevant applications: prediction of renewable energy production, air-quality monitoring, and traffic modelling for transportation in smart cities.

This year, the focus will be put on the EVEREST SDK and its application within the use cases (renewable energy prediction, air quality monitoring, and traffic modelling). The EVEREST applications are, indeed, representative use cases for the challenges in terms of programmability and quality of results for current extreme-scale Big Data systems. Application providers can broaden their target audience by using new Big Data platforms with limited portability costs, while huge potential markets in terms of design tools, platforms, and IPs are likely to be opened based on the results of the EVEREST project.

Session’s presentation here

Christian Pilato
Assistant Professor at Politecnico di Milano
Katerina Slaninova
Senior researcher at IT4Innovations National Supercomputing Center
Christoph Hagleitner
Manager Accelerator Technologies Group at IBM Research - Zurich Lab
Button to get your ticket now Button to get your ticket now