Evaluation schemes for Big data and AI Performance of high Business impact (DataBench project sponsored session)

November 4, 2020 - 10:00 - 13:30
Focus Track 3 - Market uptake: Bringing AI and Data Sciences to Practice
Share this

For many years companies have been running technical benchmarks to compare the performance of different technologies and systems. This benchmarking process provides very valuable information but in many cases, it is not correlated with the impact of the solutions at business level. This was precisely the starting point of the DataBench project, funded by the EC under the Big Data Value PPP. After three years of intensive work DataBench has created an evaluation framework that is already openly available for the community. The session will introduce the different elements of the framework, including the toolbox, the way it can be used and some concrete examples run with companies and projects that work heavily with data.

DataBench has addressed one of the major barriers to the adoption of Big Data technologies, which has to do with the ability to measure the potential benefits of using Big data technologies in different scenarios. This is not exactly about deciding between one database and another one, but about understanding what the impact of such decision with have on a business process beyond the fulfillment of a technical KPI. This session will allow participants to understand the outcomes produced by DataBench and how to use them in their context through presentations, discussion panels and demos where they will also be entitled to contribute. Attendees will get knowledge on:
  • The current landscape of Big Data and AI benchmarks
  • The DataBench framework, which includes a complete set of metrics for the assessment of Big Data technologies
  • The DataBench toolbox, a web-based tool that provides a unique environment to search, select and deploy big data benchmarking tools, giving the possibility to generate unified technical metrics and derive business KPIs
  • A comprehensive set of use cases that we have run with companies in different industrial domains and projects of the Big Data PPP to illustrate the way you can get value out of using DataBench
  • Pipelines and blueprints

The DataBench toolbox follows a platform model and provides a single place where benchmarking communities and users of such benchmarks can meet. It goes beyond any work done so far by translating major technical KPIs into business KPIs, allowing us to establish relationships between technical and business decisions. As such, the session targets a multiplicity of stakeholders: benchmark providers, companies interested in benchmarking big data, projects and members of the big data and AI communities, decision-makers of companies that are thinking about their data-driven transformation and policy makers looking for evidence-based decision-support tools.

On our side, we will bring a comprehensive set of speakers falling precisely under all those categories.

The workshop will be structured around two parts (each of them can be attended independently; like this we want to facilitate that those participants that are not available for the entire workshop, select the most interesting part for them). Attending the complete workshop is in any case recommended to take maximum advantage of the contents.

Detailed agenda

10:00-10:05 Intro. Objectives of the session (Nuria de Lama (Atos)

10:05-10:15 DataBench General Overview (Richard Stevens (IDC, DataBench coordinator)

PART I. Big Data Benchmarking landscape and Big Data Pipelines

10:15-11:15 Session 1. The current landscape of Big Data benchmarks

  • 10:15-10:25: DataBench Framework for Benchmarks, Arne J. Berre, SINTEF
  • 10:25-10:40: Benchmarking platforms and AI, Axel Ngonga, BDVA TF6 Benchmark Lead, University of Paderborn
  • 10:40-10:55: BenchCouncil Big Data and AI Benchmarks, Wanling Gao, Chinese Academy of Sciences
  • 10:55-11:10: MLPerf AI and ABench, Rekha Singhal, Senior Scientist and Head of the Computing Systems-Software Research area at TCS
  • 11:10-11:15: Conclusion on Big Data and AI Benchmarks, Todor Ivanov, LeadConsult

11:15-12:15 Session 2. A Project perspective on Big Data and AI architectural pipelines and benchmarks

  • 11:15-11:20: Introduction to Architectural pipelines, Arne J. Berre, SINTEF
  • 11:20-11:30: I- BiDaaS – Leonidas Kallipolitis, AEGIS
  • 11:30-11:40: TBFY – Brian Elvesæter, SINTEF
  • 11:40-11:50: Track&KNow – Athanasios Koumparos, Vodafone Innovus
  • 11:50-12:00: DataBio – Caj Södergård, VTT
  • 12:00-12:10: DeepHealth – Jon Ander Gómez Adrián, Universitat Politecnica de Valencia
  • 12:10-12:15: Conclusion on Pipelines and related benchmarks, Arne J. Berre, SINTEF

12:15-12:30 Short coffee break to relax and maybe grab a coffee

PART II. Big Data Business Framework and benchmarking tooling support

  • 12:30-13:00 Session 3. The DataBench Business framework: a compelling offering to measure the impact of Big Data Technologies
  • 12.30-12.40: The DataBench business framework, by Gabriella Cattaneo, Erica Spinoni and Chiara Francalanci
  • 12: 40-12.50: The Whirlpool use case, Pierluigi Petrali (Whirlpool)
  • 12.50-13.00:  The Fill use case, Harald Sehrschön (FILL)

13:00-13:30 Session 4. A practical journey on how to use the DataBench Toolbox

  • 13:00-13:15 Demo of the DataBench Toolbox (Tomás Pariente, Atos)
  • 13:15-13:25 AI Observatory (Marko Grobelnik, JSI)
  • 13:25-13:30 Fostering adoption of DataBench results. Needs from the point of view of Digital Innovation Hubs (Daniel Sáez, ITI, EUHubs4Data coordinator)

13:30 Concluding Remarks and closing of the session

Arne Berre
Chief Scientist and Innovation Director at SINTEF and NorwAI
Athanasios Koumparos
Senior Software Engineer at Vodafone Innovus
Axel Ngonga
Full Professor of Data Science at Paderborn University at University of Paderborn, Lead of BDVA TF6 Benchmarking group
Brian Elvesæter
Research Scientist at SINTEF
Caj Södergård
Secretariat at ADRA
Chiara Francalanci
Prof. at University
Daniel Sáez Domingo
Strategic Intelligence & Technology Transfer Director at ITI - Instituto Tecnológico de Informática
Erica Spinoni
Research analyst at IDC Europe
Gabriella Cattaneo
Associate Vice President European Government Consulting at IDC EMEA at IDC
Harald Sehrschön
Teamleader R&D at Teamleiter F&E
Leonidas Kallipolitis
Technical Manager at AEGIS IT RESEARCH
Marko Grobelnik
senior researcher at Jozef Stefan Institute
Nuria De Lama
Consulting Director at IDC
Pierluigi Petrali
Manufacturin R&D manager at Whirlpool EMEA
Rekha Singhal
Senior Scientist, Head Computing Systems at Tata Consultancy Services
Richard Stevens
Associate Vice President at IDC
Todor Ivanov
Senior Consultant at Lead Consult
Tomás Pariente Lobo
AI, Data & Robotics Unit - Atos Research & Innovation at Atos
Wanling Gao
Assistant Professor at Institute of Computing Technology, Chinese Academy of Sciences
Button to get your ticket now Button to get your ticket now