Destination Earth: Challenges in Data Streaming and Federation of Compute and Data resources

The Destination Earth flagship project of the European Commission has set out to build an open, reusable, and scalable infrastructure to run digital twins of the earth at unprecedented scale within this decade. The opportunity of using large fractions of the largest EuroHPC installations is posing new challenges for the earth sciences community, reaching from running operational weather codes and climate simulations at highest resolution on shared systems, across different architectures, and in coupled setups of multiple codes that previously have not been coupled tightly, to data and access control federation challenges.

We will report on the first findings and suggestions of the strategic technology agenda team, namely in the area of federation of compute resources, as well as data streaming inside and across sites, and give a lookout on the overall structure and expected outcomes of this technology agenda project (DE-380) in the coming months.