OpenWebSearch.eu – Web-data at your fingertips for Generative AI, Data Analytics and Search
OpenWebSearch.eu project aims to develop an open European infrastructure for web search. The project aspires to contribute to Europe’s digital sovereignty and help promote an open human-centred search engine market.
OpenWebSearch.eu is designing the core of the European Open Web Index, based on open source software and deployed across various high performance computing centres in Europe. The Open Web Index is particularly crucial for the provision of state-of-the-art web search services and for European innovations, such as AI/Large Language Models.
We build an infrastructure to establish and maintain relevant web-data sets at Peta- byte-scale, including raw web-data, web-index and large multilingual text corpora as well as multimodal data sets at the various EuroHPC centres: ready-to-use by researchers, start-ups, innovators and industry to build value oriented, trustworthy AI/LLM and search solutions in and for Europe. We are confident that sharing these data sets across Europe by hosting them very close to or directly at the HPC facilities would be the most effective way.
An Open Web Index will also promote an open, human-centred search engine market in Europe. It will thus contribute decisively to a shift from the current monopolised web search to a diverse search engine market. In addition, a first model for a governance structure and a financial model for the Open Search ecosystem is currently being developed. Ethical, legal, and societal considerations will also be discussed at this session.
