ELIAS Consortium Members Engage in Cutting-Edge Exploration of Multimodal Foundation Models at ELLIS Workshop

The ELIAS Project Coordinator Niculae Sebe and Consortium Member Cees Snoek, both directors of the ELLIS Program on Multimodal Learning Systems, lead discourse on advancements in Multimodal Learning Systems.

January 17 – 19, 2024, Oberwolfach-Walke, Germany – In a groundbreaking event, members of the European Lighthouse of AI for Sustainability (ELIAS) consortium gathered at the Mathematisches Forschungsinstitut Oberwolfach (MFO) in Oberwolfach-Walke, Germany, for an important workshop focusing on Multimodal Foundation Models. The workshop, organised by ELIAS Coordinator Niculae Sebe and Consortium Member Cees Snoek, both directors of the ELLIS Program on Multimodal Learning Systems, sought to delve into the intricacies of this paradigm-shifting facet of artificial intelligence.

Workshop Deliberations: At the epicenter of the discussions were Multimodal Foundation Models, a paradigmatic class of artificial intelligence models renowned for their prowess in synthesizing multifaceted content, ranging from textual information to images, sound, and videos, often via interactive prompts. Models such as ChatGPT, Stable Diffusion, Gemini, and GPT4 were scrutinised for their perceived intelligence, adaptability, and, notably, the ethical concerns associated with their capacity to produce potentially misleading information. The workshop facilitated an international assembly of researchers from both academia and industry to scrutinise recent advancements, deliberate on their impact, and envisage future implications.

ELLIS Workshop_1
ELLIS Workshop_4
ELLIS Workshop_5

The ELIAS Project Coordinator Niculae Sebe and Consortium Member Cees Snoek, leading figures in the ELLIS Program Multimodal Learning Systems, guided the discussions throughout the workshop. Their leadership underscored the collaborative efforts propelled by ELLIS Programs, fostering interdisciplinary engagements across European scientific communities to propel the frontiers of artificial intelligence.

ELLIS Workshop_6

The ELIAS Project Coordinator Niculae Sebe and Consortium Member Cees Snoek, leading figures in the ELLIS Program Multimodal Learning Systems, guided the discussions throughout the workshop. Their leadership underscored the collaborative efforts propelled by ELLIS Programs, fostering interdisciplinary engagements across European scientific communities to propel the frontiers of artificial intelligence.

ELLIS Workshop_6

In-Depth Program Structure: The workshop unfolded through meticulously structured sessions, encompassing Foundation Models, Vision & Language, Generative AI, and Multimodality. Renowned experts, including Karteek Alahari, Cristian Sminchisescu, Xavier Alameda-Pineda, Elisa Ricci, Hakan Bilen, Shaogang Gong, Jakob Verbeek, Juergen Gall, Ioannis Patras, Dima Damen, and Dimosthenis Karatzas, shared their insights and research findings.

European Union’s Scientific Support: This scientific endeavor received financial backing from the European Lab for Learning and Intelligent Systems (ELLIS), aligning with the European Union’s Horizon 2020 research and innovation programme under grant agreement No 951847, known as ELISE – European Network of AI Excellence Centres.

Future Trajectory: The fruitful discussions and presentations at the workshop are expected to pave the way for further advancements in the field of Multimodal Foundation Models. The collaborative efforts of ELIAS and ELLIS Programs continue to drive progress in artificial intelligence, ensuring its responsible and sustainable development.

For more information:

For a detailed breakdown of the workshop’s scientific program, refer to The ELLIS Multimodal Learning Systems Workshop on Multimodal Foundation Models

For a broader understanding of the ongoing initiatives and endeavors within the ELLIS Program, we invite you to explore the full spectrum of activities at ELLIS Programs.