elias-ai

ELLIS Multimodal Learning Systems Workshop on Multimodal Foundation Models
2024-01-17 - 2024-01-19 | Mathematisches Forschungsinstitut Oberwolfach (MFO), Schwarzwaldstraße 9-11, Oberwolfach-Walke, 77709, Germany
Theme

Multimodal foundation models are a revolutionary class of AI models that provides impressive abilities to generate content (text, images, sound, videos, protein structures, and more), and do so by interactive prompts in a seemingly creative manner. These foundation models are often autoregressive, self-supervised, transformer-based models that are pre-trained on large volumes of data, typically collected from the web. They already form the basis of all state-of-the-art systems in computer vision and natural language processing across a wide range of tasks and have shown impressive few-shot learning abilities. The perceived intelligence and adaptability of models like ChatGPT, Stable Diffusion, Gemini, and GPT4 impress, but their aptitude to produce inaccurate, misleading, or false information (and present it confidently and convincingly) makes them unsuitable for any task of importance and poses serious societal concerns. In this workshop we present recent advances on multimodal foundation models from academia and industry and discuss their impact and implications moving forward.

Location

The workshop is hosted at the Mathematisches Forschungsinstitut Oberwolfach (MFO). Accommodation and meals will take place in Hotel Hirschen, which is within walking distance of the MFO.

Event Programme

Programme

Wednesday January 17

15:00 - 19:00
Arrival of attendees, time for socializing and discussion

19:00 - 22:00
Opening dinner at Hotel Hirschen

Thursday January 18

08:00 - 09:30
Breakfast at Hotel Hirschen

09:30 - 11:00

Morning Session I: Foundation Models (Chair: Yiannis Kompatsiaris)
FoMO without FOMO by Karteek Alahari (Inria)
Towards 3D Human Foundation Models by Cristian Sminchisescu (Google)
What multimodal foundation models cannot perceive by Cees Snoek (University of Amsterdam)
Are Foundation Models the tool for Social Embodied AI? by Xavier Alameda-Pineda (Inria)

11:00 - 11:30
Coffee break

11:30 - 12:30

Morning Session II: Vision & Language (Chair: Dima Damen)
Vocabulary-free Image Classification by Elisa Ricci (University of Trento)
Coreference resolution in narrated images by Hakan Bilen (The University of Edinburgh)
Vision-Language Self-Supervised Learning by Shaogang Gong (Queen Mary, University of London)

12:30 - 14:00
Lunch break at Hotel Hirschen

14:00 - 15:30

Afternoon Session I: Generative AI (Chair: Karteek Alahari)
Images & text: alignment, generation and compression by Jakob Verbeek (Meta)
Measuring the Quality of Generative Neural Networks - An Unsolved Problem by Juergen Gall (University of Bonn)
Controllable generation for Analysis and Synthesis by Ioannis Patras (Queen Mary, University of London)
Improving Fairness using Vision-Language Driven Image Augmentation by Nicu Sebe (University of Trento)

15:30 - 16:00
Coffee break

16:00 - 17:00

Afternoon Session II: Multimodality (Chair: Xavier Alameda-Pineda)
Multi-modality in Egocentric Vision - Contradictory and complementary signals by Dima Damen (University of Bristol)
Multimodal LLMs for Document Understanding by Dimosthenis Karatzas (Universitat Autónoma de Barcelona)
Large Multimodal Models for Media and Journalism by Yiannis Kompatsiaris (Information Technologies Institute, CERTH)

17:00 - 18:30
Discussion Session (Chairs: Cees Snoek & Nicu Sebe)

19:00 - 22:00
Dinner at Hotel Hirschen

Friday January 19

08:00 - 10:00
Breakfast at Hotel Hirschen

10:00 onwards
Departure

Event Footer

Details
Start: January 17
End: January 19
Event Category: Workshops

Add to Calendar

Website
Multimodal FoMO Workshop
Organiser
ELLIS, ELISE & ELIAS

Venue
Location: Mathematisches Forschungsinstitut Oberwolfach (MFO)
Schwarzwaldstraße 9-11, Oberwolfach-Walke, 77709 Germany
Phone: +49 (0) 7834 979-0
View Venue Website

© 2024 ELLIS, ELISE & ELIAS. All rights reserved.

AI Excellence Lecture on Hybrid & Causal Machine learning in the Earth sciences
2024-01-16 17:00 - 18:00 (Europe/Rome) | ONLINE
Abstract: Most problems in Earth sciences aim to do inferences about the system, where accurate predictions are just a tiny part of the whole problem. Inferences mean understanding variables relations, deriving models that are physically plausible, that are simple parsimonious, and mathematically tractable. While machine learning models excel as approximators, they often disregard fundamental physics laws, compromising consistency and confidence. To address these challenges, we propose exploring the interplay between domain knowledge and machine learning. Physics-aware and hybrid machine learning models are seen as necessary steps toward understanding the data-generating process, with causality offering significant advancements. I will discuss recent hybrid and causal machine learning methodologies to attain consistent and explainable results. This work outlines a collective, long-term AI agenda for developing algorithms that can discover knowledge in the Earth system.
Lecturer: Professor Gustau Camps-Valls is a Full Professor in Electrical Engineering at the Universitat de Valencia. He is an expert in machine learning algorithms for geosciences and remote sensing data analysis, having published extensively. He has a Ph.D. in Physics and is an IEEE Distinguished Lecturer. He has received two European Research Council (ERC) grants and holds a Hirsch’s index h=88 (Google Scholar). He is also a Highly Cited Researcher since 2020. Gustau has achieved significant recognition with numerous awards and honors, including IEEE Fellow (2018), ELLIS Fellow (2019), Fellow of the European Academy of Sciences (EurASc), the Academia Europeae (AE), and the Asia-Pacific Artificial Intelligence Association (AAIA) all in 2021.
About AIDA: The four ICT-48 networks (AI4Media, ELISE, HumanE-AI NET, TAILOR) and the VISION project joined forces and, under the joint initiative of VISION and AI4Media, founded a new joint instrument to support a world-level AI education and research programme.
The International AI Doctoral Academy (AIDA) has been created for offering access to knowledge and expertise and attracting PhD talents in Europe. AIDA offers free, top-notch AI Excellence Lecture Series, featuring senior experts and promising juniors, encouraging active participation, and providing easy access through multiple channels
🔗 For more information: Artificial Intelligence Doctoral Academy – AI Excellence Lecture Series

Event Footer

Details
Date: January 16
Time: 17:00 - 18:00
Event Category: Lectures & Seminars

AI
ML

Add to Calendar

Website

Hybrid Causal Machine Learning in the Earth Sciences

Venue
Location: ONLINE

Organiser
AIDA – Artificial Intelligence Doctoral Academy
View Organiser Website

© 2024 AIDA – Artificial Intelligence Doctoral Academy. All rights reserved.
