ELLIS Multimodal Learning Systems Workshop on Multimodal Foundation Models

Multimodal foundation models are a revolutionary class of AI models that provides impressive abilities to generate content (text, images, sound, videos, protein structures, and more), and do so by interactive prompts in a seemingly creative manner. These foundation models are often autoregressive, self-supervised, transformer-based models that are pre-trained on large volumes of data, typically collected from the web. They already form the basis of all state-of-the-art systems in computer vision and natural language processing across a wide range of tasks and have shown impressive few-shot learning abilities. The perceived intelligence and adaptability of models like ChatGPT, Stable Diffusion, Gemini, and GPT4 impress, but their aptitude to produce inaccurate, misleading, or false information (and present it confidently and convincingly) makes them unsuitable for any task of importance and poses serious societal concerns. In this workshop we present recent advances on multimodal foundation models from academia and industry and discuss their impact and implications moving forward.

The workshop is hosted at the Mathematisches Forschungsinstitut Oberwolfach (MFO). Accommodation and meals will take place in Hotel Hirschen, which is within walking distance of the MFO.

The workshop is hosted at the Mathematisches Forschungsinstitut Oberwolfach (MFO). Accommodation and meals will take place in Hotel Hirschen, which is within walking distance of the MFO.