Körber Prize for German AI Researcher

The German computer scientist Cordelia Schmid is a pioneer in computer-aided image processing. Schmid developed revolutionary new procedures that enable computers to understand image content. Thanks to her algorithms, artificial intelligence (AI) can locate a motif or an object in a database of images within a fraction of a second. The prize winner is currently doing research on systems that can semantically interpret videos and even predict future actions. Her goals include the development of robots that will be able to respond to verbal commands and be employed, for instance, as intelligent assistants in hospitals and in care for the aged.

Artificial intelligence (AI) has developed explosively over the last ten years. In November 2022, the U.S. company OpenAI presented its chatbot ChatGPT, which understands questions and provides elaborate answers. Since then, the topic of AI has dominated the media, the tenor of reports ranging between exuberant hope and warnings of misuse. What is certain is that AI will exert a decisive influence on the coexistence of humans and future economic development.

Cordelia Schmid is one of the most important pioneers in AI research. Already in her dissertation published in 1996, she developed a fundamentally new procedure for image recognition which helped computer vision achieve enormous breakthroughs. In the following years, she also succeeded in developing powerful new computer vision algorithms that became established as the standard. Today, Cordelia Schmid focuses on researching AI systems that use image and sound information to understand the content of videos and predict upcoming actions. This field of technology is an important intermediate step to developing future robotic assistants for hospitals and care homes.

Schmid completed her studies in computer science at the Karlsruhe Institute of Technology in 1992. In 1996, she was awarded her Ph.D. at the Grenoble INP. After working as a postdoc at the British Oxford Robotics Institute, she joined the French National Institute for Research in Digital Science and Technology (Inria) in 1997, where she received her advanced academic qualification (habilitation) in 2001. She has been the research director of this institute since 2004. From 2004 to 2012, she was the co-editor and from 2013 to 2018 the editor-in-chief of the International Journal of Computer Vision. Furthermore, Schmid has worked part-time for Google Research since 2018.

AI research began as early as the 1950s in the U.S.A. In 1957, a team around computer scientist Frank Rosenblatt created a network of artificial neurons that could execute simple logical operations. It was the prototype of artificial neural networks (ANNs), which are the basis of almost all current AI applications. The first bigger successes were achieved by such networks in the 1980s. For example, the networks were trained to distinguish images of apples and pears.

When Cordelia Schmid wrote her outstanding dissertation in 1996, image classification by computers was still in its origins: “The systems in use at the time were only able to recognize simple geometric forms such as circles, triangles, and squares, and only against a uniform background,” Schmid explains. She improved such recognition significantly by having the systems locate distinctive image features. These “local image descriptors” represent the spatial dimensions of the object displayed. This enabled the systems to recall objects when they were displayed from a different perspective or partially hidden. Schmid thus created the foundation for the ability of search machines to find a desired image out of millions of images on the internet within seconds.

After the turn of the millennium, automatic image recognition made large advances and engendered many novel approaches. During this period, Cordelia Schmid designed benchmark tests that could be used to determine the most effective of these numerous new methods.

Cordelia Schmid is currently doing research on vision-language models, for example, on the system VideoBert. Without supervision, VideoBert can analyze video instructions on the internet, such as cooking videos. The AI system can train itself to predict the subsequent actions in the videos. VideoBert works multimodally, which means that it simultaneously examines the image sequences and the corresponding spoken text. In VideoBert, Schmid utilizes the principle of masking: Words or video sequences are omitted that VideoBert then has to guess. Self-supervised learning has the advantage that many thousands of free cooking videos available on the internet can be used for training.

After the self-training, when VideoBert “saw” a bowl of flour and cocoa in previously unfamiliar videos, it could predict that these ingredients would be used to bake a chocolate cake. The system would additionally generate appropriate images of the expected final product. “Coming versions of VideoBert will even be capable of preparing written recipes after viewing new cooking videos,” Schmid added. Based on this multimodal vision-language understanding, the prize winner is also planning to develop intelligent robot assistants for hospitals and care homes in the future.

With the funding from the Körber-Stiftung, one of the projects that Cordelia Schmid wants to pursue is to conceive a kind of vision-enabled competitor to the chatbot ChatGPT. ChatGPT is a deep artificial neural network consisting of particularly numerous layers. It is proficient in processing natural language and can provide suitable answers thanks to training with countless millions of data sets from the internet. Schmid considers the performance of ChatGPT “impressive,” but criticizes that the “model is not self-explanatory, has a limited context window, and cannot learn from experience.” Schmid wants to develop a “really intelligent” bot that can also process visual information and 3D environmental data and later find its way in unfamiliar environments.

Cordelia Schmid is an optimist: “Responsibly developed, AI has the potential to revolutionize our society – just as steam power and electricity once did. AI can help solve some of the most urgent problems in the world, from sustainability to health. I am enthusiastic about the research opportunities that it opens up.”

The Körber European Science Prize in 2023 will be awarded to Cordelia Schmid on September 8, 2023, in the Great Hall of Hamburg City Hall. With a value of one million euros, it is one of the world’s most highly endowed research prizes. Five percent of the prize money is to be used for science communication. Every year since 1985, the Körber-Stiftung has conferred the Körber Prize to honor a breakthrough in the physical or life sciences in Europe. It is awarded for excellent and innovative approaches in research that demonstrate great potential for application. To date, eight recipients of the Körber Prize have subsequently been awarded the Nobel Prize.