Harnessing Vision Foundation Models: Fraunhofer HHI launches new project REFRAME

The Fraunhofer Heinrich-Hertz-Institut (HHI), together with its partners, has launched the new consortium project titled REFRAME (Flexible, Resilient, and Efficient Machine Learning Models). The goal of the project is to enhance the explainability of Artificial Intelligence (AI), thereby improving its trustworthiness and enabling its safe application in high-stakes sectors like healthcare and mobility. Funded by the Federal Ministry of Education and Research (BMBF) with a budget of 1.39 million euros, the project will run for three years, from October 1, 2024, to September 30, 2027.

The advent of "Vision Foundation Models" (VFM) has marked a transformative shift in deep learning, offering significant advance in AI applications such as image classification, object detection, and facial recognition. These powerful, pre-trained models are built on large image datasets and form the basis for various AI applications.

Despite their impressive performance, important concerns persist regarding the trustworthiness of these technologies. In particular, there is limited understanding of how the accuracy and reliability of a model might degrade applied outside its original training domain. Addressing these concerns is critical for the safe deployment of VFM-based applications in safety-critical areas.

The REFRAME team will examine the current limitations of VFMs, particularly around explainability and uncertainty quantification, and will develop new methods to enhance their trustworthiness and transparency. The project will also focus on creating efficient methods to adapt these models to specific domains and tasks, even with limited training data.

The results of the REFRAME project will lay the groundwork for the robust and flexible use of VFMs, opening up new opportunities for scientific advancements, societal innovations, and economic growth.

In addition to Fraunhofer HHI, the Leibniz Institute for Agricultural Engineering and Bioeconomy (ATB) and the University of Wuppertal are key partners in the project.