We develop modern computer vision and AR-based technologies for medical applications. We apply state-of-the-art image and video analysis methods, e.g. for intraoperative assistance systems. Current work includes real-time 3D measurement for implant fitting, hyperspectral-based tissue analysis, videobased blood flow and vital sign monitoring as well as surgical instrument detection and phase recognition. Additionally, also human-machine interaction and human-in-the-loop plays an important role.
Current solutions cover the entire surgical pipeline.
Intraoperative AR/MR Tools
We work on interactive and multimodal-based augmented reality (AR) systems for computer-assisted surgery in the context of ear, nose and throat (ENT) treatment. The processing pipeline uses fully digital stereoscopic imaging devices, which support multispectral and white light imaging to generate high resolution image data, and consists of several AR and mixed-reality (MR) modules with the ability of remote communication.
Camera-based analysis of vital parameters and local blood flow
The extraction of heart rate and other vital parameters from surgical video recordings has attracted much attention. We develop tools for the extraction of global vital parameters, such as heart and respiratory rate, or even locally resolved perfusion intensity and blood flow from remote photoplethysmography (rPPG) signals. Applications range from intraoperative assistance during transplantation to patient monitoring for contactless intensive care services.
Hyperspectral Capturing and Intraoperative Tissue Analysis
We develop and investigate several hyperspectral camera setups to monitor the different optical behavior of tissue types in vivo. Especially, we address the automatic spectral examination of human tissues using multispectral imaging with promising potential for automatic visualization during surgery. The aim of this work is to collect and analyze behaviors of specific tissues and organs to use that knowledge for intraoperative guidance. For differentiation and visualization, different model-based as well as artificial methods are investigated to open up optical opportunities during surgery.
Human-Machine Interactions
- Read more about our new current projects NeoSpek, KIPos, KIARA, EndoScan3D.
- Read more about hyperspectral imaging, blood flow and vital sign analysis and 3D reconstruction and measurement of the Computer Vision and Graphics Group.
- Read more about human-machine interaction of the Interactive & Cognitive Systems Group.