November 21, 2023
Marcus Zepp, research associate in the "Immersive Media and Communication" group of the "Vision and Imaging Technologies" (VIT) department at Fraunhofer Heinrich-Hertz-Institut (HHI) has received the Best Paper Award at this year's Forum Bauinformatik (FBI) in Bochum, Germany. In his paper "Hybrid semantic clustering of 3D point clouds in construction" he presents selected topics from the BMWK funded project BIMKIT (as-built modeling of buildings and infrastructure structures using artificial intelligence (AI) to generate digital twins).
The BIMKIT research project uses so-called Building Information Modeling (BIM) to leverage AI and cloud technologies for the construction industry. For this purpose, the researchers digitally model, combine and record all relevant building data in order to create, for example, a geometrically visualized virtual model (a so-called digital twin). This allows the efficient and sustainable maintenance of buildings and the planning of renovation and deconstruction measures. However, such digitally analyzable models are usually not available for existing buildings. This not only delays the implementation of BIM, but results in a lack of reliable data for the entire construction and housing industry to optimize the building life cycle.
It is therefore crucial to create accurate and up-to-date 3D information of existing buildings in order to make the housing industry more efficient and sustainable. Modern 3D scanning technologies are able to provide the necessary information in the form of 3D point clouds. Unfortunately, these point clouds contain enormous data volumes and do not provide sufficient semantic information.
In his paper, Marcus Zepp addresses this issue by presenting an AI-based semantic segmentation approach for 3D point clouds generated from 2D images with a Structure from Motion (SfM) pipeline. His approach aims to distinguish basic objects such as floor, ceiling, walls or columns in a building during the construction phase. To do this, he uses state-of-the-art neural networks (RandLA-Net) to classify the 3D points. These networks are performed in the hybrid 2D/3D domain and not only in the 3D domain to benefit from the high efficiency and accuracy of the already trained networks in the 2D domain. His approach registers the 2D images to the 3D point cloud and thus fuses the individual 2D segmentation results to produce a consistently segmented 3D point cloud. The award-winning paper thus presents a near-automatic workflow to produce training data for 3D point cloud segmentation in construction.
Marcus Zepp joined Fraunhofer HHI in 2017 and is working on 3D human body reconstruction, virtual, augmented and mixed reality as well as interactive applications, immersive computing and volumetric video.
The FBI has been organized annually since 1989 by a chair of the participating universities of the Arbeitskreis Bauinformatik. This year, young researchers from the field of construction informatics met at the Ruhr-Universität Bochum (RUB) to report on and discuss their work. The FBI caters especially to young researchers and offers them the opportunity to gain first experiences in publishing scientific contributions in the context of a conference.