Wieland Morgenstern is a Research Associate at the Computer Vision and Graphics Group in the Vision and Imaging Technologies Department.
News
Our paper Compact 3D Scene Representation via Self-Organizing Gaussian Grids has been accepted at ECCV 2024 and will be presented in Milano in October!
We presented our paper Animatable Virtual Humans: Learning pose-dependent human representations in UV space for interactive performance synthesis at IEEE VR 2024!
Our paper "X-maps: Direct Depth Lookup for Event-based Structured Light Systems" has been accepted at the CVPR 2023 Workshop on Event-based Vision. Find more information on the project page https://fraunhoferhhi.github.io/X-maps/
Biography
Wieland Morgenstern is performing research in animation of humans from volumetric video, pairing mesh-based approaches with machine learning, and investigating hybrid model-based Neural Radiance Field (NeRF) approaches. A recent focus is a compact 2D-grid based representation of 3D scenes (including humans) with 3D Gaussian Splatting. Another research area is building Spatial Augmented Reality experiences with event cameras.
Before joining Fraunhofer HHI in 2018, Wieland worked as a Computer Vision Engineer at VideoStitch/Orah from 2014-2018. He received a B.Sc. degree in 2012 and a M.Sc. degree in 2015 from Ilmenau Technical University.
Publications
Articles in Journals
2024
- Wieland Morgenstern, Milena Bagdasarian, Anna Hilsmann, Peter Eisert
Animatable Virtual Humans: Learning pose-dependent human representations in UV space for interactive performance synthesis,
IEEE Transactions on Visualization and Computer Graphics, March 2024, doi: 10.1109/TVCG.2024.3372117, arXiv: https://arxiv.org/abs/2310.03615, Open Access
2022
- Oliver Schreer, Markus Worchel, Rodrigo Mauricio Diaz Fernandez, Sylvain Renault, Wieland Morgenstern, Ingo Feldmann, Marcus Zepp, Anna Hilsmann, Peter Eisert
Preserving Memories of Contemporary Witnesses Using Volumetric Video,
i-com, Oldenbourg Wissenschaftsverlag, ISSN: 2196-6826, vol. 21, no. 1, pp. 71-82, April 2022, doi: https://doi.org/10.1515/icom-2022-0015, Open Access, Invited Paper - Alexandra Zimmer, Anna Hilsmann, Wieland Morgenstern, Peter Eisert
Imposing Temporal Consistency on Deep Monocular Body Shape and Pose Estimation,
Computational Visual Media, February 2022, arXiv: https://arxiv.org/abs/2202.03074, Open Access
2020
- Anna Hilsmann, Philipp Fechteler, Wieland Morgenstern, Wolfgang Paier, Ingo Feldmann, Oliver Schreer, Peter Eisert
Going beyond Free Viewpoint: Creating Animatable Volumetric Video of Human Performances,
IET Computer Vision, Special Issue on Computer Vision for the Creative Industries , May 2020, doi: doi: 10.1049/iet-cvi.2019.0786 , arXiv: http://arxiv.org/abs/2009.00922, Open Access
Conference Contributions
2024
- Florian Tim Barthel, Arian Beckmann, Wieland Morgenstern, Anna Hilsmann, Peter Eisert
Gaussian Splatting Decoder for 3D-aware Generative Adversarial Networks,
IEEE Conference on Computer Vision and Pattern Recognition Workshops, Seattle, June 2024, doi: https://doi.org/10.48550/arXiv.2404.10625, arXiv: https://arxiv.org/abs/2404.10625 - Paul Knoll, Wieland Morgenstern, Anna Hilsmann, Peter Eisert
Animating NeRFs from Texture Space: A Framework for Pose-Dependent Rendering of Human Performances,
19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: VISAPP, Rome, Italy, February 2024, doi: 10.5220/0012373800003660, arXiv: https://arxiv.org/abs/2204.06307
2023
- Wieland Morgenstern, Niklas Gard, Simon Baumann, Anna Hilsmann, Peter Eisert
X-maps: Direct Depth Lookup for Event-based Structured Light System,
CVPR 2023 Workshop on Event-based Vision, Vancouver, Canada, June 2023
2021
- Oliver Schreer, Markus Worchel, Rodrigo Mauricio Diaz Fernandez, Sylvain Renault, Wieland Morgenstern, Ingo Feldmann, Marcus Zepp, Anna Hilsmann, Peter Eisert
Preserving Memories of Contemporary Witnesses Using Volumetric Video,
ACM Conference Culture and Computer Science, Berlin, Germany, September 2021, doi: 10.1515/icom-2022-0015, Science Physical und Virtual Spaces
2020
- Jangwoo Son, Serhan Gül, Gurdeep Singh Bhullar, Gabriel Hege, Wieland Morgenstern, Anna Hilsmann, Thomas Ebner, Sven Bliedung, Peter Eisert, Thomas Schierl, Thomas Buchholz, Cornelius Hellge
Split Rendering for Mixed Reality: Interactive Volumetric Video in Action,
SIGGRAPH Asia 2020 XR, Daegu, South Korea, December 2020, doi: 10.1145/3415256.3421491 - Serhan Gül, Dimitri Podborski, Anna Hilsmann, Wieland Morgenstern, Peter Eisert, Oliver Schreer, Thomas Buchholz, Thomas Schierl, Cornelius Hellge
Interactive Volumetric Video from the Cloud,
International Broadcasting Convention (IBC), Amsterdam, Netherlands, September 2020 - Anna Hilsmann, Philipp Fechteler, Wieland Morgenstern, Serhan Gül, Dimitri Podborski, Cornelius Hellge, Thomas Schierl, Peter Eisert
Interactive Volumetric Video Rendering and Streaming,
Culture and Computer Science (KUI) – Extended Reality, Berlin, Germany, ISBN: 978-3-86488-169-5, 2020
2019
- Wieland Morgenstern, Anna Hilsmann, Peter Eisert
Progressive Non-rigid Registration of Temporal Mesh Sequences,
16th ACM SIGGRAPH European Conference on Visual Media Production (CVMP), London, United Kingdom, ISBN: 978-1-4503-7003-5, December 2019, doi: 10.1145/3359998.3369411
Contributions to Standardizations
2021
- David Foerster, Yago Sanchez de la Fuente, Wieland Morgenstern, Cornelius Hellge, Thomas Schierl, Anna Hilsmann, Peter Eisert
[SD] Updated Implementation of MPEG_mesh_linking extension,
m58152, 136th MPEG Meeting online, October 2021 - Yago Sanchez de la Fuente, David Foerster, Wieland Morgenstern, Cornelius Hellge, Anna Hilsmann, Peter Eisert
[42.1][SD] Initial Implementation of MPEG_mesh_linking extension,
m57427, 135th MPEG Meeting online, July 2021 - Yago Sanchez de la Fuente, Wieland Morgenstern, Robert Skupin, Cornelius Hellge
[41.1] Carriage format for Timed Animations,
m56812, 134th MPEG Meeting online, April 2021 - Yago Sanchez de la Fuente, Wieland Morgenstern, Robert Skupin, Cornelius Hellge
[41.1] Animation support for dynamic objects,
m56811, 134th MPEG Meeting online, April 2021 - Gurdeep Singh Bhullar, Yago Sanchez de la Fuente, Wieland Morgenstern, Robert Skupin, Cornelius Hellge
[41.1] On timed animations for glTF,
m56039, MPEG-I, MPEG 133, January 2021 - Gurdeep Singh Bhullar, Yago Sanchez de la Fuente, Wieland Morgenstern, Robert Skupin, Cornelius Hellge
[41.1] On animations and interactivity for volumetric videos,
m56040, MPEG-I, MPEG 133, January 2021
Student Projects Supervised
Student Projects Supervised
- Alexandra Zimmer: Efficient Sparse Stereo Depth Estimation for Event Cameras, Master Thesis (TU Berlin), 2024.
- Paul Knoll: Posed Neural Radiance Fields for Human Animation, Master Thesis (Johannes Kepler University Linz), 2024.
- Kilian L. Krause: Real-Time Rendering of Dynamic Scenes with Neural Radiance Fields, Bachelor Thesis (TU Berlin), 2024.
- Simon Baumann: 3D Reconstruction with Structured Light using Event-Based Vision and a Video Laser Projector, Master Thesis (TU Berlin), 2022.
Projects/Research Topics

SPIRIT
Scalable Platform for Innovations on Real-time Immersive Telepresence
https://www.spirit-project.eu/
October 2022 - September 2025

VoluProf
Volumetric professor for omnipresent and user-optimized teaching in mixed reality
September 2021 – August 2024

INVICTUS
Innovative Volumetric Capture and Editing Tools for Ubiquitous Storytelling
October 2020 – September 2022

Content4All
Personalised Content Creation for the Deaf Community in a Connected Digital Single Market
September 2017 - November 2020
REPLICATE
cReative-asset harvEsting PipeLine to Inspire Collective-AuThoring and Experimentation
January 2016 – December 2018
Awards
CVMP Best Paper Award 2019
The paper "Progressive Non-rigid Registration of Temporal Mesh Sequences" received the Best Paper Award at the 6th ACM SIGGRAPH European Conference on Visual Media Production (CVMP) in London. The prize is sponsored by the ACM Europe Council. Dr.-Ing. Anna Hilsmann, group leader Computer Vision and Graphics at Fraunhofer HHI, and Prof. Dr.-Ing. Peter Eisert, head of the Vision & Imaging Technologies department at Fraunhofer HHI, received the award as co-authors.