Fraunhofer Institute for Telecommunications,
Heinrich-Hertz-Institut, HHI
Video Communication and ApplicationsNewsTechnologies and SolutionsResearch GroupsProjectsPublicationsPeople and Contact

3D Video Processing

The Image & Video Coding and the former 3D Coding group have been active in developing 3D video processing technology related to 3D coding, including 3D computer graphics models and free viewpoint video.

3D processing chain for multiview video plus geometry (MVG), using surface mesh reconstruction, mesh coding, and multi-texture rendering.

Multiview video plus geometry (MVG) is an alternative 3D video format that uses polygon meshes for representing scene geometry as known from computer graphics. In this regard, we have played an important role in developing the MPEG-4 / FAMC (Frame-based Animated Mesh Compression) standard.

Example of scene geometry reconstruction steps from voxel-based shape-from-silhouette to refined surface mesh.

Moreover, we have developed 3D processing solutions for the following topics:

  • Scene geometry (surface mesh) reconstruction from multi-camera video data, Including visual hull segmentation and voxel-based shape-from-silhouette approaches.
  • View-dependent multi-texture rendering, using weighted virtual view interpolation of calibrated texture views.
  • Advanced view synthesis for multiview video plus depth (MVD) and layered depth video (LDV), enhancing depth-image-based rendering (DIBR) by reliability layers.

References

  1. K. Müller, A. Smolic, M. Kautzner, T. Wiegand, Rate-Distortion Optimization in Dynamic Mesh Compression, IEEE International Conference on Image Processing (ICIP 2006), pp.533-536, 8-11 Oct. 2006.
  2. D. Marpe, H. Kirchhoffer, K. Müller, T. Wiegand, Efficient representation and coding of prediction residuals and parameters in frame-based animated mesh compression, IEEE International Conference on Image Processing (ICIP 2008), pp.2680-2683, 12-15 Oct. 2008.
  3. A. Smolic, K. Müller, P. Merkle, M. Kautzner, T. Wiegand, 3D video objects for interactive applications, 13th European Signal Processing Conference, pp.1-4, 4-8 Sept. 2005.
  4. A. Smolic, K. Müller, P. Merkle, T. Rein, M. Kautzner, P. Eisert, T. Wiegand, Free viewpoint video extraction, representation, coding, and rendering, IEEE International Conference on Image Processing (ICIP 2004), pp.3287-3290 Vol. 5, 24-27 Oct. 2004.
  5. A. Smolic, K. Müller, K. Dix, P. Merkle, P. Kauff, T. Wiegand, Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems, IEEE International Conference on Image Processing (ICIP 2008), pp.2448-2451, 12-15 Oct. 2008.