Compressed video data are growing at a faster rate than ever before. Already today, over 80% of the worldwide Internet traffic and over half of the traffic over mobile networks are video-related. This demonstrates the need for even more efficient compression, which goes beyond the current state-of-the-art High Efficiency Video Coding standard (HEVC/H.265). As a response to this demanding challenge, the ITU-T Video Coding Expert Group (VCEG) and the ISO/IEC Moving Pictures Expert Group (MPEG) have been working together to develop the Versatile Video Coding standard (VVC/H.266). VVC provides a 50% bit-rate reduction over HEVC and its first version was finalized in July 2020. Unlike previous video coding standards, VVC already includes in its first version specific coding tools and systems functionalities for a wide range of applications such as resolution-adaptive multi-rate video streaming services, 360-degree immersive video, ultra low-delay streaming, gaming, and screen sharing.

At IBC 2022, Fraunhofer HHI will showcase the latest versions of its optimized VVC open-source software implementations together with their integration into third-party applications. This includes the fastest highly efficient open source VVC encoder (VVenC) as well as the fastest fully compliant open source VVC decoder (VVdeC). Based on VVenC, Fraunhofer HHI will be demonstrating how VVC achieves significant bit-rate reductions over HEVC within practical use. This ranges from integrating VVenC into a cloud-based platform for adaptive multi-resolution streaming to using VVenC for efficiently encode 8K high dynamic range (HDR) video content. Additionally, based on VVdeC, Fraunhofer HHI will be presenting various applications for live playback of VVC encoded video on desktop and mobile platforms as well as in web browsers.

Volumetric video has become increasingly popular in recent years and is considered as the emerging key technology for the representation of real persons as dynamic 3D objects enabling truly immersive applications for Mixed- and Virtual-Reality with 6 Degrees of Freedom.

Fraunhofer HHI presents advanced techniques for interactive volumetric video streaming at this year’s IBC. The range of applications as well as the level of immersion can be significantly increased by interactivity of the user with the volumetric assets of persons. This requires online and real-time manipulation of the mesh sequence. A good example is head rotation of the volumetric asset of the captured person according to the user’s position in space. This will allow eye contact and increase the immersive experience as the user feels being looked at. In order to avoid computationally complex real-time manipulation of the volumetric asset in the end-device, processing is performed in the edge cloud. In this context, the Fraunhofer HHI has developed a cloud rendering system for interactive streaming of volumetric videos that drastically reduces the problem of high processing requirements on the client side. The rendering of a complex scene with interactive objects is performed in the cloud and finally sent to the user as a simple 2D video and intergrated into a Mixed-Reality Scene on phone, tablet or Mixed-Reality glasses.