Fraunhofer Institute for Telecommunications,
Heinrich-Hertz-Institut, HHI
Video Communication and ApplicationsNewsTechnologies and SolutionsResearch GroupsProjectsPublicationsPeople and Contact

Versatile Video Coding (VVC): High-level Syntax and System integrations

Introduction

Considerable progress has been made in the field of video coding [e.g., versatile video coding (VVC)] for the purpose of improving compression efficiency (e.g., reducing the amount of data needed to store or transmit a video). Previous generations of codecs have consistently managed to boost compression efficiency by a factor of roughly two. Just as relevant, however, is the progress that has been made in enabling video applications through design of the codec high-level syntax and the system integration. Some of these advances are already taken for granted (e.g., fast forwarding or streaming a video over the internet).

The Multimedia Communications group is exploring the limits of existing video codecs and working towards enabling new applications through developing technologies, contributing to standardization, and internal product work.1

Bitstream extraction and merging

There is an increasing demand for videos that convey 360-degree material (e.g., to capture live sports events). However, these high-resolution videos require immense amounts of decoder resources. To avoid this issue, we contribute to aspects of VVC, which allow the strategic enhancement or reduction of video resolution based on a user’s needs (i.e., via advanced picture partitioning methods). For instance, if a user has a certain viewing orientation, we reduce the resolution outside of the field of view, reducing the unnecessary consumption of resources. This method is demonstrated in the figure below.

Conformance of bitstreams and decoders

Specifications for video coding standards are described in a lengthy, detailed, and jargon-enriched document that is available in the JVET Document Management System. To ensure that the guidelines contained in this document are followed by devices on the market, the compatibility of bitstreams and decoders must be verified. The VVC design contains features that facilitate such compatibility evaluation for all features of the codec (e.g., the picture partitioning methods described above). Another example of conformance testing involves temporal scalability (i.e., different frame rates within a single coded video). This phenomenon is demonstrated in the figure below. Digital Video Broadcasting (DVB), which is responsible for video broadcasting in many regions of the world (including Europe), faced issues when supporting high frame rates (HFR) with the previous codec generation [high-efficiency video coding (HEVC)]. In collaboration with an industry partner, we incorporated feedback from DVB to resolve this issue when developing VVC (see the list of standardization contributions for more details).

Publications

  1. R. Skupin, Y. Sanchez, L. Jiao, C. Hellge, T. Schierl, "Tile-Based Rate Assignment for 360-Degree Video Based on Spatio-Temporal Activity Metrics",
    20st IEEE International Symposium on Multimedia (ISM 2018), Taichung, Taiwan, December 2018. [download]
  2. Y. Sanchez, G. Bhullar, R. Skupin, C. Hellge, T. Schierl, "Delay Impact on MPEG OMAF’s Tile-Based Viewport-Dependent 360° Video Streaming",
    IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 1, pp. 18-28 , March 2019. [download]

[more]

Standardization

  1. B. Bross, J. Chen, S. Liu, Y.-K. Wang, "Versatile Video Coding (Draft 8)", JVET-Q2001, 2020. [download]
  2. R. Skupin, Y. Sanchez, K. Suerhing (Fraunhofer HHI), J. Boyce, L. Xu (Intel), "Subpicture level info for extraction and merging, AHG17", JVET-P0984, m51407, Geneva, October 2019.[download]
  3. Y. Sanchez, R. Skupin, K. Suerhing, T. Schierl, "On HRD for VVC: Splicing, Open GOP and DRAP support, AHG17", JVET-P0446, m50414, Geneva, October 2019. [download]
  4. K. Suerhing, R. Skupin, Y. Sanchez, T. Schierl, "Independent parsing of parameter sets, AHG17", JVET-O0490, m48615, Gothenburg, June 2019. [download]
  5. Y.-K. Wang, Hendry, J. Chen, S. Hong (Futurewei), J. Boyce, L. Xu (Intel), R. Skupin, Y. Sanchez, K. Suehring (Fraunhofer HHI), M. Hannuksela (Nokia), "Text for subpicture agreements integration, AHG12", JVET-O1143, m49823, Gothenburg, June 2019. [download]
  6. V. Drugeon (Panasonic), R. Skupin, Y. Sanchez (Fraunhofer HHI), "Harmonized proposal on HRD parameters for temporal scalability", JVET-N0867, m47989, Geneva, March 2019. [download]

[more]

Relevant links

Collaboration partners

1 These activities take place within the framework of the Joint Video Experts Team, a collaborative effort by Video Coding Experts Group (Q6/16) and International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) Joint Technical Committee (JTC) 1 SC29/WG11 (Moving Picture Experts Group) in charge of developing the VVC.