Fraunhofer Institute for Telecommunications,
Heinrich-Hertz-Institut, HHI
Video Communication and ApplicationsNewsTechnologies and SolutionsResearch GroupsProjectsPublicationsPeople and Contact

Multiview Video Coding

Figure 1: Stereo video example, showing the same scene from two horizontally aligned perspectives (left and right eye view).

We have been active in developing coding solutions for 3D video for several years and have successfully contributed to the major international standards for 3D video coding. 3D video applications using stereo and multiview video (MVV) have become very popular and our experts have played a leading role in developing the multiview video coding (MVC) extension of H.264/AVC, which is for instance used by the Blu-ray 3D format. Recently, the same concept has been applied to H.265/HEVC for specifying a stereo and multiview video coding extension (MV-HEVC). We have especially contributed to the high-level syntax of MV-HEVC and a comprehensive overview of the technical features can be found in [2].

Figure 2: Multiview video (MVV) processing chain, highlighting the role of multiview video coding.

The role of coding and transmission in the multiview video processing chain is illustrated in figure 2. The acquisition of MVV consists of capturing a real world 3D scene by two or more cameras and optional sender-side processing like rectification and color correction of the views. After encoding the MVV sequences, the bit stream is transmitted to the receiver-side, where the views are decoded and presented on a suitable 3D display.

Figure 3: Typical multiview prediction structure for stereo video, using temporal (black arrows) and inter-view reference pictures (blue arrows) for motion- and disparity-compensated prediction, respectively.

MVC and MV-HEVC are based on the same concept, namely disparity-compensated prediction for increasing the coding efficiency by reducing inter-view redundancies. Given the multi-layer coding design of HEVC, each of the MVV sequences represents a different layer and disparity-compensated prediction is enabled by so-called inter-layer dependencies. In practice, this is achieved by including pictures of other views at the same time instance in the reference picture list. The additional inter-view reference pictures are highlighted in the multiview prediction structure in figure 3, where each view represents a different layer. The left view has no inter-layer dependencies and is consequently fully compliant to HEVC, while the right view depends on the left view. Compared to simulcast coding, bit rate savings of about 70% are achieved by disparity-compensated prediction for dependent views [1][2].

 

NOTE — Links to the specification text and the reference software of MV-HEVC can be found on the HEVC support site.

References

  1. P. Merkle, A. Smolic, K. Müller, and T. Wiegand, Efficient Prediction Structures for Multiview Video Coding, IEEE Transactions on Circuits and Systems for Video Technology, November 2007.
  2. G. Tech, Y. Chen, K. Müller, J.-R. Ohm, A. Vetro, and Y.-K. Wang, Overview of the Multiview and 3D Extensions of High Efficiency Video Coding, IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, pp. 35-49, January 2016, doi: 10.1109/TCSVT.2015.2477935.