What do tiles have to do with Virtual Reality Streaming?

In September 2017, Fraunhofer HHI presented a new media profile for streaming Virtual Reality content based on the use of HEVC tiles at the IBC in Amsterdam. The video transmission standard OMAF (Omnidirectional Media Application Format) is currently being extended to cover this tile-based streaming and thus to allow the transmission of ultra-high resolution content on mobile devices.

When viewing Virtual Reality content, the viewer is always only looking viewport-dependent at the material, which means he or she sees angle-dependently only a section of the video. The question arises, why all the 360-degree video should always have to be transmitted and decoded in full resolution, especially since these traditional transmission methods have a negative impact on video quality in the visible range.

A pioneering solution for the streaming of 360-degree VR videos is the viewport-dependent streaming based on HEVC tiles implemented by Fraunhofer HHI. The entire 360-degree video is divided into rectangles that are encoded independently of each other. This so-called tile-based streaming ensures that the image is high-resolution in the direction of the user's gaze and low-resolution outside the direction of gaze, thus achieving high resolution with low memory requirements. For the first time, the end device decides which tiles are downloaded in high resolution and which in low resolution.

Through its participation in the MPEG Expert Group for Video Compression, Fraunhofer HHI is actively involved in integrating viewport-dependent VR video streaming into the current OMAF transmission standard.