Enabler for high quality VR streaming
Fraunhofer HHI's technique for Compressed Domain Tile Aggregation drastically reduces decoder requirements for VR enabling 16K VR video to be packed in 4K resolution. Video quality and resolution is adapted on-the-fly based on user viewport without transcoding.
Ultra-high-resolution within the user field of view (FoV) is required to foster a sense of immersion in VR video applications. Covering the full 360° surroundings could easily lead to multiple times UHD resolution. This large amount of data poses a major challenge to the whole chain of current state video streaming. The throughput required for streaming of such content over the public internet cannot be provided for many users. Furthermore, most VR relevant devices such as mobile phones contain hardware video decoders that are tailored to conventional FullHD and 4K resolution services.
Hence, a new approach for adaptivity of bitrate and resolution in VR video applications is required. One approach would be to provide a stream per user or FoV orientation, which does not scale well and comes at the cost of a significant encoding overhead. Our solution is facilitation of HEVC Tiles. After encoding, this approach allows emphasis of the current user FoV within the 360° video through higher quality or resolution on-the-fly. This is done without transcoding by our technique for aggregation of tiles of different quality or resolution into a single common bitstream. The underlying technique for Compressed Domain Tile Aggregation merges separate HEVC encoded videos into a single HEVC bitstream through lightweight rewriting.
Using Fraunhofer HHI’s technique, a suitable bitstream for each user can be easily generated on-the-fly without intensive processing on the server or client side. This allows a more efficient bitrate usage, because video content outside the users current FoV can be transmitted with low quality or resolution. Therefore, our technique enables an efficient decoder utilisation providing 16K VR video quality within the user FoV on devices with a 4K decoder. Furthermore, our solution enables usage of a single video decoder on end devices. Complete and market ready streaming solutions can be implemented based on well-established standard families such as MPEG ISO Base Media File Format and MPEG DASH which are fully compatible to our Compressed Domain Tile Aggregation technique.
