Better quality for 360-degree video
The MPEG-OMAF standard (officially ISO/IEC 23090-2) is the first international standard for storage and distribution of 360° Video [1], with significant contributions from Fraunhofer HHI’s Video Communication and Applications department. OMAF specifies tile-based streaming of 360° Video which allows for significantly higher resolution at the end device.
Fraunhofer HHI researchers have developed a complete real time transmission chain of the HEVC-based viewport-dependent OMAF video profile using HEVC Tiles, aka tile-based streaming.
The 360-degree video is recorded using HHIs OmniCam-360. The output of the camera is encoded using HHIs HEVC live encoder and transmitted via OMAF to head-mounted display glasses running HHIs OMAF Software player on an Android Smartphone Samsung S7/S8/S9.
Fraunhofer HHI researchers have also developed tools for MPEG-OMAF Viewport-Dependent Streaming from OMAF generator to OMAF player for usages in Apps and MSE capable browsers.
Freely available tools for HEVC-based viewport-dependent OMAF video profile:
OMAF offline Generator
Fraunhofer HHI has developed an OMAF offline Generator for creation of files for HEVC-based viewport-dependent OMAF video profile. It takes as input a raw 360° video, encodes it using the HEVC HM Reference software and packages the encoded bitstreams to OMAF compliant files for local playback and streaming. The OMAF test vectors have been generated with this software.
The software is freely available for testing and evaluation and is a part of our omaf.js GitHub repository.
OMAF JavaScript Player for web browsers with HTML5 Media Source Extensions API
This software allows for playback of OMAF Tile-based content inside a web browser without any plugins. It consists of a pure JavaScript implementation of the OMAF Player for VPD profile files with multi-resolution HEVC tiles. Today, browsers do not provide sufficient support for mp4 files with OMAF metadata and therefore "on the fly" repackaging is required to enable OMAF streaming using a web browser. This player implementation is a proof of concept for OMAF VPD profile using a Safari web browser and its source code is freely available on GitHub repository.
OMAF Android Player
Fraunhofer HHI has developed a standard compliant OMAF VR player for offline playback of VPD video profile on Android. The player uses the OMAF metadata in an mp4 file to properly render the video based on the user’s viewport orientation. It allows playback of the output of the OMAF offline Generator and the OMAF test vectors.
The software can be freely provided for testing and evaluation. If you are interested, please send an Email here.
OMAF test vectors for HEVC-based viewport-dependent OMAF video profile
Fraunhofer HHI provides OMAF test vectors for the local playback or streaming using MPEG-DASH. The test vectors are generated with the HHI OMAF Generator or OMAF live packager and can be freely provided for testing and evaluation. If you are interested, please send an Email here.
Other components for Real-time transmission chain:
HHI HEVC Tile-based encoder
The latest version of the software-based live encoder from Fraunhofer HHI integrates HEVC tiles for efficient encoding and transmission of 360-degree video.
OMAF live packager
For live streaming services Fraunhofer HHI has developed the OMAF live packager software. It receives multi resolution HEVC Tiles from Tile-based encoder, segments and packages the segments with the corresponding OMAF metadata after which the segments can be published on HTTP server.
White Paper:
Live Streaming of Ultra-High Resolution 360-Degree Video with MPEG-OMAF and HEVC Tiles
Flyer:
OMAF Real-time transmission chain
Tools for generation and playback of MPEG-OMAF content
Press:
A new language for VR Streaming: MPEG technically finalized first international standard for VR Streaming
HHI Newsletter:
Fraunhofer HHI presents latest VR and 5G technologies at Mobile World Congress 2018
Event:
IBC 2018
Related Scientific
- D. Podborski, J. Son, G.S. Bhullar, R. Skupin, Y. Sanchez, C. Hellge, T. Schierl, "HTML5 MSE playback of MPEG 360 VR tiled streaming: JavaScript implementation of MPEG-OMAF viewport-dependent video profile with HEVC tiles", Proceedings of the 10th ACM Multimedia Systems Conference, MMSys 2019, 324-327, doi:10.1145/3304109.3323835
- D. Podborski, E. Thomas, M.M. Hannuksela, S. Oh, T. Stockhammer, S. Pham, "360-degree video streaming with MPEG-DASH", SMPTE Motion Imaging Journal 127 (7), 20-27, 2018, doi:10.5594/JMI.2018.2838779
- D. Podborski, E. Thomas, M.M. Hannuksela, S. Oh, T. Stockhammer, S. Pham, "Virtual reality and DASH", International Broadcasting Convention, IBC 2017. [link]
- Yago Sanchez, Robert Skupin, and Thomas Schierl:
Video processing for panoramic streaming using HEVC and its scalable extensions,
Multimedia Tools and Applications, special issue on "Interactive Media: Technology and Experience", pp. 1–29, December 2016,
doi:10.1007/s11042-016-4097-4 - Robert Skupin, Yago Sanchez, Y.-K. Wang, M. M. Hannuksela, J. Boyce, and M. Wien:
Standardization Status of 360 degree Video Coding and Delivery,
Proceedings of IEEE Visual Communications and Image Processing (VCIP 2017), St. Petersburg, FL, USA, December 2017.
doi:10.1109/VCIP.2017.8305083 - Robert Skupin, Yago Sanchez, Dimitri Podborski, Cornelius Hellge, and Thomas Schierl:
Viewport-dependent 360 Degree Video Streaming Based on the Emerging Omnidirectional Media Format (OMAF) Standard,
Proceedings of IEEE International Conference on Image Processing (ICIP 2017), Beijing, China, September 2017.
doi:10.1109/ICIP.2017.8297155 - Yago Sanchez, Robert Skupin, Cornelius Hellge, and Thomas Schierl:
Random Access Point period optimization for viewport adaptive tile based streaming of 360° video,
Proceedings of IEEE International Conference on Image Processing (ICIP 2017), Beijing, China, September 2017.
doi:10.1109/ICIP.2017.8296615 - Robert Skupin, Yago Sanchez, Dimitri Podborski, Cornelius Hellge, and Thomas Schierl:
HEVC Tile Based Streaming to Head Mounted Displays,
Proceedings of the 14th Annual IEEE Consumer Communications & Networking Conference (CCNC 2017), Las Vegas, USA, January 2017.
doi:10.1109/CCNC.2017.7983191 - Robert Skupin, Yago Sanchez, Cornelius Hellge, and Thomas Schierl:
Tile Based HEVC Video for Head Mounted Displays,
Proceedings of the IEEE International Symposium on Multimedia (ISM), San Jose, CA, USA, December 2016.
doi:10.1109/ISM.2016.0089 - Yago Sanchez, Ralf Globisch, Thomas Schierl, and Thomas Wiegand:
Low Complexity Cloud-video-Mixing Using HEVC,
Proceedings of IEEE Consumer Communications and Networking Conference, Las Vegas, NV, USA, ISBN 978-1-4799-2356-4, pp. 213-218, January 2014.
doi:10.1109/CCNC.2014.6866573
References
[1] MPEG, ISO/IEC FDIS 23090-2, Information technology -- Coded representation of immersive media -- Part 2: Omnidirectional media format, San Diego, April 2018