In this paper, we present a novel framework for quality control in cinematic VR (360-video) based on Voronoi patches and saliency which can be used in post-production workflows. Our approach first extracts patches in stereoscopic omnidirectional images (ODI) using the spherical Voronoi diagram. The subdivision of the ODI into patches allows an accurate detection and localization of regions with artifacts. Further, we introduce saliency in order to weight detected artifacts according to the visual attention of end-users. Then, we propose different artifact detection and analysis methods for sharpness mismatch detection (SMD), color mismatch detection (CMD) and disparity distribution analysis. In particular, we took two state of the art approaches for SMD and CMD, which were originally developed for conventional planar images, and extended them to stereoscopic ODIs. Finally, we evaluated the performance of our framework with a dataset of 18 ODIs for which saliency maps were obtained from a subjective test with 17 participants.
Sharpness Mismatch Detection
Color Mismatch Detection
Simone Croci, Sebastian Knorr, Lutz Goldmann, Aljosa Smolic
A Framework for Quality Control in Cinematic VR Based on Voronoi Patches and Saliency
International Conference on 3D Immersion, Brussels, Belgium, Dec. 11-12, 2017
360-degree video, also called live-action virtual reality (VR), is one of the latest and most powerful trends in immersive media, with an increasing potential for the next decades. In particular, head-mounted display (HMD) technology like e.g. HTC Vive, Oculus Rift and Samsung Gear VR is maturing and entering professional and consumer markets. On the other side, capture devices like e.g. Facebook’s Surround 360 camera, Nokia Ozo and Google Odyssee are some of the latest technologies to capture 360-degree video in stereoscopic 3D (S3D).
However, capturing 360-degree videos is not an easy task as there are many physical limitations which need to be overcome, especially for capturing and post-processing in S3D. In general, such limitations result in artifacts which cause visual discomfort when watching the content with a HMD. The artifacts or issues can be divided into three categories: binocular rivalry issues, conflicts of depth cues and artifacts which occur in both monocular and stereoscopic 360-degree content production. Issues of the first two categories have been investigated for standard S3D content e.g. for cinema screens and 3D-TV. The third category consists of typical artifacts which only occur in multi-camera systems used for panorama capturing. As native S3D 360-degree video production is still very error-prone, especially with respect to binocular rivalry issues, many high-end S3D productions are shot in 2D 360-degree and post-converted to S3D.
Within the project QualityVR, our group is working on video analysis tools to detect, assess and partly correct artefacts which occur in stereoscopic 360-degrees video production, in particular, conflicts of depth cues and binocular rivalry issues.
Virtual Play: Beckettian Experiments in Virtual Reality Journal Article
Contemporary Theatre review, 28.1 , 2018.
Saliency-Based Sharpness Mismatch Detection For Stereoscopic Omnidirectional Images Inproceedings
14th European Conference on Visual Media Production, London, UK, 2017.
A Framework for Quality Control in Cinematic VR Based on Voronoi Patches and Saliency Inproceedings
International Conference on 3D Immersion, Brussels, Belgium, 2017.
A Modular Scheme for Artifact Detection in Stereoscopic Omni-Directional Images Inproceedings
Irish Machine Vision and Image Processing Conference, Maynooth, Ireland, 2017.