Tutorial on Immersive Imaging Technologies: from Capture to Display
5th December 2019
V-SENSE Research Fellows Dr. Martin Alain, Dr. Cagri Ozcinar, and Dr. Emin Zerman presented the tutorial entitled on “Immersive Imaging Technologies: from Capture to Display” at IEEE ICME 2020, ACM Multimedia 2020, and VCIP 2021. The video of the tutorial presented online at ICME 2020 is available below. The video presented online at ACM MM 2020 can be downloaded from the ACM webpage in the supplemental material section. While there is no video recording of the VCIP 2021 tutorial, the slides are available here.
Abstract
The advances in imaging technologies in the last decade brought a number of alternatives to the way we acquire and display visual information. These new imaging technologies are immersive as they provide the viewer with more information which either surrounds the viewer or helps the viewer to be immersed in this augmented representation. These immersive imaging technologies include light fields, omnidirectional images and videos, and volumetric (also known as free-viewpoint) videos. These different modalities cover the full spectrum of immersive imaging, from 3 degrees of freedom (DoF) to 6DoF, and can be used for virtual reality (VR) as well as augmented reality (AR). Applications of immersive imaging notably include education, cultural heritage, tele-immersion, remote collaboration, and communication. In this tutorial, we cover all stages of the immersive imaging technologies from content capture to display. In particular, we discuss the learning-based approaches on the relevant stages of immersive imaging pipeline. The main concepts of immersive imaging will first be introduced, and creative experiments based on immersive imaging will be presented as a specific illustration of these technologies. Next, content acquisition based on single or multiple camera systems is presented, along with the corresponding data formats. Content coding is then discussed, notably ongoing standardisation efforts, followed by adaptive streaming strategies. Immersive imaging displays are then presented, as they play a crucial role in the user’s sense of immersion. Image rendering algorithms related to such displays are also explained. Finally, perception and quality evaluation of immersive imaging is presented.
Keywords: immersive imaging, emerging media, light fields, omnidirectional videos, volumetric videos, 3DoF, 6DoF
Schedule
The tutorial follows the outline presented below:
- Part I: Immersive Imaging Technologies (~20 minutes)
- Immersion & Tele-Immersion
- Different Imaging Modalities
- Creative Experiments
- Part II: Acquisition and Data Format (~40 minutes)
- Single-camera systems
- Multi-camera systems
- Part III: Content Delivery (~40 minutes)
- Coding
- Adaptive Streaming
- Part IV: Rendering and Display Technologies (~30 minutes)
- Immersive imaging on 2D screens
- HMDs for VR
- HMDs for AR
- Part V: Perception & Quality Evaluation (~50 minutes)
- Visual Perception
- Visual Attention
- Quality Assessment
References
In order to help participants find the relevant publications and sources, we provide the references per tutorial part below: