Depth Map Estimation from Light Field Images

7th July 2017
Depth Map Estimation from Light Field Images

Abstract

Depth map estimation is a crucial task in computer vision, and new approaches have recently emerged taking advantage of light fields, as this new imaging modality captures much more information about the angular direction of light rays compared to common approaches based on stereoscopic images or multi-view. We propose a novel depth estimation method from light fields based on existing optical flow estimation methods. The optical flow estimator is applied on a sequence of images taken along an angular dimension of the light field, which produces several disparity map estimates. Considering both accuracy and efficiency, we choose the feature flow method as our optical flow estimator. Thanks to its spatio-temporal edge-aware filtering properties, the different disparity map estimates that we obtain are very consistent, which allows a fast and simple aggregation step to create a single disparity map, which can then converted into a depth map. Since the disparity map estimates are consistent, we can also create a depth map from each disparity estimate, and then aggregate the different depth maps in the 3D space to create a single dense depth map.

This paper received the Jonathan Campbell Best Paper Award at The Irish Machine Vision and Image Processing Conference, 2017.

Pipeline

Related publications

2020

Zerman, Emin; O’Dwyer, Néill; Young, Gareth W.; Smolic, Aljosa

A Case Study on the Use of Volumetric Video in Augmented Reality for Cultural Heritage Conference

Proceedings of the 11th Nordic Conference on Human-Computer Interaction (NordiCHI '20), Association for Computing Machinery (ACM), Tallinn, Estonia, 2020, ISBN: 978-1-4503-7579-5.

Abstract | Links | BibTeX

O’Dwyer, Néill; Young, Gareth W.; Johnson, Nicholas; Zerman, Emin; Smolic, Aljosa

Mixed Reality and Volumetric Video in Cultural Heritage: Expert opinions on augmented and virtual reality Inproceedings

In: Rauterberg, Matthias (Ed.): Culture and Computing, pp. 195 – 214, Human Computer Interaction International Springer, Copenhagen, 2020, ISBN: 978-3-030-50267-6.

Abstract | Links | BibTeX

Moynihan, Matthew; Pagés, Rafael; Smolic, Aljosa

A Self-regulating Spatio-Temporal Filter for Volumetric Video Point Clouds Book Chapter

In: vol. 1182, pp. 391-408, Springer International Publishing, 2020, ISBN: 978-3-030-41590-7.

Abstract | Links | BibTeX

2019

Zerman, Emin; Valenzise, Giuseppe; Smolic, Aljosa

Analysing the Impact of Cross-Content Pairs on Pairwise Comparison Scaling Inproceedings

In: 11th International Conference on Quality of Multimedia Experience (QoMEX 2019), IEEE 2019.

Abstract | Links | BibTeX

Moynihan, Matthew; Pagés, Rafael; Smolic, Aljosa

Spatio-Temporal Upsampling for Free Viewpoint Video Point Clouds Conference

In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications VISAPP, vol. 5, SciTePress, 2019, ISBN: 978-989-758-354-4.

Abstract | Links | BibTeX

O’Dwyer, Néill; Johnson, Nicholas

Exploring volumetric video and narrative through Samuel Beckett’s Play Journal Article

In: International Journal of Performance Arts and Digital Media, 2019, ISSN: 1479-4713.

Abstract | Links | BibTeX

2018

O’Dwyer, Néill; Ondřej, Jan; Pagés, Rafael; Amplianitis, Konstantinos; Smolić, Aljoša

Jonathan Swift: Augmented Reality Application for Trinity Library ’s Long Room Conference

International Conference on Interactive Digital Storytelling (ICIDS 2018) 2018.

Abstract | Links | BibTeX

Gao, Pan; Ozcinar, Cagri; Smolic, Aljosa

Optimization of Occlusion-Inducing Depth Pixels in 3-D Video Coding Conference

IEEE International Conference on Image Processing (ICIP 2018), Athens, Greece 2018.

Abstract | Links | BibTeX

O’Dwyer, Néill; Johnson, Nicholas; Pagés, Rafael; Ondřej, Jan; Amplianitis, Konstantinos; Bates, Enda; Monaghan, David; Smolic, Aljoša

Beckett in VR: exploring narrative using free viewpoint video Inproceedings

In: Proceeding SIGGRAPH '18, ACM SIGGRAPH ACM SIGGRAPH, New York, NY, USA, 2018, ISBN: 978-1-4503-5817-0 .

Abstract | Links | BibTeX

Pagés, Rafael; Amplianitis, Konstantinos; Monaghan, David; Ondrej, Jan; Smolic, Aljosa

Affordable Content Creation for Free-Viewpoint Video and VR/AR Applications Journal Article

In: Journal of Visual Communication and Image Representation, vol. Volume 53, pp. 192-201, 2018.

Abstract | Links | BibTeX

O’Dwyer, Néill; Johnson, Nicholas

Virtual Play: Beckettian Experiments in Virtual Reality Journal Article

In: Contemporary Theatre review, vol. 28.1, 2018.

Abstract | Links | BibTeX

2017

O’Dwyer, Néill; Johnson, Nicholas; Bates, Enda; Pagés, Rafael; Ondrej, Jan; Amplianitis, Konstantinos; Monaghan, David; Smolic, Aljosa

Virtual Play in Free-viewpoint Video: Reinterpreting Samuel Beckett for Virtual Reality Inproceedings

In: 16th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 262-267, IEEE Xplore digital library, 2017.

Abstract | Links | BibTeX

Results

Numerical results between our method (red) and state-of-the-arts on synthetic HCI dataset

Visual results on synthetic HCI dataset

Visual results on real world EPFL dataset