Light Field Super-Resolution via LFBM5D Sparse Coding

In this paper, we propose a spatial super-resolution method for light fields, which combines the SR-BM3D single image super-resolution filter and the recently introduced LFBM5D light field denoising filter. The proposed algorithm iteratively alternates between an LFBM5D filtering step and a back-projection step. The LFBM5D filter creates disparity compensated 4D patches which are then stacked together with similar 4D patches along a 5th dimension. The 5D patches are then filtered in the 5D transform domain to enforce a sparse coding of the high-resolution light field, which is a powerful prior to solve the ill-posed super-resolution problem. The back-projection step then impose the consistency between the known low-resolution light field and the-high resolution estimate. We further improve this step by using image guided filtering to remove ringing artifacts. Results show that significant improvement can be achieved compared to state-of-the-art methods, for both light fields captured with a lenslet camera or a gantry.

Implementation

The MATLAB/C/C++ source code is available on github.

Related publications

Additional results

Visual results complementing the paper are shown below.
We then show in Tables 1 and 2 the average results presented in the paper. The corresponding detailed results are shown in Table 3 and 4.

Lytro Illum dataset

We use in our experiments a Lytro Illum dataset consisting of light fields taken from the EPFL and the INRIA datasets.
These light fields have been further processed with the V-SENSE enhancement pipeline described here.

Visual results

We show in videos below (click on an image to start a video) side by side comparisons of low-resolution (uspampled with a bicubic filter) and super-resolved light fields for different scaling factors ?. On the top left corner of each video is highlighted the sub-aperture image being displayed. Note that some videos may exhibit encoding artifacts.

	?=2	?=3	?=4
Chess
Lego Bulldozer
Lego Knights
Tarot Cards and Crystal Ball (Large Angle)
Bee 2
Bikes
Danger de mort
Vespa

Below, we show results for ?=4 without (left) and with (right) the light field guided filtering described in section 2.2 of the paper to improve back-projection (click on an image to start a video).

Chess	Lego Knights	Bikes	Vespa

Average PSNR results

Values highlighted in bold and italic correspond to the best and second best performing methods respectively for a given magnification factor.

Detailed PSNR results