V-SENSE research seminar presented by new V-SENSE team member, Dr. Pan Gao!

Presenter: Dr. Pan Gao, Postdoctoral Research Fellow in V-SENSE.

Date & Time: 4pm, Wednesday, 7^th March.

Venue: Large Conference Room, O’Reilly Institute (ORI LCR)

Title:

Analysis of Packet-Loss-Induced Distortion in Decoded Depth Map and Its Application to Error Control for 3-D Visual Communications

Abstract:

With the growing demand for a more immersive experience, depth-map-based 3-D video representation has gained increasing popularity from both industry and academia alike, with which an arbitrary number of  views can be synthesized at the receiver using the transmitted textures and depths via depth-image-based rendering technology. As the depth map only provides geometry information for view synthesis instead of being viewed by end users, the packet-loss-errors in depth map introduce a significantly different type of error (i.e., geometry error) from the ordinary intensity errors, which would lead to unexpected holes and overlaps in the rendered virtual views, thus substantially deteriorating the overall quality of the views presented to users. Therefore, it is desirable to provide an in-depth theoretical analysis of how the depth errors affect the rendered views and design corresponding practical error resilience and protection mechanisms for 3-D visual communications.

In this talk, we will firstly present an analytical distortion model to estimate the distortion in decoded depth map caused by packet loss and mathematically analyse its adverse effect on the rendered views. In this model, the depth errors impaired by random errors are estimated through a recursive function, without requiring the prior knowledge of the probability distribution. Further, the proposed model takes into consideration filtering operations with a very low level of complexity, e.g., interpolation invoked for both fractional pixel motion-compensated prediction. Especially, the expected view synthesis distortion is characterized in the frequency domain using a new approach, which combines the power spectral densities of the reconstructed texture image and the channel errors. Simulation results quantitatively and qualitatively demonstrate the proposed analytic model is capable of modelling the depth-error-induced synthesis distortion. Then, this talk will elaborate on  how the developed distortion model is applied to real error control 3-D video coding for enhancing the robustness of 3-D visual communications. In order to keep backward compatibility with the existing standards, we design an approach in the form of rate-distortion optimal joint texture and depth coding mode selection without modifications to the bit stream syntax. To overcome the adjacent block interdependency induced by warping operation in synthesis, we develop a dynamic programming method to optimally locate the computationally-feasible solution. Further, we extend the Lagrange minimization method to the more general variable-block-size prediction case, where the optimal quadtree tree structure and the combined coding modes are jointly determined using a specially-designed multi-level dual trellis. Experimental results also demonstrate the advantage of the proposed error-resilient algorithm combined with the proposed distortion model in improving both the objective and subjective reconstruction quality of  3-D visual communication.

Welcome Pan!