Deep Spherical Harmonics

Proposed by Matis Hudon
Email: hudonm at scss.tcd.ie

Figure 1: Spherical Harmonics Representation

Figure 2: Convolutional Neural Network Example

In [RH01] it is shown that one only needs 9 spherical harmonics coefficients, corresponding to the lowest frequency modes of the illumination, to compute a diffuse shading with an error of 1%. On the other hand, Convolutional Neural Networks have proven to be very interesting and efficient image descriptors for classification. Recent works also use them to estimate 3D geometry directly from RGB images. The goal of this project is to make use of Deep Neural Networks and Convolutional Neural Networks to estimate the natural (unknown) lighting of a scene,  computing the coefficients of the 9 first spherical harmonics. The first part of the project is building a database of rendered images (Blender/Maya or other), and this database will then be used to train the CNN/DNN (Python/C++, Tensorflow/Caffe).

Reference:
[RH01] Ravi Ramamoorthi and Pat Hanrahan. An efficient representation for irradiance environment maps. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 497–500.ACM, 2001. 22Data augmentation with artistic style