Deep Learning for 3D Human Digitisation(PIFu)

22nd October 2020

Proposed by Amar Arslaan, Darragh Luttrell, Olivier Riviere
Primary Contact: Amar Arslaan
Email: aarslaan(at)logitech.com

Context
Pixel-aligned Implicit Function (PIFu) uses deep learning models to turn 2D images of human beings into 3D meshes. The output can have a multitude of applications like game character creation, animated movie creation and entertainment. This study aims to explore possibilities of utilising the output from the PIFu model and reuse into creative applications.


Scope
We want the candidate to explore a specific area around 3D Human Digitization: the starting point for the project is this. It is essential to tune the algorithm to get an output that can be directly used in context without further adaptations.

The following minimum deliverables are expected from student undertaking the project:

  • The candidate would be expected to undertake a small literature review, what’s PiFu alternatives as well as state of the art implementations of human digitization.
  • Expanding the body of research and proposing new ways or methods to improve the final output , providing higher quality meshes as well as complex set of texture: normal maps, albedo and potentially ambient occlusion maps.

Stretch Goals
While the PiFU concept works great for digitisation of human beings, it is limited in a sense that we cannot digitise any objects other than human beings. This can be considered as a great opportunity to explore the possibility of applying the same concept to achieve 3D models out of 2D images for other objects (e.g. cups,dogs,…).

Deliverables
Deliverables would include commented code, a demo video and a detailed report (college dissertation doc).

Requirements
Computer science background preferred, experience in Machine Learning and Computer Vision would be beneficial.

References

  1. Saito, S., Simon, T., Saragih, J. and Joo, H., 2020. PIFuHD: Multi-Level Pixel Aligned Implicit Function for High-Resolution 3D Human Digitization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 84-93).
  2. Saito, S., Huang, Z., Natsume, R., Morishima, S., Kanazawa, A. and Li, H., 2019. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 2304-2314).
CategoryOngoing