Position filled
Detailed information on the 2020 uptake will become available here : http://univ-cotedazur.fr/fr/recherche/boosturcareer
If you are interested in this position please contact Andrew Comport : This email address is being protected from spambots. You need JavaScript enabled to view it.
Real-time 3D Biomechanics Capture
using deep-learning and computer vision
Supervisors :
- Doctor Andrew Comport, I3S (Laboratory of Information and communication Science of Sophia Antipolis),
- Professor Tarek Hamel, I3S (Laboratory of Information and communication Science of Sophia Antipolis),
International partners :
- Youdome, Monaco
- Google, USA
Presentation of the PhD topic :
Capturing and tracking high-detail human motion in real-time is a hot research topic that is fundamental to a wide range of applications including e-health, sport performance analysis, human-robot interaction, augmented reality and many more. This multidisciplinary thesis aims to work across the domains of real-time computer vision, deep learning and bio-mechanics. The aim is to address the problem of acquiring the pose, shape, appearance, motion and dynamics (torques, forces and velocities) of humans in 3D using multi-camera environment in real-time. One of the major challenges in live motion capture is the problem of dense modelling of non-rigid scenes.
The objective of this thesis will be to design an end-to-end approach such that the input to a training network will be the set of images from multiple cameras observing the scene. The output of the network will be the high detail 3D geometry and dynamics acting on the human body. To this end we aim to use RGB-D sensor consistency to train the network in an unsupervised manner such that all images transform correctly to every other image with minimal error. For the training phase we will use many sensors, however, the use of the network for reconstructing the bio-mechanics will use much fewer sensors (even potentially with a single sensor). Such a low-cost set-up with a single camera could be used by a medical (or sport) practitioner for diagnosis.
We aim to train the system using large amounts of training data acquired in collaboration with our partners. In particular, this project is part of a collaboration between Youdome startup (Monaco), Google (USA), the Rosella Hightower dance school (Cannes, France) and the CNRS-I3S/UCA laboratory (Sophia Antipolis, France). The PhD will be supervised by Dr Andrew Comport and Professor Tarek Hamel. The two industrial partners Google and Youdome will also collaborate with the PhD student. Their participation attests a strong applicative interest in the domain and a high potential for future employability.
The PhD candidate will carry out a 6 month stay with one or several of the project partners. The candidate will therefore need to have a technical background with experience in computer vision, machine learning and kinematics with a strong mathematical background and knowledge in C++, Python, Pytorch, Tensorflow, RGB-D sensors along with a strong capacity to write publications in English. Experience with GPU acceleration and real-time systems would also be of interest.