In this reseach a 3D model-based tracker is described based on a formulation of pose computation involving a full scale non-linear optimization: Virtual Visual Servoing (VVS). More precisely it consists in specifying a cost function based on a set of visual feature in the image along with the perspective projection of a 3D model. A non-linear iterative procedure is used to minimize the error between the current and desired position of these visual features by estimating the unknown motion of the camera. In this work an image feature based system has been developed which is capable of treating complex scenes in real-time without the need for markers.

Overview of the approach

This research addresses the problem of real-time model-based tracking of 3D objects in monocular image sequences. This fundamental vision problem has applications in many domains ranging from Augmented Reality to Visual Servoing and even Medical Imaging or Industrial applications.

A markerless model-based algorithm is used for the tracking of 3D objects in monocular image sequences. The main advantage of a model based method is that the knowledge about the scene (the implicit 3D information) allows improvement of robustness and performance by being able to predict hidden movement of the object and acts to reduce the effects of outlier data introduced in the tracking process.

Description of the approach

In this reseach a 3D model-based tracker is described based on a formulation of pose computation involving a full scale non-linear optimization. More precisely it consists in specifying a cost function based on a set of visual feature in the image along with the perspective projection of a 3D model. A non-linear iterative procedure is used to minimize the error between the current and desired position of these visual features by estimating the unknown motion of the camera. This work has shown an image feature based system which is capable of treating complex scenes in real-time without the need for markers.

An animation showing a traditional visual servoing approach. (Click on image to view video)	An animation showing a virtual visual servoing approach. (Click on image to view video)

Some results and demonstrations

Robust tracking of a cylinder and and two straight lines. (Click on image to view video)	Augmented Reality using the output of the tracking process. (Click on image to view video)
Tracking of an airconditionner with heavy occlusion. An augmented version of this sequence in given in the AR page. (Click on image to view video)	Tracking using various types of visual information simultaneously. Here a circle, cylinder and two lines are used. (Click on image to view video)