This research is based on a new and accurate technique for Visual 3D Odometry using a stereo-pair of cameras. It investigates a new image-based approach to tracking the 6dof trajectory of a stereo camera pair using a corresponding reference image pairs instead of explicit 3D feature reconstruction of the scene. A dense minimisation approach is employed which directly uses all grey-scale information available within the stereo pair (or stereo region) leading to very robust and precise results. Metric 3D structure constraints are imposed by consistently warping corresponding stereo images to generate novel viewpoints at each stereo acquisition. An iterative non-linear trajectory estimation approach is formulated based on a quadrifocal relationship between the image intensities within adjacent views of the stereo pair. A robust M-estimation technique is used to reject outliers corresponding to moving objects within the scene or other outliers such as occlusions and illumination changes. The technique is applied to recovering the trajectory of a moving vehicle in long and difficult sequences of images.

The 3D visual odometry algorithm was tested on real full-scale sequences as can be seen in the videos below. Several test sequences from different streets in Versailles, France, were used to validate the results. The estimated 3D trajectory from a stereo-pair of cameras was superimposed on Google Earth satelite images of the area.

This first sequence is that of a relatively straight road. The distance travelled by the car has been measured using road markings in the images and satellite views with a precision of 2.9 cm/pixel for the Versailles region. The path length measured by both Google earth and the tracker was about 440m. It is difficult to register the satellite image with the projection of the trajectory since no three non-collinear points were available and the best that can be said is that they have approximately the same absolute length (ignoring tilt of the cameras and the incline of the road). Throughout the sequence several moving vehicles pass in front of the cameras and at one stage a car is overtaken.

This second sequence is particularly illustrative since a full loop of the round-about was performed. In particular this enables the drift to be measured at the crossing point in the trajectory. In the case of this round-about the drift at the crossing point was approximately 20cm in the vertical direction to the road-plane. Considering that the trajectory around the round-about is approximately 200m long (measured using Google earth), this makes a drift of 0.01% on the measurable axis. In the case of large scale scenes such as this one it was necessary to detect and update the reference image periodically when it was no longer visible or too approximate. Due to the highly redundant amount of data, the robust estimator was able to successfully reject pedestrians and moving cars from the estimation process. It can be noted, however, that all static information available was used to estimate the pose (including the parked cars) therefore leading to a very precise result with minimal drift over large displacements.

(Click on image or here to play)

Recently a publication was presented at the International Conference on Robotics and automation (ICRA'07 ) in Rome where we were selected as finalist for the best vision paper prize. The publication is now available here:

• A.I. Comport, E. Malis, P. Rives. Accurate Quadri-focal Tracking for Robust 3D Visual Odometry. In IEEE Int. Conf. on Robotics and Automation, ICRA'07, Rome, Italy, April 2007.

Some of the new results for the LAAS blimp sequence were presented at the conference. To see the sequence and the estimated trajectory click here (approximately 12M).

The full round about sequence showing the reference images as well as robust outlier rejection can be downloaded here (approximately 6.8M).

Since 2005 I have been working on new techniques for dense localisation and mapping techniques that are "Direct" in the sense that no features are extracted or matched from subsequent images in a time sequence. Some of the recent work on this has involved using RGB-D sensors, both stereo and kinect along with monocular 3D tracking. On this subject I was invited to do a talk at LDRMC 2011. Our presentation is available here: comport_LDRMC_talk.pdf This was the First International Workshop on Live Dense Reconstruction from Moving Cameras in conjunction with the ICCV 2011, November 12, Barcelona, Spain. The workshop was highly successful and brought together researchers in robotics and vision to synthesise state-of-the-art theory and practice in live dense reconstruction from moving cameras including passive and active RGB-D devices.

Some videos of this work can be found here:

1. Kinect dense RGB-D SLAM - video

  Related publication: Audras, C., Comport, A. I., Meilland, M. & Rives, P (2011). Real-time dense RGB-D localisation and mapping. In Australian Conference on Robotics and Automation. Monash University, Australia.

2. A symmetric dense mapping and localisation using and combining different sensor modalities including stereo, mono and kinect - video 

Related publication:  Comport, A. I., Meilland, M. & Rives, P (2011). An asymmetric real-time dense visual localisation and mapping system. In First International Workshop on Live Dense Reconstruction from Moving Cameras In conjunction with the International Conference on Computer Vision (ICCV). Barcelona, Spain.