I recently visited  Bob Wang for two weeks at the NTU Department of Computer Science and Information Engineering. We have initiated a collaboration between NTU (National Taiwan University) and I3S.

I will be giving a seminar on the 29th of May 10:10 – 10:30 at Microsoft Research Cambridge at the invitation only workshop "At the intersection of Vision, Graphics, Learning and Sensing --- Representations and Applications Workshop". The presentation witll be on the topic of "Metric-Topologic dense real-time visual localisation and world mapping".

Metric-Topologic dense real-time visual localisation and world mapping

In this talk I will present an combined metric-topological framework for large scale dense localisation and mapping in real-time. In a first part it will be shown how large-scale dense photometric models are acquired using RGB-D sensors including both multi-camera and Kinect devices. Several different RGB-D systems will be presented.

In a second part it will be shown how different sensors may then be used with these prior models to perform robust localisation in dynamic environments including a monocular, stereo or Kinect sensor. The proposed approach to integrating scene structre is based on our "Direct ICP" technique. It will be shown to handle dynamic changes in the scene involves combining the prior dense photometric model with online visual odometry. In particular it will be shown how the technique takes into account large illumination variations and subsequently improves direct techniques which are intrinsically prone to illumination change. This is achieved by exploiting the relative advantages of both model-based and visual odometry techniques for tracking. Direct 6 dof tracking is performed by an accurate method, which directly minimizes dense image measurements iteratively, using non-linear optimisation.

This presentation is based on collaborations with the authors:

• Audras, C., Comport, A. I., Meilland, M. & Rives, P (2011). Real-time dense RGB-D localisation and mapping. In Australian Conference on Robotics and Automation. Monash University, Australia. [More] [Bibtex] [RIS] [MODS]
• Comport, A. I., Meilland, M. & Rives, P (2011). An asymmetric real-time dense visual localisation and mapping system. In First International Workshop on Live Dense Reconstruction from Moving Cameras In conjunction with the International Conference on Computer Vision (ICCV). Barcelona, Spain. [More] [Bibtex] [RIS] [MODS]
• Tykkälä, T. M. & Comport, A. I (2011). Direct Iterative Closest Point for Real-time Visual Odometry. In The Second international Workshop on Computer Vision in Vehicle Technology: From Earth to Mars in conjunction with the International Conference on Computer Vision. Barcelona, Spain. [More] [Bibtex] [RIS] [MODS]

• Meilland, M., Comport, A. I. & Rives, P (2011). Dense visual mapping of large scale environments for real-time localisation. In IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, California. [More] [Bibtex] [RIS] [MODS]

 I recently was invited to Corte by Dr. Lucile Rossi to work on her project aimed at observing fire with cameras and modelling its behaviour. During this visit we set up 11 cameras around a fire and flimed it. Here are some of the images of this acquisition campaign:

I will be giving a seminar on the 8th of March at 2pm for the Robot Vision Group at Imperial College London on the topic of Large scale dense localisation and mapping:

Large scale dense localisation and mapping

In this talk I will present an asymmetric framework for real-time dense localisation and mapping. In a first part it will be shown how large-scale dense photometric models are acquired using RGB-D sensors including both multi-camera and Kinect devices. Several different multi-camera systems for automatically acquiring the 3D spherical photometric model with depth will be presented.

In a second part it will be shown how different sensors may then be used with these prior models to perform robust localisation in dynamic environments including a monocular, stereo or Kinect sensor. The proposed approach to handling dynamic changes in the scene involves combining the prior dense photometric model with online visual odometry. In particular it will be shown how the technique takes into account large illumination variations and subsequently improves direct techniques which are intrinsically prone to illumination change. This is achieved by exploiting the relative advantages of both model-based and visual odometry techniques for tracking. Direct 6 dof tracking is performed by an accurate method, which directly minimizes dense image measurements iteratively, using non-linear optimisation.

If time allows I will also present a generalised imaging model for visual servoing which allows to combine monocular cameras, multi-camera systems and non-central projection cameras into a same framework. In this case cameras are modelled as sets of 3D viewing rays. This model leads to a new generalised visual servoing control formalism that can be applied to any type of imaging system whether it be multi-camera, catadioptric, non-central, etc. An example will be presented with a pair of non-overlapping cameras.

This presentation is based on collaborations with the authors:

• Audras, C., Comport, A. I., Meilland, M. & Rives, P (2011). Real-time dense RGB-D localisation and mapping. In Australian Conference on Robotics and Automation. Monash University, Australia. [More] [Bibtex] [RIS] [MODS]
• Comport, A. I., Meilland, M. & Rives, P (2011). An asymmetric real-time dense visual localisation and mapping system. In First International Workshop on Live Dense Reconstruction from Moving Cameras In conjunction with the International Conference on Computer Vision (ICCV). Barcelona, Spain. [More] [Bibtex] [RIS] [MODS]
• Tykkälä, T. M. & Comport, A. I (2011). Direct Iterative Closest Point for Real-time Visual Odometry. In The Second international Workshop on Computer Vision in Vehicle Technology: From Earth to Mars in conjunction with the International Conference on Computer Vision. Barcelona, Spain. [More] [Bibtex] [RIS] [MODS]
• Comport, A. I., Mahony, R. & Spindler, F (2011). A Visual Servoing Model for Generalised Cameras: Case study of non-overlapping cameras. In IEEE International Conference on Robotics and Automation, ICRA'11. Shanghai, China. [More] [Bibtex] [RIS] [MODS]

In collaboration with Patrick Rives and Maxime Meilland from the Arobas team at INRIA, we presented two demonstrations for the open day at INRIA for the European Robotics Week:

• Demo 1 (indoors) : Photorealistic 3D reconstruction and localisation in real time using vision
• Demo 2 (outdoors) : Autonomous vehicule navigation in urban environments.

The following are some of the images of the day kindly taken by the organisor, Rose-Marie Cornus from INRIA:

For more info please have a look at the following links:

INRIA robotics day website.

The european robotics coordination action.

Several releases have been made in the press following this day:

Diaporama:

http://www-sop.inria.fr/manifestations/portes-ouvertes-robotique/photos/Portes-Ouvertes-Robotique-29nov2011/index.html

01 Net articles with videos:

http://pro.01net.com/editorial/547722/eu-robotics-week-la-kinect-de-microsoft-devient-un-outil-de-vision-pour-les-robots/

http://pro.01net.com/editorial/547710/eu-robotics-week-le-vehicule-automatique-cycab/

Article from Nice-Matin:

NM-1-12-2011.pd

I was invited to do a talk at LDRMC 2011. Our presentation is available here: comport_LDRMC_talk.pdf

This was the First International Workshop on Live Dense Reconstruction from Moving Cameras in conjunction with the ICCV 2011, November 12, Barcelona, Spain.

http://www.doc.ic.ac.uk/livedensereconstruction/

The workshop was highly successful and brought together researchers in robotics and vision to synthesise state-of-the-art theory and practice in live dense reconstruction from moving cameras including passive and active RGB-D devices.

APPLICATIONS ARE NOW CLOSED

Real-time dense localisation and mapping

for autonomous traversal of un-even ground

The computer vision and robotics group at the CNRS-I3S, Sophia-Antipolis, France is seeking an outstanding Postdoctoral candidate.

The position is under the supervision of Dr. Andrew Comport and and is associated with a project developing new techniques in real-time Simultaneous Localisation and Mapping (SLAM) using RGB-D sensors. The principal objective of this project will be to develop a localisation module which produces a 6D (position and orientation) position of a camera during the displacement of a mobile robot. This localisation will allow a control module to traverse un-even terrain and obstacles in an autonomous manner. The localisation will be carried out by estimating the motion of the platform using both photometric texture information and geometric measurements of the immediate environment around the robot. A fusion step is also to be considered which will integrate data from an embedded inertial sensor. Another objective of this project will be to develop a module which produces a 3D representaiton of the environement. This model will be augmented with information that is necessary for generating the movement of the robot like, for example, accesible zones in the environment for the robot or the geometric simplification of the scene using planar regions. Both the localisation and mapping phases will be integrated into a coupled real-time process.

This project aims to apply these techniques to the Caméléon mobile robot (see above image) designed by ECA. This will involve working closely with their platform and using compatible open source software such as ROS and OpenCV. 

The candidate for the Postdoc position is expected to have a PhD degree in computer vision or visual servoing. Strong programming skills in C, C++, MATLAB or equivalent are required. Applicants demonstrating excellence in other related areas are encouraged to apply. International applications are encouraged. The salary is 2300-2500 Euro/month gross and will be indexed with inflation. The three year project is due to begin early in 2012.

If you are interested in this position please contact :

Dr. Andrew Comport - comport@i3s.unice.fr

and join a CV including at least two referees and a motivation letter.

Some related references include:

[1] Comport, A. I., Malis, E. & Rives, P (2007). Accurate Quadri-focal Tracking for Robust 3D Visual Odometry. In IEEE International Conference on Robotics and Automation, ICRA'07. Rome, Italy.

[2] Tykkälä, T. M. & Comport, A. I (2011). Direct Iterative Closest Point for Real-time Visual Odometry. In The Second international Workshop on Computer Vision in Vehicle Technology: From Earth to Mars in conjunction with the International Conference on Computer Vision. Barcelona, Spain.

[3] Meilland, M., Comport, A. I. & Rives, P (2011). Dense visual mapping of large scale environments for real-time localisation. In IEEE/RSJ International Conference on Intelligent Robots and Systems. San Francisco, California.

[4] Kurt Konolige, Motilal Agrawal: FrameSLAM: From Bundle Adjustment to Real-Time Visual Mapping. IEEE Transactions on Robotics 24(5): 1066-1077 (2008)

[5] Meilland, M., Comport, A. I. & Rives, P (2010). A Spherical Robot-Centered Representation for Urban Navigation. In IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS'10. Taipei, Taiwan.

[6] Richard A. Newcombe and Andrew J. Davison. Live Dense Reconstruction with a Single Moving Camera. IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2010)

 Title: Life Long Navigation and Map Learning

APPLICATIONS ARE NOW CLOSED

The computer vision and robotics group at Sophia-Antipolis, France is seeking an outstanding PhD candidate for a CNRS funded scholarship. The position will begin in October 2011. The position is under the supervision of Dr. Patrick Rives and Dr. Andrew Comport. This project will focus on life long navigation and map learning using vision sensors in collaboration with Astrium.

Title: Life Long Navigation and Map Learning 

Subject: The objective of this PhD is to investigate the general problem of visual mapping of complex 3D environments that evolve over time. This requires explicitly taking into account the variability of viewing conditions and content within a 3D geometric and photometric model of the environment. A central objective will be to investigate how to efficiently and accurately represent this model. The aim is to build models that remain usable over long periods of time for navigation tasks (localisation and path following) for autonomous systems or for people that require guidance. This research problem, which is already active in the vision community, has only just begun to be investigated by the robotics community via the new perspective of incremental SLAM (Simultaneous Localization and Mapping).

Now that the problem of modelling large-scale environments is well understood, the next major technical challenge on the horizon is the problem of being able to maintain a valid environment representation over a long period of time. This problem is referred to as Life-Long Learning. This topic is in continuity with the research carried out conjointly between P.Rives from the Arobas team at INRIA Sophia-Méditerrané and A.Comport from CNRS-I3S UNSA. The existing approach relies on a ego centered representation that combines a spherical image and a depth map (augmented visual sphere). This representation synthesises information collected in an local area of space by an embedded acquisition system. The representation of the global environment consists of a collection of augmented visual spheres that provide the necessary coverage of an operational area and each sphere is precisely geo-referenced globally. A "pose" graph that links these spheres together, in six degrees of freedom, also defines the domain which is reachable by an autonomous agent and that is potentially exploitable for navigation tasks in real time. As part of this research, it is proposed to develop an approach to Life-long learning map-based representation by considering the following issues:

1. How to determine the quantity and optimal placement of the augmented spheres to cover a local environment completely.
2. How to compactly represent the information contained in the augmented sphere to ensure robustness, accuracy and stability over time (saliency map).
3. How to exploit this representation in a navigation task in real time.
   
   

The implementation of the work will be integrated, tested and validated on robotic platforms available at INRIA Sophia-Antipolis and partner laboratories.

Requirements : For this position, candidates must be enrolled in a Masters stream with a focus on computer vision, visual servoing, and/or automatic control. The candidate should have also good abilities for software development and report writing.

Contacts : Send a CV and a motivation letter to  Patrick.Rives@inria.fr or   comport@i3s.unice.fr

Location :

Projet ARobAS INRIA,

Centre Sophia-Antipolis Méditerranée

2004 Route des Lucioles

06902 SOPHIA-ANTIPOLIS Cedex

A detailed description of this position can be found in French here :

http://www.i3s.unice.fr/~comport/outgoing/sujet_these/Sujet-Astrium-2011.pdf

I will be attending the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2010).