Multi-camera and Model-based Robot Vision

In this talk full 6 d.o.f pose estimation and tracking for robot perception will be addressed in two parts. In a first part a real-time monocular 3D Model based approach is presented where it is assumed that an a priori CAD model of the envrionment is available. This is particularly applicable to industrial robotic environments which are known in detail and are structured with litte dense textured information. This kind of structured approach is derived within a virtual visual servoing formalism. In this case, the spatio-temporal pose estimation problem is considered as synonymous to moving a virtual camera so as to minimize an error criterion in the image. More precisely, the method presented here involves projecting a CAD model of an object onto the image and comparing this with the corresponding visual features representing the object's actual position in the image. The error obtained from the difference then forms a minimization criterion and a non-linear iterative minimization technique is then used to converge upon the actual pose.

Figure 1: 3D Model-based tracking of rigid and articulated structures

In a second part a stereo pair of cameras are introduced to avoid requiring a an a-priori 3D CAD model in unknown environments. This approach is focused on the concept of 6dof visual odometry and bypasses explicit online 3D reconstruction by working in disparity space. A dense minimisation approach is proposed which uses all greyscale information available within the stereo pair (or stereo region). Dense correspondences are first made between reference stereo images using epipolar geometry. Following this the stereo reference images are warped to generate novel viewpoints. This allows fomulation of a non-linear minimisation approach based on a quadrifocal relationship between adjacent image pairs in a video stream. A robust M-estimation technique is used to reject outliers corresponding to moving objects within the scene or other outliers such as occlusions and illumination changes. The technique is applied to recovering the trajectory of a moving vehicle in long and difficult sequences of images with robust and precise results.

Figure 2: Dense Stereo Visual Odometry