The kinematic set idea consists of modeling symmetrically the motion and velocity of an articulated object so as to respect mechanical constraints. This approach is likened to a Lagrange-d'Alembert formulation in classical physics. The advantages of this new model over pre-existing methods include improved precision, robustness and efficiency, leading to real-time performance. Furthermore, a general class of mechanical joints can be considered and the method can track objects where previous approaches have failed due to a lack of visual information. In summary, a joint configuration is modeled by using Pfaffian velocity constraints. The configuration and location of a joint is then used to build a general Jacobian Matrix which relates individual rigid body velocities(twists) to an underlying minimal subspace. A closed loop control law is then derived in order to minimize a set of distance errors in the image and estimate the system parameters. The tracking is locally based upon efficient distance criterion. Experimental results show prismatic, rotational and helical type links and up to eight general parameters. A statistical M-estimation technique is applied to improve robustness. A monocular camera system was used as a real-time sensor to verify the theory.

Some movies of the experiments are presented below. More detail can be found in the related publications.

Rotational Link

This experiment was carried out for a class 1 type link with a single degree of rotation on the x-axis. The experiment shows rotation of a hinged door while undergoing various movements. In this and all the following figures the reference frames for each component are shown in yellow, blue and red for each axis. The projection of the CAD model contour onto the image using the estimated pose parameters is shown in blue. It can be seen that the projected position of the contours are visually acceptable. The points rejected by the M-estimator are shown in green.

CLICK HERE TO SEE THE VIDEO

Prismatic Link

This second experiment was carried out for class one link with a single degree of translation on the x axis. Translation along a sliding mechanism while the camera is moving. The first component is the rail and the second is the square which slides along the rail. It should also be noted that all CAD models were measured very roughly by hand using a simple ruler. Therefore, it can be seen that the tracking is able to support a certain degree of modeling error also.

CLICK HERE TO SEE THE VIDEO

Helical Link

This experiment was carried out for class one link with helical movement simultaneously along and around the z axis. The constraint vector was defined by taking into consideration that for 10*2pi rotations of the screw, the bold translated 4.5 centimeters along the z axis.

CLICK HERE TO SEE THE VIDEO 1

CLICK HERE TO SEE THE VIDEO 2

Robotic Arm

An experiment was carried out for two class one links on a robotic arm so as to generalize the approach to 3 or more joints. The articulations tracked were rotational links around the z and x axes. In this experiment it can be seen that even when very little visual information is available, the estimation of unknown joint parameters can be achieved by relying on the more stable estimation obtained from the base of the robot. Various positions and configurations have been tested.

CLICK HERE TO SEE THE VIDEO 1

CLICK HERE TO SEE THE VIDEO 2

Related Publications

A.I. Comport, E. Marchand, F. Chaumette. Kinematic Sets for Real-time Robust Articulated Object Tracking. Image and Vision Computing (IVC) Special Issue on Articulated and Nonrigid Motion, to appear, 2006.

A.I. Comport, E. Marchand, F. Chaumette. Complex articulated object tracking. In Int. Workshop on Articulated Motion and Deformable Objects, AMDO'04, LNCS, Palma de Mallorca, Spain, September 2004.