The kinematic set idea consists of modeling symmetrically the motion and velocity of an articulated object so as to respect mechanical constraints. This approach is likened to a Lagrange-d'Alembert formulation in classical physics. The advantages of this new model over pre-existing methods include improved precision, robustness and efficiency, leading to real-time performance. Furthermore, a general class of mechanical joints can be considered and the method can track objects where previous approaches have failed due to a lack of visual information. In summary, a joint configuration is modeled by using Pfaffian velocity constraints. The configuration and location of a joint is then used to build a general Jacobian Matrix which relates individual rigid body velocities(twists) to an underlying minimal subspace. A closed loop control law is then derived in order to minimize a set of distance errors in the image and estimate the system parameters. The tracking is locally based upon efficient distance criterion. Experimental results show prismatic, rotational and helical type links and up to eight general parameters. A statistical M-estimation technique is applied to improve robustness. A monocular camera system was used as a real-time sensor to verify the theory.
Some movies of the experiments are presented below. More detail can be found in the related publications.
Rotational Link
This experiment was carried out for a class 1 type link with a single degree of rotation on the x-axis. The experiment shows rotation of a hinged door while undergoing various movements. In this and all the following figures the reference frames for each component are shown in yellow, blue and red for each axis. The projection of the CAD model contour onto the image using the estimated pose parameters is shown in blue. It can be seen that the projected position of the contours are visually acceptable. The points rejected by the M-estimator are shown in green.
Prismatic Link
This second experiment was carried out for class one link with a single degree of translation on the x axis. Translation along a sliding mechanism while the camera is moving. The first component is the rail and the second is the square which slides along the rail. It should also be noted that all CAD models were measured very roughly by hand using a simple ruler. Therefore, it can be seen that the tracking is able to support a certain degree of modeling error also.
Helical Link
This experiment was carried out for class one link with helical movement simultaneously along and around the z axis. The constraint vector was defined by taking into consideration that for 10*2pi rotations of the screw, the bold translated 4.5 centimeters along the z axis.
Robotic Arm
An experiment was carried out for two class one links on a robotic arm so as to generalize the approach to 3 or more joints. The articulations tracked were rotational links around the z and x axes. In this experiment it can be seen that even when very little visual information is available, the estimation of unknown joint parameters can be achieved by relying on the more stable estimation obtained from the base of the robot. Various positions and configurations have been tested.
Related Publications
This research was carried out during the PhD of Andrew Comport at INRIA Rennes, under the surpervision of Eric Marchand and Francois Chaumette. The related publication can be found here:
- A.I. Comport, E. Marchand, F. Chaumette. Kinematic Sets for Real-time Robust Articulated Object Tracking. Image and Vision Computing (IVC) Special Issue on Articulated and Nonrigid Motion, to appear, 2006.
- A.I. Comport, E. Marchand, F. Chaumette. Complex articulated object tracking. In Int. Workshop on Articulated Motion and Deformable Objects, AMDO'04, LNCS, Palma de Mallorca, Spain, September 2004.