Category: CMUTR
Institution: Department of Computer Science, Carnegie
        Mellon University
Abstract: We propose a method for estimating the three-dimensional shape
        of objects and the motion of the camera from a stream of images
        The goal is to give a robot the ability to localize itself with
        respect to the environment, draw a map of its own surroundings,
        and perceive the shape of objects in order to recognize or 
        grasp them.
        Solutions proposed in the past were so sensitive to noise as to
        be of little use in practical applications.
        This sensitivity is closely related to the viewercentered 
        representation of scene geometry known as a depth map, and to
        the use of stereo triangulation to infer depth from the images.
        In fact, when objects are more than a few focal lengths away
        from the camera, parallax effects become subtle, and even a 
        small amount of noise in the images produces large errors in 
        the final shape and motion results.
        In our formulation, we represent shape in object-centered 
        coordinates, and model image formation by orthographic, rather
        than perspective projection.
        In this way, depth, the distance between viewer and scene, play
        no role, and the problem's sensitivity to noise is critically
        reduced.
        We collect the image coordinates of P feature points tracked
        through F frames into a 2F X P measurement matrix.
        If these coordinates are measured with respect to their cent-
        roid, we show that represent the measurement matrix can be 
        written as the product of two matrices that represent the 
        camera rotation and the positions of the feature points in
        space.
        The bilinear nature of this model, and its matrix formulation,
        lead to a factorization method for the computation of shape 
        and motion, based on the Singular Value Decomposition.
        Previous solutions assumed motion to be smooth, in one form or
        another, in an attempt to constrain the solution and achieve 
        reliable convergence.
        The factorization method, on the other hand, makes on assump-
        tion about the camera motion, and can deal with the large jumps
        from frame to frame found, for instance, in sequences taken 
        with  a hand-held camera.
        To make the factorization method into a working system, we 
        solve several corollary problems: how to select image features,
        how to track them from frame to frame, how to deal with 
        occlusions, and how to cope with the noise and artifacts that
        corrupt image features, how to track them from frame to frame,
        how to deal with occlusions, and how to cope with the noise
        and artifacts that corrupt images recorded with ordinary equip-
        ment.
        We test the entire system with a series of experiments on real
        images taken both in the lab, for an accurate performance 
        evaluation, and outdoors, to demonstrate the applicability of
        the method in real-life situations.  
        
        
        
        
        
        
Number: CMU-CS-91-172
Bibtype: TechReport
Month: Sep
Author: Carlo Tomasi		
Title: Shape and Motion from Image Streams: a Factorization Method
Year: 1991
Address: Pittsburgh, PA
Super: @CMUTR