Non-Rigid Structure from Motion with incremental Shape Prior

Abstract

Most existing approaches to the non-rigid structure from motion problem use batch type algorithms with all the data collected before 3D shape reconstruction takes place. Such a methodology is not suitable for real-time applications. Concurrent on-line estimation of the camera position and 3D structure, based only on the measurements up to that moment is much more a challenging problem. In this paper, a novel approach is proposed for recursive recovery of non-rigid structures from image sequences captured by an orthographic camera. The main novelty in the proposed method is an adaptive algorithm for construction of shape constraints imposing stability on the on-line reconstructed shapes. The proposed, adaptively learned constraints have two aspects, consisting of constraints imposed on the basic shapes, the basic “building blocks” from which shapes are reconstructed, as well as constraints imposed on the mixing coefficients in a form of their probability distribution. The constraints are updated when the current model inadequately represents new shapes. This is achieved by means of Incremental Principal Component Analysis (IPCA). Results of the proposed method are shown on synthetic and real data of articulated face.