Estimation of 3D human motion kinematics model from multiple cameras

Abstract

Estimation of articulated human motion based on video sequences acquired from multiple synchronised cameras is an active and challenging research area. This is mainly due to the need of high dimensional non-linear models to describe the
human motion, cluttered data, and occlusions present in the captured images. Although many diverse techniques have been proposed to solve this problem, none of the existing solutions is fully satisfactory. In this thesis, upper body motion tracking and full body motion tracking based on the annealed particle filter (APP) approach are presented.
To successfully implement a body motion tracking algorithm, the first requirement is to prepare and pre-process the data. The work performed in this area includes calibration of multiple cameras, colour image segmentation to extract body silhouettes from the cluttered background, and visual hull reconstruction to provide voxels representing a human volume in 3D space. The second requirement is to build the models. Two set of models are proposed in this thesis. The first set is
for upper body tracking and it contains point models and two-segment articulated arm models; the second set is for full body tracking and it contains five articulated chains as a full human model. The final requirement is to design a measurement
method for aligning the models to the data. Two novel measurement methods are proposed for the motion tracking: one is based on a combination of different penalties tailored to each body part based on the percentage of the 3D to 2D projected
body points, falling inside and outside the body silhouette, and the other is based on the symmetrical property of the intensity profile obtained from the body silhouette bisected by the 3D to 2D projection of the estimated skeletal model.
Various evaluations were carried out to demonstrate the effectiveness of the algorithms implemented and the excellent performance of the proposed methods for upper body and full body motion tracking. These include the accuracy analysis
of cameras calibration and image segmentation; the accuracy and speed of APF applied to the articulated arm model in tracking of the infra-red marker based human motion data; as well as the visual and quantitative assessments of the final
results obtained from the proposed upper body and full body motion tracking.