Video-Based 3D pose estimation for residential roofing-dataset

Methods

Ground Truth

• The whole-body marker data (total 79 makers placed on each subject) were recorded using a Vicon motion capture system with 14 optical cameras (Vantage V16, Vicon Motion System Ltd., Oxford, UK).
• An open-source musculoskeletal modeling software—OpenSim—was used to estimate the joint positions based on the recorded marker locations. The joint definitions were similar as in the Human3.6M (the largest existing public dataset of 3D human poses).

Multi-view model (stage 1)

• The learnable triangulation model was chosen for the multi-view model since it is one of the state-of-art multi-camera 3D human pose models. This network used ResNet-152 as the backbone to obtain heat maps of 2D human pose and computed the softmax across the spatial axes to get the 2D positions of the joints.
• The learnable triangulation model was pre-trained by the public Human3.6M video data and fine-tuned by the recorded roofing video data.
• 3D coordinates of the algebraic model results were normalized for each subject.

Multi-frame model (stage 2)

• Multi-frame model consisted of several grouped temporal convolution layers with the dilation.
• Before using the ground truth to train our model, the model was pre-trained using the human pose data with random noise. Human pose data with noise has been used as both the input and the label of the network to train the network.
• Mean Squared Error (MSE) was used to calculate the error between the processed coordinate sequence and the ground truth as a loss function. A temporal smoothness constraint was used in the loss function by calculating the mean of the L2 norm of the first order derivative of the 3D joint locations.