Fingertip force accuracy

1.002 N mean absolute error for the normal component across 32 sequences, compared with 1.326 N for the strongest baseline.
The plot shows one example recording.
1 Tsinghua University2 Parity Lab3 Hangzhou Dianzi University
Reconstructing hand–object interactions is an important task, and achieving it with a single camera makes large-scale capture practical but challenging due to heavy occlusions. We introduce HOForce to track hands and objects, and further recover interaction forces from single-view RGB-D video and known object properties. For two-hand manipulation, we jointly model fingertip forces of both hands and the contact forces between objects when two objects are presented. The reconstructed forces also improve motion tracking by giving pose constraints in the occluded regions. HOForce recovers hand and object motion more accurately on DexYCB and TACO and produces less hand–object penetration than interaction baselines on DexYCB. On our dataset of fingertip forces recorded with tactile sensors, HOForce reduces normal-force MAE by 24.5% relative to the strongest baseline. The recovered forces reproduce object motion in MuJoCo and transfer to other engines after adaptation.
HOForce first reconstructs motion from RGB-D observations, then estimates contact forces and uses those forces to refine the reconstruction.
We evaluate the recovered forces against tactile sensors and test whether they reproduce the recorded motion during simulation.

1.002 N mean absolute error for the normal component across 32 sequences, compared with 1.326 N for the strongest baseline.
The plot shows one example recording.
19 of 21 insertions succeed in each target engine after adapting the MuJoCo forces to that engine.
The application points and physical parameters are retained.
Five minutes of motion reconstruction, force comparisons, replay across engines, and ablations. The video includes English narration and captions.
@misc{wang2026hoforcemotionforcereconstruction,
title = {HOForce: Motion and Force Reconstruction of Hand-Object Interaction Using a Single-View RGB-D Video},
author = {Jiahui Wang and Wanqing Du and Yuanbin Zou and Zhefeng Xu and Feng Xu},
year = {2026},
eprint = {2611.23871},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2611.23871}
}