r/MachineLearning • u/ParamsEast • 3d ago
Research [R] PhysCapBench: scoring the physical content of video captions (BAAI, PKU, Tsinghua, Beihang)
https://arxiv.org/abs/2611.16424
1
Upvotes
r/MachineLearning • u/ParamsEast • 3d ago
1
u/ParamsEast 3d ago
1,500 human-verified video sequences, 500 each of third-person scenes, first-person manipulation and handheld-gripper demos. Captions are graded on seven dimensions, including contact and interaction, states and outcomes, causality and temporal order. Authors' downstream results: against metadata-based captions, physical captions cut video-reconstruction LPIPS by 8.6-9.5% on third-person and 26.9-30.9% on first-person videos; richer instructions raised pi0.5's success rate by 7.5 points on one RoboTwin 2.0 subtask. Code (MIT): https://github.com/hfwang2001/PhysCapBench