HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution
Abstract
HumanoidToolBench evaluates humanoid robots on tool selection and coordinated manipulation and locomotion across diverse tasks, revealing gaps between selection and execution.
As robotic hardware and learning methods advance, humanoids need tools to perform tasks beyond their inherent physical limits. Successful tool use requires selecting a suitable tool and coordinating manipulation and, when needed, locomotion to complete the task. Existing benchmarks do not jointly evaluate these capabilities on a humanoid. We introduce HumanoidToolBench, an 18-task benchmark spanning three scenarios, three execution levels, and two tool-set modes, together with ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1. Evaluation of seven policies in simulation and three on the real robot reveals substantial gaps between selecting a suitable tool and completing the task. Focused GR00T N1.7 probes show reduced selection accuracy on unseen tools and continued task execution under unrelated instructions. Code and data are available at https://snu-pi.github.io/HumanoidToolBench/.
Community
Can humanoids choose the right tool and then actually use it?
We introduce HumanoidToolBench, an 18-task benchmark evaluating humanoid tool selection and execution across three scenarios, three execution levels, and two tool-set modes. We also introduce ToolBook, a dataset of 3.1k demonstrations collected in simulation and on a real Unitree G1.
Evaluating seven policies in simulation and three on the real robot reveals a substantial gap between selecting a suitable tool and successfully completing the task. Additional GR00T N1.7 probes examine generalization to unseen tools and behavior under unrelated instructions.
Explore the benchmark and demonstrations on our project page. We welcome feedback from the humanoid learning and tool-use communities!
Get this paper in your agent:
hf papers read 2610.02089 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 2
snupilab/humanoidtoolbench-act-sim-3003
Datasets citing this paper 1
snupilab/humanoidtoolbench-teleop
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper