Who Moved the Robot?
Humanoid Datasets Remember Their Operators

TL;DR We show that human-to-humanoid motion retargeting preserves operator movement dynamics despite discarding body shape, enabling recovery of gender, age, height, and identity from robot joint-angle trajectories alone. We measure and interpret this effect at scale, and ask what it implies for how the robotics community collects and shares demonstration data.

Overview

Operator movement signatures survive the retargeting transform. Retargeting projects diverse teleoperators onto a uniform robot skeleton (Unitree G1), stripping body-shape parameters entirely. Operator-specific movement dynamics nevertheless persist in the joint-angle trajectories, and our proposed UNVEIL recovers the operator’s attributes directly from the humanoid data.

Abstract

Inside UNVEIL: The GCN

UNVEIL overview. A retargeted humanoid trajectory is encoded into per-joint motion features, processed by two hierarchical spatial graph convolutions (intra-limb, limb-torso), aggregated across time with N spatiotemporal layers into a trajectory embedding that is decoded by task-specific heads into operator attributes.

Interpretable Features

Retargeting throws away body shape, but features of movement dynamics — how each operator's joints move through time — persist in the humanoid trajectory. That signal is enough to recover the operator's biometrics from the robot motion alone. The two panels below illustrate the effect for age and gender: each panel pairs two G1 demonstrations of the same activity by operators at opposite ends of the attribute, with the motion feature most predictive of that attribute plotted between them.

Similarly, UNVEIL reveals operator height and weight from the same retargeted trajectories.

Feature Statistics

Unlike the single demonstration pairs above, the plots below aggregate the effect across the entire dataset: on a single activity, a single feature of movement dynamics is enough to separate operators whose attribute values sit at the two extreme ends. The two boxplots show this for age and gender — each dot is one demonstration clip. Hover any dot to see the original operator beside the retargeted humanoid motion.

Quantitative Results

Comparison between UNVEIL and two well-tested action-recognition architectures adapted with our framework — SGN and DSGCN. We consistently improve over both adaptations across all tasks. Gray shows task-specific chance performance for reference.

Performance across tasks and evaluation settings

Beyond the head-to-head comparison above, the table below reports UNVEIL’s absolute numbers for every inference task under each evaluation setting. Unseen operators are people never in training (zero-shot biometrics); unseen demos of seen operators are held-out activities for operators we trained on (cross-activity generalization). (↑ higher is better; ↓ lower is better.)

BibTeX

@inproceedings{afnan2026whomoved,
    title={Who Moved the Robot? Humanoid Datasets Remember Their Operators},
    author={Afnan, Sihat and Jain, Unnat and Farrukh, Habiba},
    booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
    year={2026}
}