Teleoperation vs human demonstration vs egocentric video: which robot data do you need?

In teleoperation, a human operator controls a real robot through a task while sensors record everything. Each episode pairs synchronized camera video with the robot's own joint and end-effector state, the action commands it received, and task labels. Some packages also capture contact-force and torque signals.

Because the data comes from the robot itself, it closes the gap between what a policy sees in training and what it sees in deployment. That makes it the most direct data for fine-tuning a policy on one embodiment, for evaluating policies against real episodes, and for contact-rich tasks such as connector insertion where force matters.

The trade-off is cost and scale. Every hour needs a robot, an operator and a working cell, so teleoperation volumes are usually smaller than other paradigms. In the RData catalog, teleoperation accounts for about 7,000 hours across 7 commercial platforms, including bipedal humanoids, wheeled humanoids and mobile dual-arm systems.

Human demonstration data is recorded while people perform tasks with handheld capture devices that mimic a robot gripper. A calibrated, synchronized multi-camera rig (for example head plus left and right wrist cameras) records the scene, and the device logs end-effector pose and gripper state over time.

The data is robot-agnostic: it describes what the hand and gripper did, not what a specific robot did. That makes it retargetable to many embodiments and much cheaper to scale, because collection does not need a robot. It is well suited to large-scale policy pre-training, cross-embodiment transfer, and dexterous or deformable-object tasks such as clothing folding.

The trade-off is the embodiment gap. Before deployment, most teams fine-tune on some robot-specific data. Human demonstration is the largest paradigm in the RData catalog at about 80,000 hours.

Egocentric video is first-person footage of people doing real work, recorded step by step with task metadata. Capture can be mono or stereo. It does not include robot state or action trajectories.

What it offers is breadth: many environments, objects and long task sequences. Teams use it for world-model and vision-language-action (VLA) pre-training, task understanding and long-horizon planning priors. The RData catalog holds about 22,000 hours of egocentric video across homes, warehouses, retail and hospitality.

• Robot state: teleoperation only.
• Motion / trajectory: teleoperation and human demonstration.
• Force / tactile: teleoperation, in one selected package.
• Hand / gripper motion: human demonstration only.
• Video and task labels: all three.
• Relative cost per hour: highest for teleoperation, lowest for egocentric video.

Start from your bottleneck. If your policy works in simulation but fails on hardware, real-robot teleoperation on your embodiment is usually the fastest fix. If your model lacks general manipulation skill, add volume with human demonstration data. If it struggles to understand tasks or plan over many steps, egocentric video adds the missing breadth.

A common pattern is a data pyramid: egocentric video at the base for breadth, human demonstration in the middle for scale, and teleoperation at the top for grounding on real robots. See the full data types overview or browse examples such as Home & Living and Manufacturing & Assembly.

Yes. The most reliable way to choose is to train on a sample in your own pipeline and compare results. RData shares samples and detailed statistics under NDA, so you can test each paradigm on your own tasks before licensing.