What to check before buying robot manipulation data

Conversion is where many data projects lose time. Ask which formats the provider delivers natively and whether conversion is included. Common targets are LeRobot, RLDS, HDF5 and MCAP / ROS bag. Check that the schema covers every field you train on: camera streams, robot state, actions, gripper state and task labels.

Ask for a data spec, not just a format name. Two HDF5 files can be organized very differently. Our format comparison covers the trade-offs.

Multi-camera manipulation data is only useful if the cameras and motion streams line up in time and space. Ask how intrinsic and extrinsic calibration is done and how often it is checked, how streams are timestamped and aligned, and what tolerance is accepted before an episode is rejected.

For human demonstration data, ask how end-effector poses are recovered and how the gripper state is measured. For teleoperation data, ask how robot state and action logs are aligned with video.

Ask what happens between collection and delivery. Good answers describe defined task specifications, trained operators working to standard procedures, automated checks for multimodal alignment and missing data, cleaning, and human review. Ask whether data that fails QC is removed or simply flagged.

Ask which QC metrics you will receive with a sample: for example episode counts, success labels, dropped frames, or sync error. Numbers you can check beat general claims.

Licensing is only as clean as the chain of rights behind it. Ask who holds the rights to the data, how operators and participants consented, and whether the rights and compliance documentation can be shared under NDA. Confirm permitted uses, such as training commercial models, and whether exclusivity is available.

If data was collected in another country, ask how cross-border delivery is handled and whether there is a defined compliance process.

Video of real homes, stores and workplaces can capture faces and other personal details. Ask whether data is de-identified before delivery, what is removed, and how it is checked. De-identification should happen before the data reaches you, not as your job after.

A sample is the cheapest way to verify everything above. Ask for a small slice of the package you care about plus its data spec and statistics, load it into your training pipeline, and check that it trains. Then compare paradigms and scenarios before you commit to a volume.

Browse the RData catalog to pick packages, or read the FAQ for licensing details.