Robot learning data formats compared: LeRobot, RLDS, HDF5 and MCAP / ROS bag

LeRobot is an open-source robot learning library with its own dataset format. Episodes store tabular data such as robot state and actions alongside video streams, with metadata describing features and frame rates. It is a natural choice if you train with LeRobot policies or want to share data with that community, because loading and visualization tools come with the library.

RLDS (Reinforcement Learning Datasets) is an episodic format built on TensorFlow Datasets. Each episode is a sequence of steps holding observations, actions and other fields. It is widely used for large cross-embodiment collections and suits pipelines built on TensorFlow or JAX that already stream episodes this way.

HDF5 is a general-purpose hierarchical file format for large numerical arrays. Many robot learning codebases use it directly, with one file per episode and groups for cameras, state and actions. It is flexible and fast to read, but it has no single standard layout, so you need a clear data spec that describes groups, shapes, units and timestamps.

MCAP and ROS bag are log formats from the robotics stack. They record time-stamped messages from many topics, such as cameras, joint states and transforms, much as the robot produced them. They are ideal when you want raw, replayable logs or work inside ROS tooling, but most teams convert them to an episodic format before training.

• Match your training code. The best format is the one your data loader already supports.
• Check the schema, not just the container. Confirm field names, units, frame rates and coordinate frames.
• Keep raw logs if you may re-process. MCAP / ROS bag preserves everything, and episodic formats are easier to train on.
• Mind storage and streaming. Video-backed formats save space, and array formats can be faster for random access.

Yes. RData delivers in LeRobot, RLDS, HDF5, MCAP / ROS bag or a custom format, with task- and step-level labels and optional custom annotation schemas. Tell us your loader and we will match it. See the catalog, or read what to check before buying robot data.