We present Humanoid World Models (HWM) 🤖🧠 — a family of lightweight, open-source video prediction models for humanoid robots — showcased at the ICML 2025 Workshop on Physically Plausible World Models. It predicts 8 future video frames conditioned on 9 past video frames and humanoid control tokens (e.g., joint angles, velocities). Such world models can facilitate synthetic policy evaluation, synthetic policy learning, and long-horizon planning.

Motivation

World foundation models, when scaled appropriately, are able to predict high fidelity and physically plausible future video conditioned on the past. These models can be used by humanoid robots for synthetic policy evaluation, synthetic policy training, and long-horizon planning. However many world foundation models are closed-source and/or require large amounts of computational resources (e.g. 8+ H100s).

Key Features:

Two model families: While Masked-HWM achieves higher visual fidelity and faster inference, Flow-HWM offers benefits in continuous latent modeling. Our findings favor Masked-HWM for most practical deployments:
- Masked-HWM: non-autoregressive masked video transformer using VQ-VAE latents.
- Flow-HWM: continuous latent space model using flow-matching.
Architecture Variants: Various attention styles (joint vs. split attention) and weight-sharing strategies were explored, providing up to 53% parameter reduction with minimal performance degradation.
Efficiency: Trained and deployed using only 1–2 GPUs.
Data: Trained on 100+ hours of egocentric humanoid video and control traces from the 1xGPT dataset.

Sample Predictions

Sample predictions from each model variant are shown below. Masked-HWM achieves higher visual fidelity and faster inference. In the comparison images, the bottom row is generated while top row is ground truth.