PREDICT THE WORLD. CONNECT TO ACTION.

World-Action Modelsfor Robot Learning and Control: A Survey

A robotics-oriented review of models that connect future world prediction
with executable action generation.

Zuxing Lu^1,*, Hongjia Zhai^1,*, Guanzhi Wang², Huajian Zeng¹, Jiaqi Yang¹, Jingyu Liu¹, Lei Cheng¹, Yuantai Zhang¹, Yuheng Qiu³, Zezhou Cheng⁴, Ivan Laptev¹, Danfei Xu⁵, Benjamin Riviere⁶, Giuseppe Loianno⁷, Eric Xing¹, Xingxing Zuo^1,†

¹ MBZUAI² Caltech³ Amazon FAR⁴ University of Virginia⁵ Georgia Tech⁶ New York University⁷ UC Berkeley

^* Equal contribution^† Corresponding author

Abstract

Robots operating in open environments act under partial observability, physical constraints, and dynamic task contexts. Beyond mapping observations and language instructions to actions, they must anticipate how candidate actions may affect future states and task-relevant outcomes. Recent advances in world models, video generation, and Vision-Language-Action (VLA) policies have motivated the development of World-Action Models (WAMs), which couple future world prediction with executable action generation. This survey provides a robotics-oriented review of WAMs. We clarify their scope relative to conventional world models, model-based reinforcement learning, action-conditioned video generation, and reactive VLA policies, and organize existing methods through a unified taxonomy covering representations, transition modeling, action interfaces, architectures, training pipelines, data modalities, and scaling strategies. We further review applications of WAMs in manipulation, navigation, and autonomous driving, and we summarize the datasets, benchmarks, metrics, and protocols used to evaluate WAM systems. Finally, we discuss key challenges in action alignment, world-action factorization, spatial and multi-view consistency, long-horizon memory, neural simulation for closed-loop policy learning, and efficient inference. Taken together, this survey aims to provide a concise technical foundation for integrating predictive world modeling with action generation, toward more reliable embodied robot intelligence.

Overview

World-Action Models (WAMs) connect future-state modeling with executable action generation to support predictive robot control. World models learn how the environment evolves, often conditioned on actions; VLA policies map observations and language to actions. In our survey, WAMs connect these capabilities within a shared learning or inference process, so action generation is informed by predicted consequences.

Components of WAMs: language and visual encoders condition a predictive backbone, which connects world prediction and action decoding to robot execution. — **Component view.** Language provides task conditioning, the visual encoder defines the predictive state, the backbone models dynamics, and the action decoder produces executable controls.

Architecture

We categorize WAMs along two independent axes: architecture and transition modeling. One Model uses a shared backbone, while Dual-system separates world and action experts. Joint prediction couples future observation and action generation; inverse dynamics (IDM) predicts future states and then infers executable actions. Some IDM-style systems learn this interface implicitly and decode actions without explicitly generating a full future rollout at inference.

Training and data

Training WAMs generally follows a two-stage pipeline: pre-training and post-training. Pre-training learns spatiotemporal and action representations from large-scale video data, while post-training adapts the model through fine-tuning, data augmentation, and reinforcement learning. Different data sources provide complementary supervision: internet videos and egocentric demonstrations mainly provide visual dynamics priors, while embodiment-specific trajectories supply direct action supervision.

Policy training pipeline: pre-training includes spatiotemporal representation, action representation, and video–action alignment; post-training includes policy fine-tuning, data augmentation, and reinforcement learning. — **Policy training pipeline for WAMs.** (a) Pre-training leverages internet-scale video data to learn spatiotemporal and action representations and align video with actions; (b) post-training adapts the pretrained model through policy fine-tuning, data augmentation, and reinforcement learning.

Data modality and scaling structure. The left panel shows language, multi-view videos, 3D geometry, contact and auditory signals, and proprioception. The right panel shows the data pyramid of embodied trajectories, egocentric demonstrations, and internet videos. — **Data modality and scaling structure for World-Action Models.** The left panel illustrates how different sensing modalities expose complementary action-relevant state variables, while the right panel shows a data-pyramid view in which large action-free video corpora provide broad world priors and smaller embodied trajectories provide action grounding.

Applications

WAMs are used differently across embodied domains. Their value may lie in representation learning before policy training, in imagined look-ahead during inference, or in synthetic trajectories used for policy improvement. We therefore organize this section by application domain, including robot manipulation, navigation, and autonomous driving, and examine the role that prediction plays in each domain.

Citation

If you find our survey or WAMs resources useful in your research, please cite our paper.

@misc{lu2026worldactionmodels,
  title         = {{World-Action Models for Robot Learning and Control: A Survey}},
  author        = {Zuxing Lu and Hongjia Zhai and Guanzhi Wang and Huajian Zeng and
                   Jiaqi Yang and Jingyu Liu and Lei Cheng and Yuantai Zhang and
                   Yuheng Qiu and Zezhou Cheng and Ivan Laptev and Danfei Xu and
                   Benjamin Riviere and Giuseppe Loianno and Eric Xing and Xingxing Zuo},
  year          = {2026},
  eprint        = {2609.16074},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url           = {https://arxiv.org/abs/2609.16074}
}

arXiv:2609.16074 [cs.RO]