Research map
Browse the collection by category, architecture and prediction paradigm, or research topic.
Category board
Scroll each category to browse references. Expand subcategories, or open the paper index to combine filters.
Foundational work
Theory, planning, and foundational world-model research
Subcategories 10
Critique of World Model
Eric Xing; Mingkai Deng; Jinyu Hou
Precise and Dexterous Robotic Manipulation via Human-in-the-Loop Reinforcement Learning
Jianlan Luo; Charles Xu; Jeffrey Wu; Sergey Levine
Mean Flows for One-step Generative Modeling
Zhengyang Geng; Mingyang Deng; Xingjian Bai; Zico Kolter; Kaiming He
AdaWorld: Learning Adaptable World Models with Latent Actions
Shenyuan Gao; Siyuan Zhou; Yilun Du; Jun Zhang; Chuang Gan
Mastering diverse control tasks through world models
Danijar Hafner; Jurgis Pasukonis; Jimmy Ba; Timothy Lillicrap
DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning
Gaoyue Zhou; Hengkai Pan; Yann LeCun; Lerrel Pinto
Diffusion policy: Visuomotor policy learning via action diffusion
Cheng Chi; Zhenjia Xu; Siyuan Feng; Eric Cousineau; Yilun Du; Benjamin Burchfiel; Russ Tedrake; Shuran Song
Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining
Jie Cheng; Ruixi Qiao; Yingwei Ma; Binhua Li; Gang Xiong; Qinghai Miao; Yongbin Li; Yisheng Lv
World Models via Policy-Guided Trajectory Diffusion
Marc Rigter; Jun Yamada; Ingmar Posner
State Estimation for Robotics
Timothy D. Barfoot
TD-MPC2: Scalable, Robust World Models for Continuous Control
Nicklas Hansen; Hao Su; Xiaolong Wang
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
Jianlan Luo; Zheyuan Hu; Charles Xu; You Liang Tan; Jacob Berg; Archit Sharma; Stefan Schaal; Chelsea Finn; Abhishek Gupta; Sergey Levine
Improved Distribution Matching Distillation for Fast Image Synthesis
Tianwei Yin; Michaël Gharbi; Taesung Park; Richard Zhang; Eli Shechtman; Frédo Durand; William T. Freeman
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Rishabh Agarwal; Nino Vieillard; Yongchao Zhou; Piotr Stanczyk; Sabela Ramos Garea; Matthieu Geist; Olivier Bachem
Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning
Gautham Vasan; Yan Wang; Fahim Shahriar; James Bergstra; Martin Jägersand; A. Rupam Mahmood
One-step Diffusion with Distribution Matching Distillation
Tianwei Yin; Michaël Gharbi; Richard Zhang; Eli Shechtman; Frédo Durand; William T. Freeman; Taesung Park
Diffusion for World Modeling: Visual Details Matter in Atari
Eloi Alonso; Adam Jelley; Vincent Micheli; Anssi Kanervisto; Amos Storkey; Tim Pearce; François Fleuret
Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
Xingchao Liu; Chengyue Gong; Qiang Liu
DayDreamer: World Models for Physical Robot Learning
Philipp Wu; Alejandro Escontrela; Danijar Hafner; Pieter Abbeel; Ken Goldberg
Transformers are Sample-Efficient World Models
Vincent Micheli; Eloi Alonso; François Fleuret
Consistency Models
Yang Song; Prafulla Dhariwal; Mark Chen; Ilya Sutskever
Temporal Difference Learning for Model Predictive Control
Nicklas A Hansen; Hao Su; Xiaolong Wang
Rapid Exploration for Open-World Navigation with Latent Goal Models
Dhruv Shah; Benjamin Eysenbach; Nicholas Rhinehart; Sergey Levine
LoRA: Low-Rank Adaptation of Large Language Models
Edward J. Hu; Yelong Shen; Phillip Wallis; Zeyuan Allen-Zhu; Yuanzhi Li; Shean Wang; Lu Wang; Weizhu Chen
Contrastive Learning as Goal-Conditioned Reinforcement Learning
Benjamin Eysenbach; Tianjun Zhang; Sergey Levine; Russ R. Salakhutdinov
Mastering Atari with Discrete World Models
Danijar Hafner; Timothy Lillicrap; Mohammad Norouzi; Jimmy Ba
Model-Based Reinforcement Learning for Atari
Lukasz Kaiser; Mohammad Babaeizadeh; Piotr Milos; Blazej Osinski; Roy H. Campbell; Konrad Czechowski; Dumitru Erhan; Chelsea Finn; Piotr Kozakowski; Sergey Levine; Afroz Mohiuddin; Ryan Sepassi; George Tucker; Henryk Michalewski
Dream to Control: Learning Behaviors by Latent Imagination
Danijar Hafner; Timothy Lillicrap; Jimmy Ba; Mohammad Norouzi
Learning to summarize with human feedback
Nisan Stiennon; Long Ouyang; Jeffrey Wu; Daniel Ziegler; Ryan Lowe; Chelsea Voss; Alec Radford; Dario Amodei; Paul F. Christiano
When to Trust Your Model: Model-Based Policy Optimization
Michael Janner; Justin Fu; Marvin Zhang; Sergey Levine
Diversity is all you need: Learning skills without a reward function
Benjamin Eysenbach; Abhishek Gupta; Julian Ibarz; Sergey Levine
Learning Latent Dynamics for Planning from Pixels
Danijar Hafner; Timothy Lillicrap; Ian Fischer; Ruben Villegas; David Ha; Honglak Lee; James Davidson
World Models
David Ha; Jürgen Schmidhuber
Sim-to-Real Transfer of Robotic Control with Dynamics Randomization
Xue Bin Peng; Marcin Andrychowicz; Wojciech Zaremba; Pieter Abbeel
Factor Graphs for Robot Perception
Frank Dellaert; Michael Kaess
Deep Reinforcement Learning from Human Preferences
Paul F. Christiano; Jan Leike; Tom Brown; Miljan Martic; Shane Legg; Dario Amodei
Monte-Carlo Planning in Large POMDPs
David Silver; Joel Veness
Probabilistic Robotics
Sebastian Thrun; Wolfram Burgard; Dieter Fox
Real-time humanoid motion generation through ZMP manipulation based on inverted pendulum control
Tomomichi Sugihara; Yoshihiko Nakamura; Hirochika Inoue
Planning and acting in partially observable stochastic domains
Leslie Pack Kaelbling; Michael L. Littman; Anthony R. Cassandra
Reinforcement Learning: An Introduction
Richard S. Sutton; Andrew G. Barto
Model predictive control: Theory and practice—A survey
Carlos E. García; David M. Prett; Manfred Morari
VLA
Vision, language, and action models
Subcategories 15
Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models
Yang, Senqiao; Wang, Chengyao; Chen, Yuxin; Wang, Zixuan; Tang, Longxiang; Gui, Haokun; Ye, Jinhui; Lu, Changsheng; Wu, Xiaoyang; Zhu, Mingkang; Chen, Pengguang; Liu, Shu; Tian, Zhuotao; Zhao, Hengshuang; Yu, Bei; Jia, Jiaya
What Matters for Latent Actions in Robot Learning
Xizhou Bu; Qingda Hu; Lei Zhou; Lingfeng Zhang; Yingbo Tang; Zihao Liu; Xinyi Tao; Zhiqiang Ma; Qingqiu Huang; Chufeng Tang; Hongbo Wang; Jing Zhang; Jiayi Ma; Hangjun Ye; Wei Li; Xiaoshuai Hao
SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation
Wang, Jingkai; Tang, Zihan; Zhang, Gu; Cao, Mingyu; Chen, Jiapeng; Zhao, Jingjiao; Chen, Xiansheng; Wang, Pengwei; Liu, Lemao; Dou, Dejing
Artificial Foveated Perception for Mitigating Shortcut Learning in Robotic Foundation Models
Sun, Xiatao; Zhuang, Yuan; Negrete, Mateo Sanchez Lopez; Coldea, Matei-Victor; Liang, Chen; Zhang, Haoyang; Liu, Che; Zeng, Ziyao; Li, Shawn; Wang, Qian; Miao, Fei; Rakita, Daniel
Harness VLA: Steering Frozen VLAs into Reliable Manipulation Primitives via Memory-Guided Agents
Yixian Zhang; Huanming Zhang; Feng Gao; Xiao Li; Zhihao Liu; Chunyang Zhu; Jiaxing Qiu; Yuchen Yan; Jiyuan Liu; Wenhao Tang; Zhengru Fang; Yi Nie; Changxu Wei; Yu Wang; Wenbo Ding; Chao Yu
dVLA-RL: Reinforcement Learning over Denoising Trajectories for Discrete Diffusion Vision-Language-Action Models
Wu, Yuhao; Liu, Yitian; Shen, Weijie; Han, Mishuo; Xu, Wenjie; Liang, Haotian; Liu, Zhongshan; Mao, Yinan; Xu, Lei; Guan, Xinping; Ying, Ru; Zheng, Ran; Sui, Wei; Yang, Xiaokang; Ding, Wenbo; Mu, Yao
EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies
Ganlin Yang; Zhangzheng Tu; Yuqiang Yang; Sitong Mao; Junyi Dong; Tianxing Chen; Jiaqi Peng; Jing Xiong; Jiafei Cao; Jifeng Dai; Wengang Zhou; Yao Mu; Tai Wang
HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation
Sun, Xiaoquan; Zhang, Ruijian; Cao, Chen; Sun, Yihan; Chen, Jiahui; Xu, Zetian; Chen, Bo; Chen, Haijier; Yang, Zhen; Zhu, Jiarun; Hong, Yijun; Xu, JingZhe; Pang, Jingrui; Yuan, Mingqi; Chen, Jiayu
AttenA+: Rectifying Action Inequality in Robotic Foundation Models
Peng, Daojie; Ma, Fulong; Cao, Jiahang; Zhang, Qiang; Xie, Xupeng; Guo, Jian; Luo, Ping; Luo, Andrew F.; Zhou, Boyu; Ma, Jun
OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation
Chen, Xinzhe; Ren, Sihua; Huang, Liqi; Sun, Haowen; Li, Mingyang; Chen, Xingyu; Liu, Zeyang; Lan, Xuguang
Key-Gram: Extensible World Knowledge for Embodied Manipulation
Fan, Jingjing; Li, Siyuan; Ren, Botao; Deng, Zhidong
Geometry Guided Self-Consistency for Physical AI
Dai, Yinwei; Chen, Zhuofu; Yang, Lijie; Netravali, Ravi
RL Token: Bootstrapping Online RL with Vision-Language-Action Models
Charles Xu; Jost Tobias Springenberg; Michael Equi; Ali Amin; Adnan Esmail; Sergey Levine; Liyiming Ke
UniDriveVLA: Unifying Understanding, Perception, and Action Planning for Autonomous Driving
Yongkang Li; Lijun Zhou; Sixu Yan; Bencheng Liao; Tianyi Yan; Kaixin Xiong; Long Chen; Hongwei Xie; Bing Wang; Guang Chen; Hangjun Ye; Wenyu Liu; Haiyang Sun; Xinggang Wang
MEM: Multi-Scale Embodied Memory for Vision Language Action Models
Marcel Torne; Karl Pertsch; Homer Walke; Kyle Vedder; Suraj Nair; Brian Ichter; Allen Z. Ren; Haohuan Wang; Jiaming Tang; Kyle Stachowicz; Karan Dhabalia; Michael Equi; Quan Vuong; Jost Tobias Springenberg; Sergey Levine; Chelsea Finn; Danny Driess
Unifying Language-Action Understanding and Generation for Autonomous Driving
Xinyang Wang; Qian Liu; Wenjie Ding; Zhao Yang; Wei Li; Chang Liu; Bailin Li; Kun Zhan; Xianpeng Lang; Wei Chen
EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data
Ruijie Zheng; Dantong Niu; Yuqi Xie; Jing Wang; Mengda Xu; Yunfan Jiang; Fernando Castañeda; Fengyuan Hu; You Liang Tan; Letian Fu; Trevor Darrell; Furong Huang; Yuke Zhu; Danfei Xu; Linxi Fan
VideoWorld 2: Learning Transferable Knowledge from Real-world Videos
Zhongwei Ren; Yunchao Wei; Xiao Yu; Guixun Luo; Yao Zhao; Bingyi Kang; Jiashi Feng; Xiaojie Jin
CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos
Chubin Zhang; Jianan Wang; Zifeng Gao; Yue Su; Tianru Dai; Cai Zhou; Jiwen Lu; Yansong Tang
Vtla: Vision-tactile-language-action model with preference learning for insertion manipulation
Chaofan Zhang; Peng Hao; Xiaoge Cao; Xiaoshuai Hao; Shaowei Cui; Shuo Wang
Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Driving
Jianhua Han; Meng Tian; Jiangtong Zhu; Fan He; Huixin Zhang; Sitong Guo; Dechang Zhu; Hao Tang; Pei Xu; Yuze Guo; Minzhe Niu; Haojie Zhu; Qichao Dong; Xuechao Yan; Siyuan Dong; Lu Hou; Qingqiu Huang; Xiaosong Jia; Hang Xu
OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
Xingcheng Zhou; Xuyuan Han; Feng Yang; Yunpu Ma; Volker Tresp; Alois Knoll
Hierarchical Latent Action Model
Hanjung Kim; Lerrel Pinto; Seon Joo Kim
AutoMoT: A Unified Vision-Language-Action Model with Asynchronous Mixture-of-Transformers for End-to-End Autonomous Driving
Wenhui Huang; Songyan Zhang; Qihang Huang; Zhidong Wang; Zhiqi Mao; Collister Chua; Zhan Chen; Long Chen; Chen Lv
A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation
TRI LBM Team; Jose Barreiros; Andrew Beaulieu; Aditya Bhat; Rick Cory; Eric Cousineau; Hongkai Dai; Ching-Hsin Fang; Kunimatsu Hashimoto; Muhammad Zubair Irshad; Masha Itkina; Naveen Kuppuswamy; Kuan-Hui Lee; Katherine Liu; Dale McConachie; Ian McMahon; Haruki Nishimura; Calder Phillips-Grafflin; Charles Richter; Paarth Shah; Krishnan Srinivasan; Blake Wulfe; Chen Xu; Mengchao Zhang; Alex Alspach; Maya Angeles; Kushal Arora; Vitor Campagnolo Guizilini; Alejandro Castro; Dian Chen; Ting-Sheng Chu; Sam Creasey; Sean Curtis; Richard Denitto; Emma Dixon; Eric Dusel; Matthew Ferreira; Aimee Goncalves; Grant Gould; Damrong Guoy; Swati Gupta; Xuchen Han; Kyle Hatch; Brendan Hathaway; Allison Henry; Hillel Hochsztein; Phoebe Horgan; Shun Iwase; Donovon Jackson; Siddharth Karamcheti; Sedrick Keh; Joseph Masterjohn; Jean Mercat; Patrick Miller; Paul Mitiguy; Tony Nguyen; Jeremy Nimmer; Yuki Noguchi; Reko Ong; Aykut Onol; Owen Pfannenstiehl; Richard Poyner; Leticia Priebe Mendes Rocha; Gordon Richardson; Christopher Rodriguez; Derick Seale; Michael Sherman; Mariah Smith-Jones; David Tago; Pavel Tokmakov; Matthew Tran; Basile Van Hoorick; Igor Vasiljevic; Sergey Zakharov; Mark Zolotas; Rares Ambrus; Kerri Fetzer-Borelli; Benjamin Burchfiel; Hadas Kress-Gazit; Siyuan Feng; Stacie Ford; Russ Tedrake
Spatial Forcing: Implicit Spatial Representation Alignment for Vision-Language-Action Model
Fuhao Li; Wenxuan Song; Han Zhao; Jingbo Wang; Pengxiang Ding; Donglin Wang; Long Zeng; Haoang Li
Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation
Meng Wei; Chenyang Wan; Jiaqi Peng; Xiqian Yu; Yuqiang Yang; Delin Feng; Wenzhe Cai; Chenming Zhu; Tai Wang; Jiangmiao Pang; Xihui Liu
Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modeling
Yueru Jia; Jiaming Liu; Shengbang Liu; Rui Zhou; Wanhe Yu; Yuyang Yan; Xiaowei Chi; Yandong Guo; Boxin Shi; Shanghang Zhang
LatBot: Distilling Universal Latent Actions for Vision-Language-Action Models
Zuolei Li; Xingyu Gao; Xiaofan Wang; Jianlong Fu
π^*_0.6: a VLA That Learns From Experience
Physical Intelligence; Ali Amin; Raichelle Aniceto; Ashwin Balakrishna; Kevin Black; Ken Conley; Grace Connors; James Darpinian; Karan Dhabalia; Jared DiCarlo; Danny Driess; Michael Equi; Adnan Esmail; Yunhao Fang; Chelsea Finn; Catherine Glossop; Thomas Godden; Ivan Goryachev; Lachy Groom; Hunter Hancock; Karol Hausman; Gashon Hussein; Brian Ichter; Szymon Jakubczak; Rowan Jen; Tim Jones; Ben Katz; Liyiming Ke; Chandra Kuchi; Marinda Lamb; Devin LeBlanc; Sergey Levine; Adrian Li-Bell; Yao Lu; Vishnu Mano; Mohith Mothukuri; Suraj Nair; Karl Pertsch; Allen Z. Ren; Charvi Sharma; Lucy Xiaoyang Shi; Laura Smith; Jost Tobias Springenberg; Kyle Stachowicz; Will Stoeckle; Alex Swerdlow; James Tanner; Marcel Torne; Quan Vuong; Anna Walling; Haohuan Wang; Blake Williams; Sukwon Yoo; Lili Yu; Ury Zhilinsky; Zhiyuan Zhou
π_RL: Online RL Fine-tuning for Flow-based Vision-Language-Action Models
Kang Chen; Zhihao Liu; Tonghe Zhang; Zhen Guo; Si Xu; Hao Lin; Hongzhi Zang; Xiang Li; Quanlu Zhang; Zhaofei Yu; Guoliang Fan; Tiejun Huang; Yu Wang; Chao Yu
GigaBrain-0: A World Model-Powered Vision-Language-Action Model
GigaBrain Team; Angen Ye; Boyuan Wang; Chaojun Ni; Guan Huang; Guosheng Zhao; Haoyun Li; Jie Li; Jiagang Zhu; Lv Feng; Peng Li; Qiuping Deng; Runqi Ouyang; Wenkang Qin; Xinze Chen; Xiaofeng Wang; Yang Wang; Yifan Li; Yilong Li; Yiran Ding; Yuan Xu; Yun Ye; Yukun Zhou; Zhehao Dong; Zhenan Wang; Zhichao Liu; Zheng Zhu
StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling
Meng Wei; Chenyang Wan; Xiqian Yu; Tai Wang; Yuqiang Yang; Xiaohan Mao; Chenming Zhu; Wenzhe Cai; Hanqing Wang; Yilun Chen; Xihui Liu; Jiangmiao Pang
UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
Qingwen Bu; Yanting Yang; Jisong Cai; Shenyuan Gao; Guanghui Ren; Maoqing Yao; Ping Luo; Hongyang Li
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
NVIDIA; :; Johan Bjorck; Fernando Castañeda; Nikita Cherniadev; Xingye Da; Runyu Ding; Linxi "Jim" Fan; Yu Fang; Dieter Fox; Fengyuan Hu; Spencer Huang; Joel Jang; Zhenyu Jiang; Jan Kautz; Kaushil Kundalia; Lawrence Lao; Zhiqi Li; Zongyu Lin; Kevin Lin; Guilin Liu; Edith Llontop; Loic Magne; Ajay Mandlekar; Avnish Narayan; Soroush Nasiriany; Scott Reed; You Liang Tan; Guanzhi Wang; Zu Wang; Jing Wang; Qi Wang; Jiannan Xiang; Yuqi Xie; Yinzhen Xu; Zhenjia Xu; Seonghyeon Ye; Zhiding Yu; Ao Zhang; Hao Zhang; Yizhou Zhao; Ruijie Zheng; Yuke Zhu
Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
Moo Jin Kim; Chelsea Finn; Percy Liang
Helix: A Vision-Language-Action Model for Generalist Humanoid Control
Figure AI
π₀.₅: a Vision-Language-Action Model with Open-World Generalization
Kevin Black; Noah Brown; James Darpinian; Karan Dhabalia; Danny Driess; Adnan Esmail; Michael Robert Equi; Chelsea Finn; Niccolo Fusai; Manuel Y. Galliker; Dibya Ghosh; Lachy Groom; Karol Hausman; brian ichter; Szymon Jakubczak; Tim Jones; Liyiming Ke; Devin LeBlanc; Sergey Levine; Adrian Li-Bell; Mohith Mothukuri; Suraj Nair; Karl Pertsch; Allen Z. Ren; Lucy Xiaoyang Shi; Laura Smith; Jost Tobias Springenberg; Kyle Stachowicz; James Tanner; Quan Vuong; Homer Walke; Anna Walling; Haohuan Wang; Lili Yu; Ury Zhilinsky
AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
Zewei Zhou; Tianhui Cai; Seth Zhao; Yun Zhang; Zhiyu Huang; Bolei Zhou; Jiaqi Ma
π₀: A Vision-Language-Action Flow Model for General Robot Control
Kevin Black; Noah Brown; Danny Driess; Adnan Esmail; Michael Robert Equi; Chelsea Finn; Niccolo Fusai; Lachy Groom; Karol Hausman; Brian Ichter; Szymon Jakubczak; Tim Jones; Liyiming Ke; Sergey Levine; Adrian Li-Bell; Mohith Mothukuri; Suraj Nair; Karl Pertsch; Lucy Xiaoyang Shi; Laura Smith; James Tanner; Quan Vuong; Anna Walling; Haohuan Wang; Ury Zhilinsky
OpenVLA: An Open-Source Vision-Language-Action Model
Moo Jin Kim; Karl Pertsch; Siddharth Karamcheti; Ted Xiao; Ashwin Balakrishna; Suraj Nair; Rafael Rafailov; Ethan Paul Foster; Pannag R. Sanketi; Quan Vuong; Thomas Kollar; Benjamin Burchfiel; Russ Tedrake; Dorsa Sadigh; Sergey Levine; Percy Liang; Chelsea Finn
Octo: An Open-Source Generalist Robot Policy
Dibya Ghosh; Homer Rich Walke; Karl Pertsch; Kevin Black; Oier Mees; Sudeep Dasari; Joey Hejna; Tobias Kreiman; Charles Xu; Jianlan Luo; You Liang Tan; Lawrence Yunliang Chen; Quan Vuong; Ted Xiao; Pannag R. Sanketi; Dorsa Sadigh; Chelsea Finn; Sergey Levine
Vision-Language Foundation Models as Effective Robot Imitators
Xinghang Li; Minghuan Liu; Hanbo Zhang; Cunjun Yu; Jie Xu; Hongtao Wu; Chilam Cheang; Ya Jing; Weinan Zhang; Huaping Liu; Hang Li; Tao Kong
RT-1: Robotics Transformer for Real-World Control at Scale
Anthony Brohan; Noah Brown; Justice Carbajal; Yevgen Chebotar; Joseph Dabis; Chelsea Finn; Keerthana Gopalakrishnan; Karol Hausman; Alexander Herzog; Jasmine Hsu; Julian Ibarz; Brian Ichter; Alex Irpan; Tomas Jackson; Sally Jesmonth; Nikhil J. Joshi; Ryan Julian; Dmitry Kalashnikov; Yuheng Kuang; Isabel Leal; Kuang-Huei Lee; Sergey Levine; Yao Lu; Utsav Malla; Deeksha Manjunath; Igor Mordatch; Ofir Nachum; Carolina Parada; Jodilyn Peralta; Emily Perez; Karl Pertsch; Jornell Quiambao; Kanishka Rao; Michael S. Ryoo; Grecia Salazar; Pannag R. Sanketi; Kevin Sayed; Jaspiar Singh; Sumedh Sontakke; Austin Stone; Clayton Tan; Huong T. Tran; Vincent Vanhoucke; Steve Vega; Quan Vuong; Fei Xia; Ted Xiao; Peng Xu; Sichun Xu; Tianhe Yu; Brianna Zitkovich
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brianna Zitkovich; Tianhe Yu; Sichun Xu; Peng Xu; Ted Xiao; Fei Xia; Jialin Wu; Paul Wohlhart; Stefan Welker; Ayzaan Wahid; Quan Vuong; Vincent Vanhoucke; Huong T. Tran; Radu Soricut; Anikait Singh; Jaspiar Singh; Pierre Sermanet; Pannag R. Sanketi; Grecia Salazar; Michael S. Ryoo; Krista Reymann; Kanishka Rao; Karl Pertsch; Igor Mordatch; Henryk Michalewski; Yao Lu; Sergey Levine; Lisa Lee; Tsang-Wei Edward Lee; Isabel Leal; Yuheng Kuang; Dmitry Kalashnikov; Ryan Julian; Nikhil J. Joshi; Alex Irpan; Brian Ichter; Jasmine Hsu; Alexander Herzog; Karol Hausman; Keerthana Gopalakrishnan; Chuyuan Fu; Pete Florence; Chelsea Finn; Kumar Avinava Dubey; Danny Driess; Tianli Ding; Krzysztof Marcin Choromanski; Xi Chen; Yevgen Chebotar; Justice Carbajal; Noah Brown; Anthony Brohan; Montserrat Gonzalez Arenas; Kehang Han
Advancing AI for the physical world
Microsoft Research
WAMs
World modeling and action generation
Subcategories 20
DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation
Nisarga Nilavadi; Ralf Römer; Moritz Reuss; Michael Krawez; Tobias Jülg; Angela P. Schoellig; Rudolf Lioutikov; Wolfram Burgard
Grounding Generated Video Plans in Simulation Towards Versatile Dexterous Controllers
Tianyue Wu; Boyuan An; Shuqi Zhao; Heyu Guo; Wanli Xing; Yi Ma; Kaifeng Zhang; Ruihai Wu; Masayoshi Tomizuka
HaWMPO: Hallucination-Aware World Model-based Policy Optimization for Generalist Robot Policy
Zengjue Chen; Peidong Liu; Jiawei Li; Qi Wang
JEPA Policy: Diffusion-Free Imitation Learning via Paired Action and Future Representation Prediction
Jie Xu; Kangjin Yu; Ziyi Jin; Junjie Gao; Liqing Chen; Yixian Li; Shuai Tian; Zhongpu Xia
Compact Visuotactile World Models for Lifting: Prediction, Reward Alignment, and Force Constraints
Qinzhen Ma; Sida Peng
SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators
Yuncong Yang; Zhengtao Han; Furkan Ozyurt; Zeyuan Yang; Han Yang; Junyi Cao; Haoyu Zhen; Yilun Du; Chuang Gan
WorldAgen: Unified State-Action Prediction with Test-Time World Model Training
Chi Wan; Kangrui Wang; Yuan Si; Pingyue Zhang; Manling Li
OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
Yuran Wang; Siqiao Huang; Mingleyang Li; Chenhao Zhang; Jiaqi Liang; Weiyang Jin; Yue Chen; Xuemin Chi; Donghao Zhou; Qize Yu; Yu-Kai Wang; Yuhan Rui; Shenzhe Yao; Zhen Yuan; Zhenhao Shen; Kefei Zhu; Zijie Zhu; Ning Gao; Xiaowei Chi; Guanqi He; Shanghang Zhang; Hao Dong; Lin Shao; Hang Zhao
WM-Craftnet: World Synesthesia Model for Generalizable and Robust Dexterous In-Hand Manipulation
Jie Yin; Zeyuan Zhao; Xiaojing Tan; Yang Liu; Chiyu Wang; Xinyang Gu
Learning to Use Imagination: Progress-Conditioned Future Utilization for World Action Models
Yijie Zhu; Zitong Yu; Wei Li; Hui Ma; Wen Li; Rui Shao; Liqiang Nie
How to Learn from What a Human Would Avoid? Intervention-Aware World Models with Real-World RL for Dexterous Manipulation
Jiaju Yin; Zhenhui Zhang; Lixin Xu; Heng Zhang; Jun Shao; Yating Feng; Arash Ajoudani; Renjing Xu
Learning Counterfactual World Models for Embodied Reasoning under Partial Observability
Todd Y. Zhou; Daniel Zhang
GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
AgiBot Research Team
TacPAC: Tactile Prediction and Real-Time Action Correction in World-Action Models for Contact-Rich Manipulation
Zipei Ma; Xiaofei Wei; Junzhe Jiang; Shunlin Lu; Li Zhang
GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
Yupeng Zheng; Xiang Li; Songen Gu; Yuhang Zheng; Shuai Tian; Weize Li; Linbo Wang; Chaoyue Li; Qichao Zhang; Haoran Li; Zhongpu Xia; Ya-Qin Zhang; Shuicheng Yan; Dongbin Zhao
WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models
Chenhao Zhang; Hanyu Zhao; Hang Cheng; Tengfei Pan; Long Zeng
SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
Jinyang Wang; Shiwei Li; Junjian Wang; Zhiqiang Deng; Jianbin Gao; Yihang Zhao; Liu Liu; Yongjia Zhao; Jinlong Chen; Huirui Xu; Yifeng Pan; Kangwei Liu; Fan Ren; Ji Tao; Minghao Yang
Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
Zhaoxin Fan; Tianbao Zhang; Wenjun Wu; Xiaofeng Wang; Yeying Jin; Jian Zhao; Zheng Zhu; Shuicheng Yan
Toward Physically Grounded JEPA World Models for Goal-Conditioned Robotic Planning
Muyuan Liu; Yue Huang; Zheng Liang; Xiang Gao
Latent Energy Action Planning with World Models
Phu Pham and Aniket Bera
Spatially Aware World Action Model via Geometric Latent Diffusion
Gonzalez, Javier Alejandro Lopetegui; Pacaud, Paul; Schmid, Cordelia
World-Coherent Decoding: Self-Verifying Test-Time Planning for World Action Models
Zhang, Chuhan; Ito, Seiji; Hoshino, Kenta; Ikehata, Satoshi; Sato, Ikuro
Latent Action as Intention Enables Efficient Future Imagination for World Action Models
Li, Xiang; Zheng, Yupeng; Gu, Songen; Ma, Huailiang; Yu, Feng; Zheng, Yuhang; Nie, Xian; Yuan, Shanshuai; Zang, Yujie; Li, Weize; Tian, Shuai; Liu, Moyang; Zhang, Ya-Qin; Ding, Wenchao
ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training
Wu, Xionghao; Yang, Yijun; Zhou, Shiyang; Sun, Haoze; Liu, Jianhui; Yu, Songsong; Zhang, Jiyao; Li, Wenbo; Wang, Bo; Ma, Guoqing; Song, Lin; Liao, Renjie; Zheng, Shenghe; Tang, Wei; Qi, Xiaojuan; Li, Yanwei; Zhang, Yuan; Tian, Zhuotao; Huang, Haoyang; Duan, Nan
Motus2: A Self-Evolving General World Model for Dexterous Manipulation
Bi, Hongzhe; Zhou, Zihao; Tang, Yihang; Pang, Jingrui; Huang, Shuhe; Liu, Haitian; Wang, Runqing; Huang, Shuai; Wang, Yichen; Cheng, Yiming; Zhao, Ruowen; Li, Zhenghua; Tan, Hengkai; Liu, Xiaolong; Wan, Jinhui; Liu, Jiabao; Zhao, Min; Bao, Fan; Zhu, Jun
DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
Lei, Fenghao; Huang, Zhixiong; Yang, Long; Chen, Jiabao; Huang, Peilin; Fu, Han; Li, Zhuo; Ren, Xiaoxue
AcrossWAM1.0:A Modular Latent World-Action Stack for Compact Robot Policies
Zhang, Yafei; Wu, Nan
Flex-ππ: A Multi-Stream World-Action Model with Compute Flexibility
Yan, Ge; Liu, Jinghao; Fan, Yuzhi; Cai, Lei; Liao, Minwen; Zhang, Jesse; Fox, Dieter
Hydra: A Navigation World Action Model with Discrete Latent Planning and Continuous Flow-Matching Execution
Nazeri, Mohammad; Card, Alexandyr; Huber, Samira; Pokhrel, Anuj; Wang, Yujun; Hammele, Ruben; Song, Daeun; Pirk, Sören; Xiao, Xuesu
Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models
Lu, Xiaoxiao; Dong, Yunlong; Shi, Jiahao; Yuan, Ye
Riemann-1.0: An Embodied World Action Model for Physical AI
Sun, Haofeng; Pei, Jiangbo; Kang, Fei; Liu, Zexiang; Li, Yaokun; Jiang, Boyi; Xue, Hua; Zhou, Cindy; Li, Wei; Wei, Yichen; An, Mengyin; Zhao, Fanliang; Jiang, Biao; Wang, Zile; Liu, Yang; Li, Yangguang
Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
Zhou, Jiaming; Zhang, Qihang; Xu, Gangwei; Fan, Cunxin; Zhao, Yujie; Wang, Ruilin; Luo, Yiming; Yang, Shuai; Zhu, Xing; Shen, Yujun; Liang, Junwei; Xu, Yinghao
Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention
Wu, Siyu; You, Linjing; Zhu, Junjie; Liu, Yaozu; Kaixiang, Huang; Yonghang, Chen; Li, Jituo; Zhang, Changhao; Liu, Jian; Chu, Hengshuo; Li, Qi; Zhao, Hengshuang
4DGS-WAM: Bridging Past and Future with an Object-Centric World Action Model based on 4D Gaussian Splatting
Ma, Yueen; Xu, Zenglin; King, Irwin
SimWAM: A Simple World Action Model for End-to-End Autonomous Driving
Zhao, Zongchuang; Zhou, Xin; Xu, Tianyang; Sun, Zhengyang; Zhou, Kaixuan; Wu, Yu; Li, Honglin; Liang, Dingkang; Bai, Xiang
SANTS: A State-Adaptive Scheduler for World Action Models
Sun, Yirui; Zhuge, Guangyu; Liu, Keliang; Gu, Jie; Dai, Shiqin; Bing, Xinyu; Gan, Zhongxue; Tian, Chunxu
GameWAM: A World Action Model for Video Games
Guo, Yuncheng; Zhang, Zhanqiu; Guo, Yiwen; Li, Weijia
GaussianWAM: Distilling Geometry and Semantics from 3D Gaussian Fields into World-Action Models
Zhang, Zijian; Jiang, Yuqing; Zhou, Weitao; Li, Minglei; Zhang, Jinhao; Mu, Yao; Li, Xiaofan; Zhao, Hao; Yu, Haibao
GeoWAM: Visual Geometry World Action Models for Autonomous Driving
Lu, Yiren; Ye, Xin; Liu, Jiaming; Jacobson, Philip; Yao, Jin; Chen, Yi-chung; Merino, Liam; Kurra, Dhruva Dixith; Cai, Min; Lampo, Tom; Yin, Yu; Guo, Danhua; Yaman, Burhan
GlanceWAM: Sparse Test-Time Imagination for World-Action Models
Wang, Linhan; An, Zijian; Zhang, Mingyuan; Dai, Chen; Xu, Yi; Cui, Can; Yang, Zichong; Chen, Yinlin; Zhou, Lifeng; Lu, Chang-Tien
ReWorld: Representation Learning for World Action Models
Xia, Tianze; Zhou, Lijun; Xiong, Kaixin; Yao, Jingfeng; Zhu, Zhenxin; Sun, Haiyang; Wang, Bing; Chen, Guang; Liu, Wenyu; Ye, Hangjun; Wang, Xinggang
LD4WAM: Learning Latent Dynamics from Human Videos for World Action Models
Shen, Zhenhao; Liang, Jiaqi; Lu, Jasper; Jiang, Feng; Wang, Yuran; Wei, Chuanbo; Liu, Jiayi; Yang, Jianchun; Yu, Qize; You, Jiadi; Hao, Ce; He, Guanqi; Xie, Chen; Wu, Ruihai
WAM-OPD: On-Policy Distillation for World Action Models
Yang, Liuhaichen; Jiang, Zhuang; Sheng, Chenchao; Tang, Zezhi
ForeTime-VLA: Causal Future-Token Distillation from a World Action Model for Conveyor-Belt Manipulation
Ma, Siyuan; Zhang, Yutian; Zhang, Boshi; Wu, Qinglian; Zhai, Jiaqi; Wei, Dong; Huang, Xiaojin
TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction
Jin, Lei; Ma, Yiding; Zhang, Xin; Gao, Chen; Wu, Wei; Li, Yong
WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving
Wang, Xinlin; Xiang, Yujiao; Zhou, Yuheng; Wang, Jingqi; Huang, Minqing; Huang, Jiajie; Wei, Dongxu; Zhou, Tingguang; Wang, Xiyang; Chen, Gong; Xu, Zhi; Tan, Feiyang; Zhou, Hangning; Yang, Mu
DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation
Ma, Siyuan; Zhang, Boshi; Zhang, Yutian; Wu, Qinglian; Zhai, Jiaqi; Wei, Dong; Yu, Qiaojun
Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning
Huang, Weiliang; Liu, Huanrong; Zhang, Bob; Dou, Qi; Chen, Zhen; Gu, Yun; Rosman, Guy; Li, Qingbiao
RISE: Adaptive Imagination for World Action Models
Lu, Hongbo; Yao, Liang; He, Chenghao; Han, Hao; Liu, Fan; Liao, Wenlong; He, Tao; Peng, Pai
HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation
Xue, Chao; Zhang, Chaofan; Ma, Wenxuan; Yao, Guocai; Cui, Shaowei; Wang, Shuo
BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving
Zhan, Bing; Shang, Shuyao; Lu, Shuo; Xu, Yuan; Wang, Zhao; Wang, Yida; Zhang, Xueyang; Zhan, Kun; Gu, Jiahao
WNM-3D: A World Navigation Model with 3D Scene Conditioning for Closed-Loop VLN
Huang, Yuehao; Wu, Yunzi; Zhang, Xiaotao; Li, Xinhai; Dong, Jiankun; Lv, Jiajun; Zhang, Chi; Bai, Chenjia; Liu, Yong; Li, Xuelong
Hydra-0: Action Flow for Generalist World Modeling and Control
Li, Hongyu; Wen, Bowen; Zhu, Xinghao; Wang, Yixuan; Du, Yilun; Li, Yunzhu; Konidaris, George; Birchfield, Stan; Pouya, Soha; Li, Chenran; Chang, Yan
StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation
Liu, Xiao; Yang, Yuguang; Wang, Xi; Jiang, Kai; Chi, Cheng; Xu, Yong; Ding, Wenchao; Chen, Yilun; Wang, Yan
Keep the Future, Drop the Rollout: RIFT for World Action Models
Zhang, Chushan; Tong, Jinguang; Li, Xuesong; Wang, Yikai; Li, Hongdong
4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields
Yang, Lishan; Song, Wenxuan; Wang, Xi; Sheng, Pingyue; Fang, Zheng; Zhou, Ziyang; He, Junjie; Yan, Haodong; Chen, Jiayi; Sun, Nan; Sun, Qiao; Wang, Pengwei; Liu, Lingqiao; Wang, Yan; Gao, Yuxiang; Dayoub, Feras; Li, Haoang
Foresight Without Seeing: Latent Futures for World Action Models
Huang, Jiakai; Wu, Zhongbo; Zhang, Zheng; Wang, Zihan; You, Shan; Huang, Tao
Surgical WAM: A World-Action Model for Data-Efficient Surgical Robot Learning
Bao, Wenrui; Jiang, Tianyun; Chen, Zhiben; Lim, Ser-Nam; Peng, Peter D.; Shang, Yuzhang
FACT: Failure-Aware Causal Training for World-Action Models
Peng, Quanquan; Liang, Yutong; Yan, Rui; Hansen, Nicklas; Wang, Xiaolong
4D-WAM: 4D Consistent World Modeling for Autonomous Driving
Fu, Jiacheng; Yuan, Yibo; Tian, Meng; Li, Yue; Zhu, Jiangtong; Han, Jianhua; Zhang, Yueyi; Fang, Jianwu; Xue, Jianru; Xu, Hang; Xiong, Zhiwei
World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
Tang, Qu; Zhuang, Benhui; Yuan, Bo; Yu, Xue; Guo, Longteng; Feng, Junlan
HarnessWAM: Bridging Prediction and Deliberation in World Action Models
Gu, Zhaopeng; Zhu, Bingke; Lin, Tianxi; Zhu, Guibo; Chen, Yingying; Wang, Kai; Yuan, Tingyu; Zhao, Chaoyang; Li, Zhaowen; Su, Peng; Wang, Jinqiao
Rethink Before You Execute: Adaptive Execution for World Action Models
Ye, Feng; Zhao, Yiming; Yu, Yong; Zhou, Hongxu; Pan, Yong; Xue, Yuan; Jia, Peng; Jia, Chuanmin
JEPA-WAM: Learning Vision-Language-Action Policies with Joint-Embedding World Modeling
Lin, Yihan; He, Jiawei; Bao, Shifeng; Zhao, Chen; Li, Yang; Wang, Xiaobo; Wang, Yan; Chi, Cheng; Zhang, Jing
SG-WAM: Text-Grounded and Spatial-aware Semantic Guidance for World-Action Models
He, Junjie; Li, Junfeng; Zhong, Zhide; Yan, Haodong; Li, Ruixin; Zheng, Yangyang; Zhu, Jiaguan; Zhang, Tianran; Du, Yuqiao; Chen, Wen; Zhou, Shunbo; Li, Haoang
Vid2WAM: Distilling Video Diffusion Priors into World Action Models
Qiu, Chenhao; Wang, Ruixiang; Zhao, Runyi; Lin, Sixu; Gu, Songen; Nan, Shufeng; Liu, Guiliang; Jia, Kui; Fu, Yanwei; Wu, Simo
ω-0: A Latent Predictive World Action Model for Concurrent Humanoid Loco-Manipulation
Li, Zhe; Zhang, Zhenzhe; Wei, Yangyang; Zhang, Wenjie; Yuan, Xichen; Zhi, Peiyuan; Li, Gen; Guo, Xinying; Gao, Fengjie; Yang, Jianfei; Zhang, Shanghang
Selective Cross-View Consistency for World Action Models: Held-Out Viewpoint Robustness Without Test-Time Camera Information
Huang, Bingqi; Wei, Bingchuan; Cai, Yingkai; Wang, Zhaokui
Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Models
Ma, Xiangkai; Ma, Yue; Wang, Junjie; Xu, Sheng; Li, Mingyang; Zhang, Han; Zhuang, Yuzheng; Li, Wenzhong; Yuan, Zhihao
Robust-WAM: Bridging Generative Pretraining and Semantic Foresight in World-Action Models
Yan, Haodong; Li, Junfeng; He, Junjie; Zhong, Zhide; Yu, MingMing; Song, Wenxuan; Zhu, Jiaguan; Zheng, Yangyang; Du, Yuqiao; You, Jiadi; Cai, Yingjie; Yan, Xu; Zhao, Guanyi; Liu, Bingbing; Li, Haoang
Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features
Ang, Sining; Yang, Yuguang; Wang, Yan
MobileWAM: Bridging World Action Models to Mobile Manipulation with Chain-of-Foresight
Fan, Zehua; He, Junjie; Song, Wenxuan; Wang, Xi; Lyu, Wenqi; Zhao, Linge; Li, Fuhao; You, Zihan; Yang, Yifei; Xu, Kaiming; Jiang, Qi; Jiang, Yue; Li, Haoang; Chi, Cheng; Gao, Feng; Li, Bailin; Wang, Yan
DynamicWAM: Dual-Path Motion Conditioning for World-Action Models in Dynamic Manipulation
Lou, Yunfan; Gao, Hewen; Zhu, Xiyu; Qiao, Zhuoran; Han, Xuan; Yang, Yifan; Ye, Yifan; Yao, Boxian; Pang, Zhibo
DreamWAM: Beyond RGB Future Prediction for World Action Models
Yuan, Shanglin; Zhao, Weiheng; Shi, Xin; Jiang, Haoyi; Guo, Xianda; Liu, Liu; Liu, Wenyu; Sui, Wei; Wang, Xinggang
Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models
Zhao, Weiheng; Jiang, Haoyi; Shi, Xin; Liu, Liu; Huang, Fan; Su, Zhizhong; Sui, Wei; Wang, Xinggang
LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation
Yang, Fan; Su, Yuting; Wang, Xiaobo; You, Yuncheng; Fan, Fugui; Wu, Yuting; Wu, Minghui; Zhao, Chenxu; Ning, JiaHong; Jing, Peiguang
UniNav: A Unified World-Action Diffusion Model for Visual Navigation
Zhou, Changqing; Luo, Yueru; Jiang, Zeyu; Chen, Changhao
CoWAM: Coordination Contracts for Selective Policy Intervention with WAMs
Liu, Shuaijun; Wen, Qifu; Hao, Shuyang; Luo, Qi; Zhang, Chenglong; You, Feiyang; Wu, Chengyu; Su, Ningxin
Faster-WAM: Do World Action Models Need Deep Action Modules?
Ma, Liheng; Yang, Rui Heng; Zhang, Zhanguang; Clemente, Mateo; Hu, Ziwen; Cao, Tongtong; Zhang, Yingxue
SG-WAM: Self-Guided World Modeling in Geometry-Aware Policy Space
Zhao, Ruiteng; Zhang, Zhengshen; Su, Yue; Wang, Wenshuo; Li, Jiahui; Yang, Zhiyuan; Tay, Francis E. H.; Jr., Marcelo H. Ang; Zhu, Haiyue
EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation
Lin, Jinsong; Pan, Zikang; Liu, Wanhao; Ng, Chi Kit; Shao, Liangjing; Yu, Zihang; Wang, Ziyu; Wang, Yin; Wang, Jiaxi; Teoh, Jeremy Yuen-Chun; Xiong, Zhiyong; Gao, Huxin; Ren, Hongliang
SelfWAM: A Self-Grounded Unified World Action Model for Fast Robot Control
Pan, Bikang; Liu, Fan; Lu, Haotao; Wang, Jingya; Shi, Ye
FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation
Wang, Runqing; Yu, Ding; Min, Pengyuan; Zhang, Xinhong; Xiao, Wei; Hu, Yu; Chen, Jie; Zhang, Fu; Wang, Gang
Disentangling Visuo-Tactile Foresight: Oracle-Guided Interface Discovery for World Action Models
Yao, Zihang; Ding, Chaoyue; Yu, Yingying
FBFM: A Training-Free Asynchronous Feedback Mechanism for Flow-Matching in World-Action Models Execution
Li, Peize; Zhang, Ruimeng; Zhang, Ru; Huang, Cong; Chen, Kai; Zhang, Shanghang
ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts
Wang, Mingxin; Hu, Bin; Qian, Bin; Jiang, Kaitao; Wu, Haoning; Yan, Feng; Jing, Bowen; Hao, Ruiyang; Wang, Enyi; Niu, Kangning; Yang, Yandan; Xu, Mu; Wang, Yan; Liu, Houde; Li, Tianlun
QuantWAMs: Calibrating at the Right Granularity for World Action Models
Zhou, Jiacheng; Lv, Jinfan; Li, Ruixuan; Zhang, Longtai; Wang, Yan; Zhang, Wenqiang; Qi, Lizhe
LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments
Liu, Pei; Zheng, Nan; Zhang, Lang; Peng, Daojie; Zhang, Yanan; Kong, Feilong; Feng, Mingyue; Liu, Jiachao; Wang, Yaonong; Chen, Qifeng; Ma, Jun
DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models
Ji, Haoyuan; Fan, Lingxiang; Su, Shang; Lu, Yinqiao; Shi, Mengkai; Gao, Jun; Feng, Shuo
N₀-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation
Team, NeoteAI; Team, Fudan TEAI
GeoWorldAD: Geometry World Action Model for Autonomous Driving
Zhang, Songyan; Tian, Jinyuan; Li, Hanbing; Liu, Daqi; Chen, Hao; Huang, Wenhui; Li, Fang; Chen, Guang; Ye, Hangjun; Chen, Long; Yang, Kuiyuan; Lv, Chen
NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation
NVIDIA; Aarti Basant; Amlan Kar; Despoina Paschalidou; Fangyin Wei; Francesco Ferroni; Guillermo Garcia Cobo; Haithem Turki; Huan Ling; Jaewoo Seo; James Lucas; Jay Zhangjie Wu; Jialiang Wang; Jonathan Lorraine; Jun Gao; Kai He; Katarina Tothova; Kevin Xie; Michał Tyszkiewicz; Qi Wu; Riccardo de Lutio; Ruilong Li; Sanja Fidler; Seung Wook Kim; Tianchang Shen; Tianshi Cao; Tobias Pfaff; William Lew; Xindi Wu; Xuanchi Ren; Yifan Lu; Yuxuan Zhang; Zan Gojcic; Zian Wang
PerceptDrive: Perception Prior World-Action Modeling with Adaptive Expert Routing for End-to-End Autonomous Driving
Liu, Yushan; Lv, Tianxiong; Wang, Bohua; Fan, Hangqi; Zhao, Chenxu; Zheng, He; Zhong, Xuchang; Xie, Yifan; Zhao, Congyang; Liao, Zhihao; Luo, Leigang; Cai, Yang; Zhang, Xiao-Ping; Ding, Wenbo
WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory
Su, Haisheng; Liu, Zongdai; Jin, Xin; Dou, Haoxuan; Hu, Chengming; Li, Baorun; Liu, Zhanwang; Xu, Ruiyan; Fang, Jianjie; Zhang, Xin; Yang, Zhenjie; Yang, Xue; Gao, Chen; Yan, Junchi; Li, Yong; Wu, Wei
Test-Time Scaling for World Action Models via Zero-Shot Geometric Evaluation
Zhao, Zesen; Cho, Minkyoung; shen, Hui; Zheng, Boyuan; Gao, Kunxiao; Cao, Yulong; Mao, Z. Morley
GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch
GigaWorld Team; Angen Ye; Angyuan Ma; Boyuan Wang; Chaojun Ni; Fangzheng Ye; Guan Huang; Guo Li; Guosheng Zhao; Haodong Yan; Hengtao Li; Jiwen Lu; Kai Wang; Mingming Yu; Qitang Hu; Qiuping Deng; Songling Liu; Xiaoyu Tian; Xiaofeng Wang; Xinyu Zhou; Xiuwei Xu; Xinze Chen; Yang Wang; Yejun Zeng; Yifan Chang; Yun Ye; Zhenyu Wu; Zhanqian Wu; Zheng Zhu
AeroAct: Action-Centered World-Action Models for Language-Conditioned Quadrotor Flight
Zhang, Xinhong; Zhu, Qiyuan; Huang, Yubo; Chen, Haolin; Wang, Runqing; Mo, Yuhao; Chen, Zhongxin; Hu, Yu; Wang, Xinjiang; Sun, Jian; Wang, Gang
Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control
Hong, Jihoon; Skifstad, Julian; Dai, Qiyue; Chan, Alice; Chou, Glen
FlowWAM: Optical Flow as a Unified Action Representation for World Action Models
Chen, Yixiang; Li, Peiyan; Xu, Yuan; Ma, Qisen; Yang, Jiabing; Wang, Kai; Yang, Jianhua; An, Dong; Guan, He; Liu, Gaoteng; Si, Jianlou; Huang, Jun; Liu, Jing; Liu, Nianfeng; Huang, Yan; Wang, Liang
Towards Predictive, Aligned, and Scalable Robot Learning
Tang, Peijun; Xie, Shangjin; Huang, Baifu; Sun, Binyan; Yang, Haotian; Luo, Kuncheng; Jin, Weiqi; Fang, Shilin; Wang, Jianan
DIM-WAM: World-Action Modeling with Diverse Historical Event Memory
Wang, Kai; Gu, Zhaopeng; Chen, Yixiang; Xu, Yuan; Ma, Qisen; Yang, Jiabing; Li, Zhaowen; Huang, Yan; Wang, Liang; Su, Peng
FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space
Murray, Michael; Chen, Daphne; Bagaria, Simran; Fortier, Dean; Hellebrekers, Tess; Mullins, Galen; Gajarla, Harshavardhan; Mees, Oier; Cakmak, Maya; Kolobov, Andrey
Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio
Mishra, Utkarsh A.; Chen, Yongxin; Xu, Danfei; Liu, Yang; Chen, Xi; Mao, Jiayuan
WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time
Feng, Yusen; Han, Bingchen; Lyu, Jiangran; Liu, Kai; Zheng, Yixin; Wan, Yuxuan; Liu, Weiheng; Han, Sun; Li, Ruiqin; Zhang, Yulong; Liu, Fangfu; Shi, Xuesong; Liu, Libin; Wang, Yizhou; Zhang, Zhizheng; Wang, He
EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data
Li, Baoyu; Yin, Xinchen; Lin, Mengying; Zhang, Yixin; Xu, Danfei
WAM4D: Fast 4D World Action Model via Spatial Register Tokens
Li, Ying; Wei, Xiaobao; Cao, Jiajun; Wang, Hao; Chi, Xiaowei; Bai, Chengyu; Sun, Qianpu; Li, Jiajun; Zhang, Xiaojie; Jia, Peidong; Tang, Jian; Han, Sirui; Zhang, Shanghang
Learning 4D Geometric Priors for Inference-Efficient World Action Models
Zhang, Jianjun; Zhu, Jian; Su, Taiyi; Ma, Chong; Huang, Zitai; Xu, Yi; Wang, Hanli
UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation
Liu, Mengmeng; Zhang, Diankun; Liu, Jiuming; Cui, Jianfeng; Xie, Hongwei; Chen, Guang; Ye, Hangjun; Nex, Francesco; Cheng, Hao; Yang, Michael Ying
DSWAM: A Dual-System World Action Foundation Model for Fine-Grained Robot Manipulation
Zhu, Jian; Zhang, Jianjun; Su, Taiyi; Liu, Tianbin; Wang, Zhangyuan; Xie, Kai; Huang, Zitai; Ma, Chong; He, Youzhang; Wang, Tianjian; Wang, Hanyang; Ding, Weihao; Xu, Yi
ABot-M0.5: Unified Mobility-and-Manipulation World Action Model
Chen, Ronghan; Yang, Yandan; Tang, Zuojin; Huo, Dongjie; Lin, Tong; Wu, Haoning; Liu, Haoyun; Chen, Yuzhi; Zheng, Lulu; Yuan, Botai; Li, Tianlun; Wang, Mingxin; Qi, Dekang; Hu, Bin; Mei, Wei; Xuan, Yuze; Yang, Haolong; Zhu, Yanqing; Xu, Mu; Ma, Zhiheng; Chang, Xinyuan
HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models
Ye, Angen; Ke, Weijie; Wang, Xiaofeng; Chen, Xinze; Ni, Chaojun; Zhao, Guosheng; Wang, Boyuan; Zhu, Zheng; Xie, Junjie; Zhang, Dapeng
Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI
Team, Kairos; Wang, Fei; You, Shan; Zhang, Qiming; Huang, Tao; Fu, Zuoyi; Zheng, Zhisheng; Xi, Yunlong; Lv, Feng; Wu, Xiaoming; Liu, Zeyu; Wan, Cong; Li, Pu; Yang, Ruiqing; Li, Xiaoou; Wang, Wei; Zhu, Kangkang; Zhang, Yuwei; Fu, Shi; Zhang, Zheng; Wu, Xiaoning; Fan, Xuzeng; Tao, Dacheng; Wang, Xiaogang
VT-WAM: Visual-Tactile World Action Model for Contact-Rich Manipulation
Tian, Shuai; Zheng, Yupeng; Zheng, Yuhang; Gu, Songen; Zang, Yujie; Qin, Yuxing; Li, Weize; Li, Haoran; Ding, Wenchao; Zhao, Dongbin
Bridge-WA: Predicting Where and How the World Changes for Robotic Action
Bai, Yongjie; Wang, Hanting; Dai, Mingtong; Zhong, Qijun; Liu, Yang; Lin, Liang
Efficient Sim-to-Real Transfer of World-Action Models from Synthetic Priors
Wang, Zixing; Sivakumar, Kausik; Shang, Jinghuan; Hu, Yafei; Xie, Zhaoming; Gong, Ran; Zhang, Xiaohan; Schmeckpeper, Karl
FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation
Zhang, Lingfeng; Gong, Zeying; Hao, Xiaoshuai; Fu, Haoxiang; Zhang, Qiang; Zhou, Mingliang; Ye, Hangjun; Liang, Xiaojun; Liang, Junwei; Ding, Wenbo
Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation
Chen, Hong; Liu, Daqi; Zhang, Zehan; Wang, Haiguang; Lu, Tianhao; Yan, Longfei; Sun, Haiyang; Li, Fangzhen; Xie, Hongwei; Wang, Bing; Chen, Guang; Ye, Hangjun; Tan, Yihua
World Action Models Enable Continual Imitation Learning with Recurrent Generative Replays
Govind, Manish Kumar; Reilly, Dominick; Patel, Smit; Le, Hieu; Das, Srijan
Cosmos 3: Omnimodal World Models for Physical AI
NVIDIA; Aditi; Niket Agarwal; Arslan Ali; Jon Allen; Martin Antolini; Adeline Aubame; Alisson Azzolini; Junjie Bai; Maciej Bala; Yogesh Balaji; Josh Bapst; Aarti Basant; Mukesh Beladiya; Mohammad Qazim Bhat; Zaid Pervaiz Bhat; Dan Blick; Vanni Brighella; Han Cai; Tiffany Cai; Eric Cameracci; Jiaxin Cao; Yulong Cao; Mark Carlson; Carlos Casanova; Ting-Yun Chang; Yan Chang; Yu-Wei Chao; Prithvijit Chattopadhyay; Roshan Chaudhari; Chieh-Yun Chen; Junyu Chen; Ke Chen; Qizhi Chen; Wenkai Chen; Xiaotong Chen; Yu Chen; An-Chieh Cheng; Click Cheng; Xiu Chia; Jeana Choi; Chaeyeon Chung; Wenyan Cong; Yin Cui; Magdalena Dadela; Nalin Dadhich; Wenliang Dai; Joyjit Daw; Alperen Degirmenci; Rodrigo Vieira Del Monte; Robert Denomme; Sameer Dharur; Marco Di Lucca; Ke Ding; Wenhao Ding; Yifan Ding; Yuzhu Dong; Nicole Drumheller; Yilun Du; Aigul Dzhumamuratova; Aleksandr Efitorov; Hamid Eghbalzadeh; Naomi Eigbe; Imad El Hanafi; Hassan Eslami; Benedikt Falk; Jiaojiao Fan; Jim Fan; Amol Fasale; Sergiy Fefilatyev; Liang Feng; Francesco Ferroni; Sanja Fidler; Xiao Fu; Vikram Fugro; Prashant Gaikwad; TJ Galda; Katelyn Gao; Yihuai Gao; Wenhang Ge; Sreyan Ghosh; Arushi Goel; Vivek Goel; Akash Gokul; Rama Govindaraju; Jinwei Gu; Miguel Guerrero; Elfie Guo; Aryaman Gupta; Siddharth Gururani; Hugo Hadfield; Song Han; Ankur Handa; Zekun Hao; Mohammad Harrim; Ali Hassani; Nathan Hayes-Roth; Yufan He; Chris Helvig; Cyrus Hogg; Madison Huang; Michael Huang; Sophia Huang; Yufan Huang; Jacob Huffman; DeLesley Hutchins; Suneel Indupuru; Boris Ivanovic; Arihant Jain; Joel Jang; Ryan Ji; Yanan Jian; Dongfu Jiang; Jingyi Jin; Atharva Joshi; Nikhilesh Joshi; Pranjali Joshi; Andy Ju; Jaehun Jung; Weiwei Kang; Scott Kassekert; Jan Kautz; Ashna Khetan; Julia Kiczka; Slawek Kierat; Gwanghyun Kim; Kuno Kim; Sunny Kim; Kezhi Kong; Xin Kong; Zhifeng Kong; Tomasz Kornuta; Egor Krivov; Hui Kuang; Saurav Kumar; Chia-Wen Kuo; George Kurian; Wojciech Kutak; JF Lafleche; Himangshu Lahkar; Omar Laymoun; Jayjun Lee; Sanggil Lee; Gabriele Leone; Boyi Li; Freya Li; Jiajun Li; Jinfeng Li; Ling Li; Pengcheng Li; Shangru Li; Tingle Li; Xiaolong Li; Xuan Li; Zhaoshuo Li; Zhiqi Li; Hao Liang; Maosheng Liao; Chen-Hsuan Lin; Tsung-Yi Lin; Ming-Yu Liu; Sifei Liu; Zihan Liu; Hai Loc Lu; Xiangyu Lu; Alice Luo; Ruipu Luo; Wenjie Luo; Jiangran Lyu; Martin Ding Ma; Nic Ma; Qianli Ma; Dawid Majchrowski; Louis Marcoux; Miguel Martin; Qing Miao; Ashkan Mirzaei; Shreyas Misra; Kaichun Mo; Durra Mohsin; Hyejin Moon; Pawel Morkisz; Saeid Motiian; Kirill Motkov; Seungjun Nah; Yashraj Narang; Deepak Narayanan; Thabang Ngazimbi; Julian Ouyang; Shubham Pachori; David Page; Yatian Pang; Sehwi Park; Mahesh Patekar; Mostofa Patwary; Marco Pavone; Trung Pham; Wei Ping; Soha Pouya; Shrimai Prabhumoye; Varun Praveen; Delin Qu; Hesam Rabeti; Morteza Ramezanali; Marilyn Reeb; Xuanchi Ren; Kristen Rumley; Wojciech Rymer; Jun Saito; Yeongho Seol; John Shao; Piyush Shekdar; Tianwei Shen; Humphrey Shi; Min Shi; Stella Shi; Kevin Shih; Mohammad Shoeybi; Mateusz Sieniawski; Shuran Song; Alexander Sotelo; Amir Sotoodeh; Sunil Srinivasa; Vignesh Srinivasakumar; Bartosz Stefaniak; Rahul Heinrich Steiger; Shangkun Sun; Jiaxiang Tang; Shitao Tang; Yangyang Tang; Yue Tang; Tolou Tavakkoli; Kayley Ting; Krzysztof Tomala; Wei-Cheng Tseng; Jibin Varghese; Sergei Vasilev; Thomas Volk; Raju Wagwani; Roger Waleffe; Andrew Z. Wang; Boxiang Wang; Haoxiang Wang; Qiao Wang; Shihao Wang; Shijie Wang; Ting-Chun Wang; Yan Wang; Yu Wang; Rohit Watve; David Wehr; Fangyin Wei; Xinshuo Weng; Jay Zhangjie Wu; Kedi Wu; Hongchi Xia; Summer Xiao; Tianjun Xiao; Kevin Xie; Daguang Xu; Jiashu Xu; Mengyao Xu; Ruqing Xu; Xingqian Xu; Yao Xu; Dinghao Yang; Dong Yang; Hans Yang; Xiaodong Yang; Xuning Yang; Yichu Yang; Yurong You; Zhiding Yu; Hao Yuan; Simon Yuen; Xiaohui Zeng; Pengcuo Zeren; Cindy Zha; Haotian Zhang; Jenny Zhang; Jing Zhang; Liangkai Zhang; Paris Zhang; Shun Zhang; Xuanmeng Zhang; Zhizheng Zhang; Ann Zhao; Yilin Zhao; Yuliya Zhautouskaya; Charles Zhou; Fengzhe Zhou; Shilin Zhu; Yuke Zhu; Dima Zhylko; Artur Zolkowski
Geometric Action Model for Robot Policy Learning
Han, Jisang; Jeon, Seonghu; Jung, Jaewoo; Zurbrügg, René; An, Honggyu; Portela, Tifanny; Hutter, Marco; Pollefeys, Marc; Kim, Seungryong; Hong, Sunghwan
MV-WAM: Manifold-Aware World Action Model with Value Augmentation
Chen, Jintao; Jia, Peidong; Wuwu, Qingpo; Liu, Jiaming; Du, Mengfei; Fan, Chun-Kai; Chi, Xiaowei; Chen, Hao; Bai, Chengyu; Qian, Zezhong; Wang, Hao; Cao, Jiajun; Mi, Weishi; Ju, Xiaozhu; Tang, Jian; Zhang, Shanghang
MemoryWAM: Efficient World Action Modeling with Persistent Memory
Yang, Sizhe; Mu, Juncheng; Wei, Tianming; Lu, Chenhao; Li, Xiaofan; Xu, Linning; Xue, Zhengrong; Yuan, Zhecheng; Lin, Dahua; Pang, Jiangmiao; Xu, Huazhe
ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?
Zhang, Yuyang; Zhang, Wenyao; Qi, Zekun; Zhang, He; Lin, Haitao; Zhang, Jingbo; Mu, Yao; Yang, Xiaokang; Zeng, Wenjun; Jin, Xin
WAM-RL: World-Action Model Reinforcement Learning with Reconstruction Rewards and Online Video SFT
Qian, Zezhong; Chi, Xiaowei; Qi, Yu; Li, Haozhan; Chen, Zhi Yang; Zhang, Shanghang
Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation
Li, Jingyu; Liu, Zhe; Hu, Dongnan; Wu, Junjie; Ma, Zipei; Wu, Wenxiao; Han, Chao; Hao, Zhihui; Liu, Zhikang; Zhan, Kun; Deng, Jiankang; Zhu, Xiatian; Zhang, Li
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Chen, Jialei; Wang, Kai; Chen, Kang; Chen, Shuaihang; Gao, Feng; Tang, Wenhao; Li, Zhiyuan; Liu, Weilin; Yao, Zhuyu; Li, Boxun; Xu, Yuanbo; Yu, Chao
Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Time
Park, Jeongeun; Park, Juhan; Kim, Taekyung; Choi, Sungjoon; Han, Dongyoon; Yun, Sangdoo
MemoryVAM: Integrating Memory into Video Action Model for Robot Manipulation
Yuxin Jiang; Chang Yu; Yunuo Chen; Xiang Feng; Yin Yang; Nishank Gite; Chenfanfu Jiang
RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
Wang, Junke; Zhang, Qihang; Yang, Shuai; Luo, Yiming; Shen, Yujun; Wu, Zuxuan; Jiang, Yu-Gang; Xu, Yinghao
WAM-Nav: Asymmetric Latent World-Action Modeling for Unified Visual Navigation
Yang, Ning; Huang, Yan; Peng, Kaiwen; He, Ziheng; Wang, Kai; Miao, Cui; Lyu, Kailin; Li, Guo; Wang, Xiaofeng; Zhu, Zheng; Liu, Jing; Liu, Nianfeng
MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models
Yu, Hanyang; Lin, Haitao; Zhang, Jingbo; Zhang, Wenyao; Gu, Chenghao; Li, Heng; Tan, Ping
NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation
Azuma, Daichi; Miyanishi, Taiki; Sakamoto, Koya; Kurita, Shuhei; Zhu, Yaonan; Khrapchenkov, Petr; Kawanabe, Motoaki; Iwasawa, Yusuke; Matsuo, Yutaka
FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation
He, Haotian; Yan, Zeyu; Liu, Qipeng; Guo, Ning; Lian, Wenzhao
EWAM: An Enhanced World Action Model for Closed-Loop Online Adaptation in Embodied Intelligence
Zhou, Xin; Miao, Cong
World Pilot: Steering Vision-Language-Action Models with World-Action Priors
Lin, Zefu; Cui, Rongxu; Xu, Junjia; Jin, Xiaojuan; Li, Wenling; Fan, Lue; Zhang, Zhaoxiang
Making Foresight Actionable: Repurposing Representation Alignment in World Action Models
Qiu, Lu; Li, Yizhuo; Chen, Yi; Ge, Yuying; Ge, Yixiao; Liu, Xihui
Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination
Li, Jiajun; Guo, Tiecheng; Ye, Yifan; Zhang, Rongyu; Chi, Xiaowei; Sun, Qianpu; Li, Ying; Lou, Yunfan; Huang, Yan; Lu, Zhihe; Guo, Meng; Zhang, Shanghang
Next Forcing: Causal World Modeling with Multi-Chunk Prediction
Xu, Gangwei; Zhang, Qihang; Zhou, Jiaming; Zhu, Xing; Shen, Yujun; Yang, Xin; Xu, Yinghao
AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing
Cai, Jisong; Ling, Long; Chu, Shiwei; Liu, Zhongshan; Kang, Jiayue; Liang, Zhixuan; Xu, Wenjie; Mao, Yinan; Zhang, Weinan; Yang, Xiaokang; Ying, Ru; Zheng, Ran; Mu, Yao
MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation
Zheng, Jia; Ma, Teli; Fan, Yudong; Wang, Zifan; Yang, Shuo; Liang, Junwei
C3^3ache: Accelerating World Action Models with Cross Inference Chunk Cache
Zhao, Weisen; Nguyen, Lam; Lu, Zhicong; Shang, Yuzhang
Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation
Lou, Yunfan; Ye, Yifan; Fu, Yankai; Cen, Jun; Chi, Xiaowei; Lyu, Yaoxu; Jia, Peidong; Han, Sirui; Lu, Zhihe; Zhang, Shanghang
ImagineUAV: Aerial Vision-Language Navigation via World-Action Modeling and Kinodynamic Planning
Liu, Xuchen; Huang, Jiawei; Xia, Shihao; Liu, Bingxi; Cui, Jinqiang; Yang, Jiankun
Light-WAM: Efficient World Action Models with State-Fusion Action Decoding
Li, Ziang; Cheng, Dongzhou; Wang, Yibin; Wang, Shiyue; Xu, Xiaoyang; Weng, Lingxuan; Wang, Juan; Wang, Jiaqi
Dreaming when Necessary: Advancing World Action Models with Adaptive Multi-Modal Reasoning
Tang, Yinzhou; Xu, Jingbo; Shang, Yu; Song, Zihao; Gao, Chen; Wu, Wei; Li, Yong
World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
Yang, Yi; Liu, Zhihong; Kou, Siqi; Chen, Yiyang; Hu, Yanzhe; Zhou, Jianbo; Zhao, Boyuan; Wei, Zhijie; Xia, Xiao; Li, Xueqi; Liu, Pengfei; Deng, Zhijie
Flash-WAM: Modality-Aware Distillation for World Action Models
Akbari, Arman; Zhang, Ci; Akbari, Arash; Zhao, Lin; Chen, Yixiao; Chen, Weiwei; Zhang, Xuan; Yuan, Geng; Wang, Yanzhi
Unified Video-Action Joint Denoising for Dexterous Action and Data Generation
Wang, Dingrui; Wang, YuAn; Liu, Jinkun; Zhang, Yue; Piccinini, Mattia; Sun, Yu; Betz, Johannes
GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
Ma, Fulong; Peng, Daojie; Yue, Wenjun; Cao, Jiahang; Wang, Bintao; Zhang, Qiang; Ma, Jun
WALL-WM: Carving World Action Modeling at the Event Joints
Shalfun Li; Victor Yao; Charles Yang; Truth Qu; Regis Cheng; Ryan Yu; Howard Lu; Newton Von; Vincent Chen; Yohann Tang; Maeve Zhang; Ellie Ma; Gody Li; Sage Yang; Lorien Shu; J. W. Gao; Ethan Chen; Colin Ye; Yu Sun; Elise Mon; PS Zhang; Neo Li; Lily Li; James Wang; Ping Yang; Chris Pan; Lucy Liang; Hang Su; Roy Gan; Hao Wang; Qian Wang
DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
Shi, Chen; Xu, Jinrui; Shi, Shaoshuai; Sheng, Kehua; Zhang, Bo; Jiang, Li
Point Tracking Improves World Action Models
Guan, Jiarui; Zhao, Wenshuai; Pei, Yue; Chen, Ziliang; Solin, Arno; Kannala, Juho
WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation
Zhao, Baining; Xu, Jiacheng; Feng, Weicheng; Zhang, Xin; Wang, Zhaolu; Wang, Haoyang; Ji, Shilong; Wang, Ziyou; Fang, Jianjie; Zheng, Zhiheng; Zhang, Weichen; Shang, Yu; Wu, Wei; Gao, Chen; Chen, Xinlei; Li, Yong
The DAWN of World-Action Interactive Models
Lu, Hongbo; Yao, Liang; He, Chenghao; Wang, Haoyu; Gu, Xiang; Li, Xianfei; Liao, Wenlong; He, Tao; Peng, Pai
HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models
Feng, Qiuxuan; Yu, Jiale; Liu, Jiaming; Jia, Yueru; Wu, Zhuangzhe; Chen, Hao; Qian, Zezhong; Gu, Shuo; Jia, Peng; Ma, Siwei; Zhang, Shanghang
When to Trust Imagination: Adaptive Action Execution for World Action Models
Wang, Rui; Zhang, Yue; Lin, Jiehong; Luo, Kuncheng; Wang, Jianan; Wang, Zhongrui; Qi, Xiaojuan
NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
Huang, Wen; Sun, Haoran; Guo, Yongjian; Ma, Yunxuan; Li, Haoran; Long, Jing; Mo, Zhouying; Guan, Zhong; Guo, Yucheng; Di, Shuai; Xiong, Junwu
Learning Visual Feature-Based World Models via Residual Latent Action
Zhang, Xinyu; Xu, Zhengtong; Tao, Yutian; Wang, Yeping; She, Yu; Boularias, Abdeslam
OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation
Liu, Yushan; Sun, Peibo; Li, Shoujie; Xie, Yifan; Zhang, Lingfeng; Chao, Xintao; Dong, Shiyuan; Chen, Fang; Zhang, Xiao-Ping; Ding, Wenbo
CKT-WAM: Parameter-Efficient Context Knowledge Transfer Between World Action Models
Jiang, Yuhua; Guo, Yijun; Yang, Hongbing; Lei, Guojun; Chen, Nuo; Zhang, Yinuo; Yan, Shaoqiang; Lin, Bo; Gao, Feifei; Qi, Biqing
Being-H0.7: A Latent World-Action Model from Egocentric Videos
BeingBeyond Team (Hao Luo; Wanpeng Zhang; Yicheng Feng; Sipeng Zheng; Haiweng Xu; Chaoyi Xu; Ziheng Xi; Yuhui Fu; Zongqing Lu)
Motubrain: An Advanced World Action Model for Robot Control
Motubrain Team
Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising
Jun Guo; Qiwei Li; Peiyan Li; Zilong Chen; Nan Sun; Yifei Su; Heyun Wang; Yuan Zhang; Xinghang Li; Huaping Liu
Privileged Foresight Distillation: Zero-Cost Future Correction for World Action Models
Pengcheng Fang; Hongli Chen; Xiaohao Cai
π_0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities
Physical Intelligence; Bo Ai; Ali Amin; Raichelle Aniceto; Ashwin Balakrishna; Greg Balke; Kevin Black; George Bokinsky; Shihao Cao; Thomas Charbonnier; Vedant Choudhary; Foster Collins; Ken Conley; Grace Connors; James Darpinian; Karan Dhabalia; Maitrayee Dhaka; Jared DiCarlo; Danny Driess; Michael Equi; Adnan Esmail; Yunhao Fang; Chelsea Finn; Catherine Glossop; Thomas Godden; Ivan Goryachev; Lachlan Groom; Haroun Habeeb; Hunter Hancock; Karol Hausman; Gashon Hussein; Victor Hwang; Brian Ichter; Connor Jacobsen; Szymon Jakubczak; Rowan Jen; Tim Jones; Gregg Kammerer; Ben Katz; Liyiming Ke; Mairbek Khadikov; Chandra Kuchi; Marinda Lamb; Devin LeBlanc; Brendon LeCount; Sergey Levine; Xinyu Li; Adrian Li-Bell; Vladislav Lialin; Zhonglin Liang; Wallace Lim; Yao Lu; Enyu Luo; Vishnu Mano; Nandan Marwaha; Aikys Mongush; Liam Murphy; Suraj Nair; Tyler Patterson; Karl Pertsch; Allen Z. Ren; Gavin Schelske; Charvi Sharma; Baifeng Shi; Lucy Xiaoyang Shi; Laura Smith; Jost Tobias Springenberg; Kyle Stachowicz; Will Stoeckle; Jiaming Tang; Jimmy Tanner; Shalom Tekeste; Marcel Torne; Kyle Vedder; Quan Vuong; Anna Walling; Haohuan Wang; Jason Wang; XuDong Wang; Chris Whalen; Samuel Whitmore; Blake Williams; Charles Xu; Sukwon Yoo; Lili Yu; Wuming Zhang; Zhuoyang Zhang; Ury Zhilinsky
DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks
Yueci Deng; Guiliang Liu; Kui Jia
AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps
Liaoyuan Fan; Zetian Xu; Chen Cao; Wenyao Zhang; Mingqi Yuan; Jiayu Chen
VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis
Xiaolei Lang; Yang Wang; Yukun Zhou; Chaojun Ni; Kerui Li; Jiagang Zhu; Tianze Liu; Jiajun Lv; Xingxing Zuo; Yun Ye; Guan Huang; Xiaofeng Wang; Zheng Zhu
Learning Vision-Language-Action World Models for Autonomous Driving
Guoqing Wang; Pin Tang; Xiangxuan Ren; Guodongfang Zhao; Bailan Feng; Chao Ma
LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving
Hao Shao; Letian Wang; Yang Zhou; Yuxuan Hu; Zhuofan Zong; Steven L. Waslander; Wei Zhan; Hongsheng Li
Action Images: End-to-End Policy Learning via Multiview Video Generation
Haoyu Zhen; Zixian Gao; Qiao Sun; Yilin Zhao; Yuncong Yang; Yilun Du; Pengsheng Guo; Tsun-Hsuan Wang; Yi-Ling Qiao; Chuang Gan
DriveVA: Video Action Models are Zero-Shot Drivers
Mengmeng Liu; Diankun Zhang; Jiuming Liu; Jianfeng Cui; Hongwei Xie; Guang Chen; Hangjun Ye; Michael Ying Yang; Francesco Nex; Hao Cheng
SpatialVAM:Spatial-Aware Multi-View Video Diffusion as a Data-Efficient Robot Policy
Peiyan Li; Yixiang Chen; Yuan Xu; Jiabing Yang; Xiangnan Wu; Jun Guo; Nan Sun; Long Qian; Xinghang Li; Xin Xiao; Jing Liu; Nianfeng Liu; Tao Kong; Yan Huang; Liang Wang; Tieniu Tan
DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
Yang Zhou; Xiaofeng Wang; Hao Shao; Letian Wang; Guosheng Zhao; Jiangnan Shao; Jiagang Zhu; Tingdong Yu; Zheng Zhu; Guan Huang; Steven L. Waslander
Enhancing Policy Learning with World-Action Model
Yuci Han; Alper Yilmaz
Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving
Qiqi Liu; Huan Xu; Jingyu Li; Bin Sun; Zhihui Hao; Dangen She; Xiatian Zhu; Li Zhang
Vega: Learning to Drive with Natural Language Instructions
Sicheng Zuo; Yuxuan Li; Wenzhao Zheng; Zheng Zhu; Jie Zhou; Jiwen Lu
DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving
Pengxuan Yang; Yupeng Zheng; Deheng Qian; Zebin Xing; Qichao Zhang; Linbo Wang; Yichen Zhang; Shaoyu Guo; Zhongpu Xia; Qiang Chen; Junyu Han; Lingyun Xu; Yifeng Pan; Dongbin Zhao
Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
Linbo Wang; Yupeng Zheng; Qiang Chen; Shiwei Li; Yichen Zhang; Zebin Xing; Qichao Zhang; Xiang Li; Deheng Qian; Pengxuan Yang; Yihang Dong; Ce Hao; Xiaoqing Ye; Junyu han; Yifeng Pan; Dongbin Zhao
VTAM: Video-Tactile-Action Models for Complex Physical Interaction Beyond VLAs
Haoran Yuan; Weigang Yi; Zhenyu Zhang; Wendi Chen; Yuchen Mo; Jiashi Yin; Xinzhuo Li; Xiangyu Zeng; Chuan Wen; Cewu Lu; Katherine Driggs-Campbell; Ismini Lourentzou
VAMPO: Policy Optimization for Improving Visual Dynamics in Video Action Models
Zirui Ge; Pengxiang Ding; Baohua Yin; Qishen Wang; Zhiyong Xie; Yemin Wang; Jinbo Wang; Hengtao Li; Runze Suo; Wenxuan Song; Han Zhao; Shangke Lyu; Zhaoxin Fan; Haoang Li; Ran Cheng; Cheng Chi; Huibin Ge; Yaozhi Luo; Donglin Wang
EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards
Ruixiang Wang; Qingming Liu; Yueci Deng; Guiliang Liu; Zhen Liu; Kui Jia
GigaWorld-Policy: An Efficient Action-Centered World–Action Model
Angen Ye; Boyuan Wang; Chaojun Ni; Guan Huang; Guosheng Zhao; Hao Li; Hengtao Li; Jie Li; Jindi Lv; Jingyu Liu; Min Cao; Peng Li; Qiuping Deng; Wenjun Mei; Xiaofeng Wang; Xinze Chen; Xinyu Zhou; Yang Wang; Yifan Chang; Yifan Li; Yukun Zhou; Yun Ye; Zhichao Liu; Zheng Zhu
DreamPlan: Efficient Reinforcement Fine-Tuning of Vision-Language Planners via Video World Models
Emily Yue-Ting Jia; Weiduo Yuan; Tianheng Shi; Vitor Guizilini; Jiageng Mao; Yue Wang
Fast-WAM: Do World Action Models Need Test-time Future Imagination?
Tianyuan Yuan; Zibin Dong; Yicheng Liu; Hang Zhao
S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresight
Haodong Yan; Zhide Zhong; Jiaguan Zhu; Junjie He; Weilin Yuan; Wenxuan Song; Xin Gong; Yingjie Cai; Guanyi Zhao; Xu Yan; Bingbing Liu; Ying-Cong Chen; Haoang Li
Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation
Xingtai Gui; Meijie Zhang; Tianyi Yan; Wencheng Han; Jiahao Gong; Feiyang Tan; Cheng-zhong Xu; Jianbing Shen
LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels
Lucas Maes; Quentin Le Lidec; Damien Scieur; Yann LeCun; Randall Balestriero
DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control
Teli Ma; Jia Zheng; Zifan Wang; Chunli Jiang; Andy Cui; Junwei Liang; Shuo Yang
World2Act: Latent Action Post-Training from World Model Dynamics
An Dinh Vuong; Tuan Van Vo; Abdullah Sohail; Haoran Ding; Liang Ma; Xiaodan Liang; Anqing Duan; Ivan Laptev; Ian Reid
SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving
Zihan You; Hongwei Liu; Chenxu Dang; Zhe Wang; Sining Ang; Aoqi Wang; Yan Wang
Self-Correcting VLA: Online Action Refinement via Sparse World Imagination
Chenyv Liu; Wentao Tan; Lei Zhu; Fengling Li; Jingjing Li; Guoli Yang; Heng Tao Shen
Learning to unfold cloth: Scaling up world models to deformable object manipulation
Jack Rome; Stephen James; Subramanian Ramamoorthy
WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL
Zhennan Jiang; Shangqing Zhou; Yutong Jiang; Zefang Huang; Mingjie Wei; Yuhui Chen; Tianxing Zhou; Zhen Guo; Hao Lin; Quanlu Zhang; Yu Wang; Haoran Li; Chao Yu; Dongbin Zhao
LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion
Jiangran Lyu; Kai Liu; Xuheng Zhang; Haoran Liao; Yusen Feng; Wenxuan Zhu; Tingrui Shen; Jiayi Chen; Jiazhao Zhang; Yifei Dong; Wenbo Cui; Senmao Qi; Shuo Wang; Yixin Zheng; Mi Yan; Xuesong Shi; Haoran Li; Dongbin Zhao; Ming-Yu Liu; Zhizheng Zhang; Li Yi; Yizhou Wang; He Wang
GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learning
GigaBrain Team; Boyuan Wang; Bohan Li; Chaojun Ni; Guan Huang; Guosheng Zhao; Hao Li; Jie Li; Jindi Lv; Jingyu Liu; Lv Feng; Mingming Yu; Peng Li; Qiuping Deng; Tianze Liu; Xinyu Zhou; Xinze Chen; Xiaofeng Wang; Yang Wang; Yifan Li; Yifei Nie; Yilong Li; Yukun Zhou; Yun Ye; Zhichao Liu; Zheng Zhu
RISE: Self-Improving Robot Policy with Compositional World Model
Jiazhi Yang; Kunyang Lin; Jinwei Li; Wencong Zhang; Tianwei Lin; Longyan Wu; Zhizhong Su; Hao Zhao; Ya-Qin Zhang; Li Chen; Ping Luo; Xiangyu Yue; Hongyang Li
VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model
Jingwen Sun; Wenyao Zhang; Zekun Qi; Shaojie Ren; Zezhi Liu; Hanxin Zhu; Guangzhong Sun; Xin Jin; Zhibo Chen
World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy
Xiaokang Liu; Zechen Bai; Hai Ci; Kevin Yuchen Ma; Mike Zheng Shou
Visuo-Tactile World Models
Carolina Higuera; Sergio Arnaud; Byron Boots; Mustafa Mukadam; Francois Robert Hogan; Franziska Meier
Causal World Modeling for Robot Control
Lin Li; Qihang Zhang; Yiming Luo; Shuai Yang; Ruilin Wang; Fei Han; Mingrui Yu; Zelin Gao; Nan Xue; Xing Zhu; Yujun Shen; Yinghao Xu
Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning
Moo Jin Kim; Yihuai Gao; Tsung-Yi Lin; Yen-Chen Lin; Yunhao Ge; Grace Lam; Percy Liang; Shuran Song; Ming-Yu Liu; Chelsea Finn; Jinwei Gu
UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving
Zhexiao Xiong; Xin Ye; Burhan Yaman; Sheng Cheng; Yiren Lu; Jingru Luo; Nathan Jacobs; Liu Ren
PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation
Wenlong Huang; Yu-Wei Chao; Arsalan Mousavian; Ming-Yu Liu; Dieter Fox; Kaichun Mo; Li Fei-Fei
Motus: A Unified Latent Action World Model
Hongzhe Bi; Hengkai Tan; Shenghao Xie; Zeyuan Wang; Shuhe Huang; Haitian Liu; Ruowen Zhao; Yao Feng; Chendong Xiang; Yinze Rong; Hongyan Zhao; Hanyu Liu; Zhizhong Su; Lei Ma; Hang Su; Jun Zhu
Dyna-2: A 1-million-hour scaling law for world-action models
Dyna Robotics
MonoDream: Monocular Vision-Language Navigation with Panoramic Dreaming
Shuo Wang; Yongcai Wang; Zhaoxin Fan; Yucheng Wang; Maiyue Chen; Kaihui Wang; Zhizhong Su; Wanting Li; Xudong Cai; Yeying Jin; Deying Li
DriveLaW: Unifying Planning and Video Generation in a Latent Driving World
Tianze Xia; Yongkang Li; Lijun Zhou; Jingfeng Yao; Kaixin Xiong; Haiyang Sun; Bing Wang; Kun Ma; Guang Chen; Hangjun Ye; Wenyu Liu; Xinggang Wang
DreamDojo: A Real-Time Robot World Model from Large-Scale Human Videos
Shenyuan Gao; William Liang; Kaiyuan Zheng; Ayaan Malik; Seonghyeon Ye; Sihyun Yu; Wei-Cheng Tseng; Yuzhu Dong; Kaichun Mo; Chen-Hsuan Lin; Qianli Ma; Seungjun Nah; Loic Magne; Jiannan Xiang; Yuqi Xie; Ruijie Zheng; Dantong Niu; You Liang Tan; K. R. Zentner; George Kurian; Suneel Indupuru; Pooya Jannaty; Jinwei Gu; Jun Zhang; Jitendra Malik; Pieter Abbeel; Ming-Yu Liu; Yuke Zhu; Joel Jang; Linxi "Jim" Fan
Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation
Zijian Song; Qichang Li; Sihan Qin; Yuhao Chen; Tianshui Chen; Liang Lin; Guangrun Wang
VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model
Yanjiang Guo; Tony Lee; Lucy Xiaoyang Shi; Jianyu Chen; Percy Liang; Chelsea Finn
ViPRA: Video Prediction for Robot Actions
Sandeep Routray; Hengkai Pan; Unnat Jain; Shikhar Bahl; Deepak Pathak
DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT
Xiaotao Hu; Mingkai Jia; Xiaoyang Guo; Qian Zhang; Xiao-xiao Long; Wei Yin
DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
Feiyang jia; Lin Liu; Ziying Song; Caiyan Jia; Hangjun Ye; Xiaoshuai Hao; Long Chen
DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving
Yingyan Li; Shuyao Shang; Weisong Liu; Bing Zhan; Haochen Wang; Yuqi Wang; Yuntao Chen; Xiaoman Wang; Yasong An; Chufeng Tang; LU HOU; Lue Fan; Zhaoxiang Zhang
WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
Fangqi Zhu; Zhengyang Yan; Zicong Hong; Quanxin Shou; Xiao Ma; Song Guo
DynVLA: Learning World Dynamics for Action Reasoning in Autonomous Driving
Shuyao Shang; Bing Zhan; Yunfei Yan; Yuqi Wang; Yingyan Li; Yasong An; Xiaoman Wang; Jierui Liu; Lu Hou; Lue Fan; Zhaoxiang Zhang; Tieniu Tan
Learning Latent Action World Models In The Wild
Quentin Garrido; Tushar Nagarajan; Basile Terver; Nicolas Ballas; Yann LeCun; Michael Rabbat
Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots
Chenhao Li; Andreas Krause; Marco Hutter
Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models
Zhilong Zhang; Haoxiang Ren; Yihao Sun; Yifei Sheng; Haonan Wang; Haoxin Lin; Zhichao Wu; Pierre-Luc Bacon; Yang Yu
World Action Models are Zero-shot Policies
Seonghyeon Ye; Yunhao Ge; Kaiyuan Zheng; Shenyuan Gao; Sihyun Yu; George Kurian; Suneel Indupuru; You Liang Tan; Chuning Zhu; Jiannan Xiang; Ayaan Malik; Kyungmin Lee; William Liang; Nadun Ranawaka; Jiasheng Gu; Yinzhen Xu; Guanzhi Wang; Fengyuan Hu; Avnish Narayan; Johan Bjorck; Jing Wang; Gwanghyun Kim; Dantong Niu; Ruijie Zheng; Yuqi Xie; Jimmy Wu; Qi Wang; Ryan Julian; Danfei Xu; Yilun Du; Yevgen Chebotar; Scott Reed; Jan Kautz; Yuke Zhu; Linxi "Jim" Fan; Joel Jang
Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow
Karthik Dharmarajan; Wenlong Huang; Jiajun Wu; Li Fei-Fei; Ruohan Zhang
Act2Goal: From World Model To General Goal-conditioned Policy
Pengfei Zhou; Liliang Chen; Shengcong Chen; Di Chen; Wenzhi Zhao; Rongjun Jin; Guanghui Ren; Jianlan Luo
CoVAR: Co-generation of Video and Action for Robotic Manipulation via Multi-Modal Diffusion
Liudi Yang; Yang Bai; George Eskandar; Fengyi Shen; Mohammad Altillawi; Dong Chen; Ziyuan Liu; Abhinav Valada
mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs
Jonas Pai; Liam Achenbach; Victoriano Montesinos; Benedek Forrai; Oier Mees; Elvis Nava
World Models for Learning Dexterous Hand-Object Interactions from Human Videos
Raktim Gautam Goswami; Amir Bar; David Fan; Tsung-Yen Yang; Gaoyue Zhou; Prashanth Krishnamurthy; Michael Rabbat; Farshad Khorrami; Yann LeCun
VideoVLA: Video Generators Can Be Generalizable Robot Manipulators
Yichao Shen; Fangyun Wei; Zhiying Du; Yaobo Liang; Yan Lu; Jiaolong Yang; Nanning Zheng; Baining Guo
MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving
Bin Sun; Yaoguang Cao; Yan Wang; Rui Wang; Jiachen Shang; Xiejie Feng; Jiayi Lu; Jia Shi; Shichun Yang; Xiaoyu Yan; Ziying Song
Learning Massively Multitask World Models for Continuous Control
Nicklas Hansen; Hao Su; Xiaolong Wang
RynnVLA-002: A Unified Vision-Language-Action and World Model
Jun Cen; Siteng Huang; Yuqian Yuan; Kehan Li; Hangjie Yuan; Chaohui Yu; Bohan Hou; Yuming Jiang; Jiayan Guo; Xin Li; Hao Luo; Fan Wang; Deli Zhao; Hao Chen
Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model
John Won; Kyungmin Lee; Huiwon Jang; Dongyoung Kim; Jinwoo Shin
Ego-Vision World Model for Humanoid Contact Planning
Hang Liu; Yuman Gao; Sangli Teng; Yufeng Chi; Yakun Sophia Shao; Zhongyu Li; Maani Ghaffari; Koushil Sreenath
MoWM: Mixture-of-World-Models for Embodied Planning via Latent-to-Pixel Feature Modulation
Yangcheng Yu; Xin Jin; Yu Shang; Xin Zhang; Haisheng Su; Wei Wu; Yong Li
World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation
Zhennan Jiang; Kai Liu; Yuxin Qin; Shuai Tian; Yupeng Zheng; Mingcai Zhou; Chao Yu; Haoran Li; Dongbin Zhao
Latent Action Pretraining Through World Modeling
Bahey Tharwat; Yara Nasser; Ali Abouzeid; Ian Reid
Spatial Policy: Guiding Visuomotor Robotic Manipulation with Spatial-Aware Modeling and Reasoning
Yijun Liu; Yuwei Liu; Yuan Meng; Jieheng Zhang; Yuwei Zhou; Ye Li; Jiacheng Jiang; Kangye Ji; Shijia Ge; Zhi Wang; Wenwu Zhu
IRL-VLA: Training an Vision-Language-Action Policy via Reward World Model
Anqing Jiang; Yu Gao; Yiru Wang; Zhigang Sun; Shuo Wang; Yuwen Heng; Hao Sun; Shichen Tang; Lijuan Zhu; Jinhao Chai; Jijun Wang; Zichong Gu; Hao Jiang; Li Sun
Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation
Yue Liao; Pengfei Zhou; Siyuan Huang; Donglin Yang; Shengcong Chen; Yuxin Jiang; Yue Hu; Jingbin Cai; Si Liu; Jianlan Luo; Liliang Chen; Shuicheng Yan; Maoqing Yao; Guanghui Ren
Video Generators are Robot Policies
Junbang Liang; Pavel Tokmakov; Ruoshi Liu; Sruthi Sudhakar; Paarth Shah; Rares Ambrus; Carl Vondrick
Vidar: Embodied Video Diffusion Model for Generalist Manipulation
Yao Feng; Hengkai Tan; Xinyi Mao; Chendong Xiang; Guodong Liu; Shuhe Huang; Hang Su; Jun Zhu
DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge
Wenyao Zhang; Hongsi Liu; Zekun Qi; Yunnan Wang; Xinqiang Yu; Jiazhao Zhang; Runpei Dong; Jiawei He; Fan Lu; He Wang; Zhizheng Zhang; Li Yi; Wenjun Zeng; Xin Jin
WorldVLA: Towards Autoregressive Action World Model
Jun Cen; Chaohui Yu; Hangjie Yuan; Yuming Jiang; Siteng Huang; Jiayan Guo; Xin Li; Yibing Song; Hao Luo; Fan Wang; Deli Zhao; Hao Chen
AMPLIFY: Actionless Motion Priors for Robot Learning from Videos
Jeremy A. Collins; Loránd Cheng; Kunal Aneja; Albert Wilcox; Benjamin Joffe; Animesh Garg
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Mido Assran; Adrien Bardes; David Fan; Quentin Garrido; Russell Howes; Mojtaba; Komeili; Matthew Muckley; Ammar Rizvi; Claire Roberts; Koustuv Sinha; Artem Zholus; Sergio Arnaud; Abha Gejji; Ada Martin; Francois Robert Hogan; Daniel Dugas; Piotr Bojanowski; Vasil Khalidov; Patrick Labatut; Francisco Massa; Marc Szafraniec; Kapil Krishnakumar; Yong Li; Xiaodong Ma; Sarath Chandar; Franziska Meier; Yann LeCun; Michael Rabbat; Nicolas Ballas
3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model
Hongyan Zhi; Peihao Chen; Siyuan Zhou; Yubo Dong; Quanxi Wu; Lei Han; Mingkui Tan
Towards a Generalizable Bimanual Foundation Policy via Flow-based Video Prediction
Chenyou Fan; Fangzheng Yan; Chenjia Bai; Jiepeng Wang; Chi Zhang; Zhen Wang; Xuelong Li
ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
Xiaodong Wang; Peixi Peng
FLARE: Robot Learning with Implicit World Modeling
Ruijie Zheng; Jing Wang; Scott Reed; Johan Bjorck; Yu Fang; Fengyuan Hu; Joel Jang; Kaushil Kundalia; Zongyu Lin; Loic Magne; Avnish Narayan; You Liang Tan; Guanzhi Wang; Qi Wang; Jiannan Xiang; Yinzhen Xu; Seonghyeon Ye; Jan Kautz; Furong Huang; Yuke Zhu; Linxi Fan
Learned Perceptive Forward Dynamics Model for Safe and Platform-aware Robotic Navigation
Pascal Roth; Jonas Frey; Cesar Cadena; Marco Hutter
PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation
Wenxuan Li; Hang Zhao; Zhiyuan Yu; Yu Du; Qin Zou; Ruizhen Hu; Kai Xu
Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets
Chuning Zhu; Raymond Yu; Siyuan Feng; Benjamin Burchfiel; Paarth Shah; Abhishek Gupta
CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models
Qingqing Zhao; Yao Lu; Moo Jin Kim; Zipeng Fu; Zhuoyang Zhang; Yecheng Wu; Zhaoshuo Li; Qianli Ma; Song Han; Chelsea Finn; Ankur Handa; Ming-Yu Liu; Donglai Xiang; Gordon Wetzstein; Tsung-Yi Lin
LUMOS: Language-Conditioned Imitation Learning with World Models
Iman Nematollahi; Branton DeMoss; Akshay L Chandra; Nick Hawes; Wolfram Burgard; Ingmar Posner
Unified Video Action Model
Shuang Li; Yihuai Gao; Dorsa Sadigh; Shuran Song
VaViM and VaVAM: Autonomous Driving through Video Generative Modeling
Florent Bartoccioni; Elias Ramzi; Victor Besnier; Shashanka Venkataramanan; Tuan-Hung Vu; Yihong Xu; Loick Chambon; Spyros Gidaris; Serkan Odabas; David Hurych; Renaud Marlet; Alexandre Boulch; Mickael Chen; Éloi Zablocki; Andrei Bursuc; Eduardo Valle; Matthieu Cord
VILP: Imitation Learning with Latent Video Planning
Zhengtong Xu, Qiang Qiu, Yu She
VideoWorld: Exploring Knowledge Learning from Unlabeled Videos
Zhongwei Ren; Yunchao Wei; Xun Guo; Yao Zhao; Bingyi Kang; Jiashi Feng; Xiaojie Jin
RoboHorizon: An LLM-Assisted Multi-View World Model for Long-Horizon Robotic Manipulation
Zixuan Chen; Jing Huo; Yangtao Chen; Yang Gao
EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation
Siyuan Huang; Liliang Chen; Pengfei Zhou; Shengcong Chen; Yue Liao; Zhengkai Jiang; Yue Hu; Peng Gao; Hongsheng Li; Maoqing Yao; Guanghui Ren
Learning Robot Manipulation from Audio World Models
Fan Zhang; Michael Gienger
RLVR-World: Training World Models with Reinforcement Learning
Jialong Wu; Shaofeng Yin; Ningya Feng; Mingsheng Long
ParticleFormer: A 3D Point Cloud World Model for Multi-Object, Multi-Material Robotic Manipulation
Suning Huang; Qianzhong Chen; Xiaohan Zhang; Jiankai Sun; Mac Schwager
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
Yuntao Chen; Yuqi Wang; Zhaoxiang Zhang
World Model-based Perception for Visual Legged Locomotion
Hang Lai; Jiahang Cao; Jiafeng Xu; Hongtao Wu; Yunfeng Lin; Tao Kong; Yong Yu; Weinan Zhang
Epona: Autoregressive Diffusion World Model for Autonomous Driving
Kaiwen Zhang; Zhenyu Tang; Xiaotao Hu; Xingang Pan; Xiaoyang Guo; Yuan Liu; Jingwei Huang; Li Yuan; Qian Zhang; Xiao-Xiao Long; Xun Cao; Wei Yin
Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
Chenhao Li; Andreas Krause; Marco Hutter
World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model
Yupeng Zheng; Pengxuan Yang; Zebin Xing; Qichao Zhang; Yuhang Zheng; Yinfeng Gao; Pengfei Li; Teng Zhang; Zhongpu Xia; Peng Jia; XianPeng Lang; Dongbin Zhao
Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation
Yang Tian; Sizhe Yang; Jia Zeng; Ping Wang; Dahua Lin; Hao Dong; Jiangmiao Pang
A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation
Rongtao Xu; Jian Zhang; Minghao Guo; Youpeng Wen; Haoting Yang; Min Lin; Jianzheng Huang; Zhe Li; Kaidong Zhang; Liqiong Wang; Yuxuan Kuang; Meng Cao; Feng Zheng; Xiaodan Liang
Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations
Yucheng Hu; Yanjiang Guo; Pengchao Wang; Xiaoyu Chen; Yen-Jen Wang; Jianke Zhang; Koushil Sreenath; Chaochao Lu; Jianyu Chen
DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation
Jiangran Lyu; Ziming Li; Xuesong Shi; Chaoyi Xu; Yizhou Wang; He Wang
Navigation World Models
Amir Bar; Gaoyue Zhou; Danny Tran; Trevor Darrell; Yann LeCun
Cross-Embodiment Dexterous Manipulation through World Model Learning
Zihao He; Bo Ai; Yulin Liu; Weikang Wan; Henrik I Christensen; Hao Su
GWM: Towards Scalable Gaussian World Models for Robotic Manipulation
Guanxing Lu; Baoxiong Jia; Puhao Li; Yixin Chen; Ziwei Wang; Yansong Tang; Siyuan Huang
X-MOBILITY: End-To-End Generalizable Navigation via World Modeling
Wei Liu; Huihua Zhao; Chenran Li; Joydeep Biswas; Billy Okal; Pulkit Goyal; Yan Chang; Soha Pouya
Driving in the Occupancy World: Vision-Centric 4D Occupancy Forecasting and Planning via World Models for Autonomous Driving
Yu Yang; Jianbiao Mei; Yukai Ma; Siliang Du; Wenqing Chen; Yijie Qian; Yuxiang Feng; Yong Liu
ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model
Tengbo Yu; Guanxing Lu; Zaijia Yang; Haoyuan Deng; Season Si Chen; Jiwen Lu; Wenbo Ding; Guoqiang Hu; Yansong Tang; Ziwei Wang
Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling
Han Qi; Haocheng Yin; Aris Zhu; Yilun Du; Heng Yang
Enhancing End-to-End Autonomous Driving with Latent World Model
Yingyan Li; Lue Fan; Jiawei He; Yuqi Wang; Yuntao Chen; Zhaoxiang Zhang; Tieniu Tan
Doe-1: Closed-Loop Autonomous Driving with Large World Model
Wenzhao Zheng; Zetian Xia; Yuanhui Huang; Sicheng Zuo; Jie Zhou; Jiwen Lu
GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation
Chi-Lam Cheang; Guangzeng Chen; Ya Jing; Tao Kong; Hang Li; Yifeng Li; Yuxiao Liu; Hongtao Wu; Jiafeng Xu; Yichu Yang; Hanbo Zhang; Minzhao Zhu
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation
Homanga Bharadhwaj; Debidatta Dwibedi; Abhinav Gupta; Shubham Tulsiani; Carl Doersch; Ted Xiao; Dhruv Shah; Fei Xia; Dorsa Sadigh; Sean Kirmani
OccLLaMA: An Occupancy-Language-Action Generative World Model for Autonomous Driving
Julong Wei; Shanshuai Yuan; Pengfei Li; Qingda Hu; Zhongxue Gan; Wenchao Ding
This&That: Language-Gesture Controlled Video Generation for Robot Planning
Boyang Wang; Nikhil Sridhar; Chao Feng; Mark Van der Merwe; Adam Fishman; Nima Fazeli; Jeong Joon Park
Dreamitate: Real-World Visuomotor Policy Learning via Video Generation
Junbang Liang; Ruoshi Liu; Ege Ozguroglu; Sruthi Sudhakar; Achal Dave; Pavel Tokmakov; Shuran Song; Carl Vondrick
RoboDreamer: Learning Compositional World Models for Robot Imagination
Siyuan Zhou; Yilun Du; Jiaben Chen; Yandong Li; Dit-Yan Yeung; Chuang Gan
Prediction with Action: Visual Policy Learning via Joint Denoising Process
Yanjiang Guo; Yucheng Hu; Jianke Zhang; Yen-Jen Wang; Xiaoyu Chen; Chaochao Lu; Jianyu Chen
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation
Hongtao Wu; Ya Jing; Chilam Cheang; Guangzeng Chen; Jiafeng Xu; Xinghang Li; Minghuan Liu; Hang Li; Tao Kong
Generalized Predictive Model for Autonomous Driving
Jiazhi Yang; Shenyuan Gao; Yihang Qiu; Li Chen; Tianyu Li; Bo Dai; Kashyap Chitta; Penghao Wu; Jia Zeng; Ping Luo; Jun Zhang; Andreas Geiger; Yu Qiao; Hongyang Li
VidMan: Exploiting Implicit Dynamics from Video Diffusion Model for Effective Robot Manipulation
Youpeng Wen; Junfan Lin; Yi Zhu; Jianhua Han; Hang Xu; Shen Zhao; Xiaodan Liang
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
Wenzhao Zheng; Weiliang Chen; Yuanhui Huang; Borui Zhang; Yueqi Duan; Jiwen Lu
DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
Xiaofeng Wang; Zheng Zhu; Guan Huang; Xinze Chen; Jiagang Zhu; Jiwen Lu
ADriver-I: A General World Model for Autonomous Driving
Fan Jia; Weixin Mao; Yingfei Liu; Yucheng Zhao; Yuqing Wen; Chi Zhang; Xiangyu Zhang; Tiancai Wang
Learning Universal Policies via Text-Guided Video Generation
Yilun Du; Mengjiao Yang; Bo Dai; Hanjun Dai; Ofir Nachum; Joshua B. Tenenbaum; Dale Schuurmans; Pieter Abbeel
DREAMWALKER: Mental Planning for Continuous Vision-Language Navigation
Hanqing Wang; Wei Liang; Luc Van Gool; Wenguan Wang
Pathdreamer: A World Model for Indoor Navigation
Jing Yu Koh; Honglak Lee; Yinfei Yang; Jason Baldridge; Peter Anderson
Datasets
Resources for training and evaluation
Subcategories 32
Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation
Haoyu Wang; Songchun Zhang; Haoran Li; Haoyang Huang; Zeyue Xue; Nan Duan
HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alone
AI, Simple; :; Wei, Yuteng; Ma, Jinming; Wang, Jiawei; Zhou, Weitao; Zuo, Yushen; Rui, Ke; Li, Minglei; Zhang, Jinhao; Pan, Zhikang; Wang, Xiang; Jia, Haoran; Du, Huan; Zeng, Zicheng; Ma, Jun; Qin, Guiyu; Zhang, Di; Li, Xiaofei
How to Instruct Your Robot: Dense Language Annotations Power Robot Policy Learning
Kim, Bosung; Wang, Ruiyi; Acuna, David; Jung, Jaehun; Trevithick, Alexander; Cui, Brandon; Choi, Yejin; Ammanabrolu, Prithviraj
ActiveGlasses: Learning Manipulation with Active Vision from Ego-centric Human Demonstration
Yanwen Zou; Chenyang Shi; Wenye Yu; Han Xue; Jun Lv; Ye Pan; Chuan Wen; Cewu Lu
HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videos
Yinhuai Wang; Qihan Zhao; Yuen Fui Lau; Runyi Yu; Hok Wai Tsui; Qifeng Chen; Jingbo Wang; Jiangmiao Pang; Ping Tan
RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation
Tianxing Chen; Zanxin Chen; Baijun Chen; Zijian Cai; Yibin Liu; Zixuan Li; Qiwei Liang; Xianliang Lin; Yiheng Ge; Zhenyu Gu; Weiliang Deng; Yubin Guo; Tian Nian; Xuanbing Xie; Qiangyu Chen; Kailun Su; Tianling Xu; Guodong Liu; Mengkang Hu; Huan-ang Gao; Kaixuan Wang; Zhixuan Liang; Yusen Qin; Xiaokang Yang; Ping Luo; Yao Mu
EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video
Ryan Hoque; Peide Huang; David J. Yoon; Mouli sivapurapu; Jian Zhang
GigaWorld-0: World Models as Data Engine to Empower Embodied AI
GigaWorld Team; Angen Ye; Boyuan Wang; Chaojun Ni; Guan Huang; Guosheng Zhao; Haoyun Li; Jiagang Zhu; Kerui Li; Mengyuan Xu; Qiuping Deng; Siting Wang; Wenkang Qin; Xinze Chen; Xiaofeng Wang; Yankai Wang; Yu Cao; Yifan Chang; Yuan Xu; Yun Ye; Yang Wang; Yukun Zhou; Zhengyuan Zhang; Zhehao Dong; Zheng Zhu
MimicDreamer: Aligning Human and Robot Demonstrations for Scalable VLA Training
Haoyun Li; Ivan Zhang; Runqi Ouyang; Xiaofeng Wang; Zheng Zhu; Zhiqin Yang; Zhentao Zhang; Boyuan Wang; Chaojun Ni; Wenkang Qin; Xinze Chen; Yun Ye; Guan Huang; Zhenbo Song; Xingang Wang
RoboTransfer: Controllable Geometry-Consistent Video Diffusion for Manipulation Policy Transfer
Liu Liu; Xiaofeng Wang; Guosheng Zhao; Keyu Li; Wenkang Qin; Jiagang Zhu; Jiaxiong Qiu; Zheng Zhu; Guan Huang; Zhizhong Su
DreamGen: Unlocking Generalization in Robot Learning through Video World Models
Joel Jang; Seonghyeon Ye; Zongyu Lin; Jiannan Xiang; Johan Bjorck; Yu Fang; Fengyuan Hu; Spencer Huang; Kaushil Kundalia; Yen-Chen Lin; Lo\"ıc Magne; Ajay Mandlekar; Avnish Narayan; You Liang Tan; Guanzhi Wang; Jing Wang; Qi Wang; Yinzhen Xu; Xiaohui Zeng; Kaiyuan Zheng; Ruijie Zheng; Ming-Yu Liu; Luke Zettlemoyer; Dieter Fox; Jan Kautz; Scott Reed; Yuke Zhu; Linxi Fan
AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems
AgiBot-World-Contributors; Qingwen Bu; Jisong Cai; Li Chen; Xiuqi Cui; Yan Ding; Siyuan Feng; Shenyuan Gao; Xindong He; Xuan Hu; Xu Huang; Shu Jiang; Yuxin Jiang; Cheng Jing; Hongyang Li; Jialu Li; Chiming Liu; Yi Liu; Yuxiang Lu; Jianlan Luo; Ping Luo; Yao Mu; Yuehan Niu; Yixuan Pan; Jiangmiao Pang; Yu Qiao; Guanghui Ren; Cheng Ruan; Jiaqi Shan; Yongjian Shen; Chengshi Shi; Mingkang Shi; Modi Shi; Chonghao Sima; Jianheng Song; Huijie Wang; Wenhao Wang; Dafeng Wei; Chengen Xie; Guo Xu; Junchi Yan; Cunbiao Yang; Lei Yang; Shukai Yang; Maoqing Yao; Jia Zeng; Chi Zhang; Qinglin Zhang; Bin Zhao; Chengyue Zhao; Jiaqi Zhao; Jianchao Zhu
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots
Cheng Chi; Zhenjia Xu; Chuer Pan; Eric Cousineau; Benjamin Burchfiel; Siyuan Feng; Russ Tedrake; Shuran Song
DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset
Alexander Khazatsky; Karl Pertsch; Suraj Nair; Ashwin Balakrishna; Sudeep Dasari; Siddharth Karamcheti; Soroush Nasiriany; Mohan Kumar Srirama; Lawrence Yunliang Chen; Kirsty Ellis; Peter David Fagan; Joey Hejna; Masha Itkina; Marion Lepert; Yecheng Jason Ma; Patrick Tree Miller; Jimmy Wu; Suneel Belkhale; Shivin Dass; Huy Ha; Arhan Jain; Abraham Lee; Youngwoon Lee; Marius Memmel; Sungjae Park; Ilija Radosavovic; Kaiyuan Wang; Albert Zhan; Kevin Black; Cheng Chi; Kyle Beltran Hatch; Shan Lin; Jingpei Lu; Jean Mercat; Abdul Rehman; Pannag R. Sanketi; Archit Sharma; Cody Simpson; Quan Vuong; Homer Rich Walke; Blake Wulfe; Ted Xiao; Jonathan Heewon Yang; Arefeh Yavary; Tony Z. Zhao; Christopher Agia; Rohan Baijal; Mateo Guaman Castro; Daphne Chen; Qiuyu Chen; Trinity Chung; Jaimyn Drake; Ethan Paul Foster; Jensen Gao; David Antonio Herrera; Minho Heo; Kyle Hsu; Jiaheng Hu; Donovon Jackson; Charlotte Le; Yunshuang Li; Roy Lin; Zehan Ma; Abhiram Maddukuri; Suvir Mirchandani; Daniel Morton; Tony Nguyen; Abigail O'Neill; Rosario Scalise; Derick Seale; Victor Son; Stephen Tian; Emi Tran; Andrew E. Wang; Yilin Wu; Annie Xie; Jingyun Yang; Patrick Yin; Yunchu Zhang; Osbert Bastani; Glen Berseth; Jeannette Bohg; Ken Goldberg; Abhinav Gupta; Abhishek Gupta; Dinesh Jayaraman; Joseph J. Lim; Jitendra Malik; Roberto Mart\'ın-Mart\'ın; Subramanian Ramamoorthy; Dorsa Sadigh; Shuran Song; Jiajun Wu; Michael C. Yip; Yuke Zhu; Thomas Kollar; Sergey Levine; Chelsea Finn
CoPeD-Advancing Multi-Robot Collaborative Perception: A Comprehensive Dataset in Real-World Environments
Yang Zhou; Long Quang; Carlos Nieto-Granda; Giuseppe Loianno
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Open X-Embodiment Collaboration; Abby O'Neill; Abdul Rehman; Abhinav Gupta; Abhiram Maddukuri; Abhishek Gupta; Abhishek Padalkar; Abraham Lee; Acorn Pooley; Agrim Gupta; Ajay Mandlekar; Ajinkya Jain; Albert Tung; Alex Bewley; Alex Herzog; Alex Irpan; Alexander Khazatsky; Anant Rai; Anchit Gupta; Andrew Wang; Andrey Kolobov; Anikait Singh; Animesh Garg; Aniruddha Kembhavi; Annie Xie; Anthony Brohan; Antonin Raffin; Archit Sharma; Arefeh Yavary; Arhan Jain; Ashwin Balakrishna; Ayzaan Wahid; Ben Burgess-Limerick; Beomjoon Kim; Bernhard Schölkopf; Blake Wulfe; Brian Ichter; Cewu Lu; Charles Xu; Charlotte Le; Chelsea Finn; Chen Wang; Chenfeng Xu; Cheng Chi; Chenguang Huang; Christine Chan; Christopher Agia; Chuer Pan; Chuyuan Fu; Coline Devin; Danfei Xu; Daniel Morton; Danny Driess; Daphne Chen; Deepak Pathak; Dhruv Shah; Dieter Büchler; Dinesh Jayaraman; Dmitry Kalashnikov; Dorsa Sadigh; Edward Johns; Ethan Foster; Fangchen Liu; Federico Ceola; Fei Xia; Feiyu Zhao; Felipe Vieira Frujeri; Freek Stulp; Gaoyue Zhou; Gaurav S. Sukhatme; Gautam Salhotra; Ge Yan; Gilbert Feng; Giulio Schiavi; Glen Berseth; Gregory Kahn; Guangwen Yang; Guanzhi Wang; Hao Su; Hao-Shu Fang; Haochen Shi; Henghui Bao; Heni Ben Amor; Henrik I Christensen; Hiroki Furuta; Homanga Bharadhwaj; Homer Walke; Hongjie Fang; Huy Ha; Igor Mordatch; Ilija Radosavovic; Isabel Leal; Jacky Liang; Jad Abou-Chakra; Jaehyung Kim; Jaimyn Drake; Jan Peters; Jan Schneider; Jasmine Hsu; Jay Vakil; Jeannette Bohg; Jeffrey Bingham; Jeffrey Wu; Jensen Gao; Jiaheng Hu; Jiajun Wu; Jialin Wu; Jiankai Sun; Jianlan Luo; Jiayuan Gu; Jie Tan; Jihoon Oh; Jimmy Wu; Jingpei Lu; Jingyun Yang; Jitendra Malik; João Silvério; Joey Hejna; Jonathan Booher; Jonathan Tompson; Jonathan Yang; Jordi Salvador; Joseph J. Lim; Junhyek Han; Kaiyuan Wang; Kanishka Rao; Karl Pertsch; Karol Hausman; Keegan Go; Keerthana Gopalakrishnan; Ken Goldberg; Kendra Byrne; Kenneth Oslund; Kento Kawaharazuka; Kevin Black; Kevin Lin; Kevin Zhang; Kiana Ehsani; Kiran Lekkala; Kirsty Ellis; Krishan Rana; Krishnan Srinivasan; Kuan Fang; Kunal Pratap Singh; Kuo-Hao Zeng; Kyle Hatch; Kyle Hsu; Laurent Itti; Lawrence Yunliang Chen; Lerrel Pinto; Li Fei-Fei; Liam Tan; Linxi "Jim" Fan; Lionel Ott; Lisa Lee; Luca Weihs; Magnum Chen; Marion Lepert; Marius Memmel; Masayoshi Tomizuka; Masha Itkina; Mateo Guaman Castro; Max Spero; Maximilian Du; Michael Ahn; Michael C. Yip; Mingtong Zhang; Mingyu Ding; Minho Heo; Mohan Kumar Srirama; Mohit Sharma; Moo Jin Kim; Muhammad Zubair Irshad; Naoaki Kanazawa; Nicklas Hansen; Nicolas Heess; Nikhil J Joshi; Niko Suenderhauf; Ning Liu; Norman Di Palo; Nur Muhammad Mahi Shafiullah; Oier Mees; Oliver Kroemer; Osbert Bastani; Pannag R Sanketi; Patrick "Tree" Miller; Patrick Yin; Paul Wohlhart; Peng Xu; Peter David Fagan; Peter Mitrano; Pierre Sermanet; Pieter Abbeel; Priya Sundaresan; Qiuyu Chen; Quan Vuong; Rafael Rafailov; Ran Tian; Ria Doshi; Roberto Martín-Martín; Rohan Baijal; Rosario Scalise; Rose Hendrix; Roy Lin; Runjia Qian; Ruohan Zhang; Russell Mendonca; Rutav Shah; Ryan Hoque; Ryan Julian; Samuel Bustamante; Sean Kirmani; Sergey Levine; Shan Lin; Sherry Moore; Shikhar Bahl; Shivin Dass; Shubham Sonawani; Shubham Tulsiani; Shuran Song; Sichun Xu; Siddhant Haldar; Siddharth Karamcheti; Simeon Adebola; Simon Guist; Soroush Nasiriany; Stefan Schaal; Stefan Welker; Stephen Tian; Subramanian Ramamoorthy; Sudeep Dasari; Suneel Belkhale; Sungjae Park; Suraj Nair; Suvir Mirchandani; Takayuki Osa; Tanmay Gupta; Tatsuya Harada; Tatsuya Matsushima; Ted Xiao; Thomas Kollar; Tianhe Yu; Tianli Ding; Todor Davchev; Tony Z. Zhao; Travis Armstrong; Trevor Darrell; Trinity Chung; Vidhi Jain; Vikash Kumar; Vincent Vanhoucke; Vitor Guizilini; Wei Zhan; Wenxuan Zhou; Wolfram Burgard; Xi Chen; Xiangyu Chen; Xiaolong Wang; Xinghao Zhu; Xinyang Geng; Xiyuan Liu; Xu Liangwei; Xuanlin Li; Yansong Pang; Yao Lu; Yecheng Jason Ma; Yejin Kim; Yevgen Chebotar; Yifan Zhou; Yifeng Zhu; Yilin Wu; Ying Xu; Yixuan Wang; Yonatan Bisk; Yongqiang Dou; Yoonyoung Cho; Youngwoon Lee; Yuchen Cui; Yue Cao; Yueh-Hua Wu; Yujin Tang; Yuke Zhu; Yunchu Zhang; Yunfan Jiang; Yunshuang Li; Yunzhu Li; Yusuke Iwasawa; Yutaka Matsuo; Zehan Ma; Zhuo Xu; Zichen Jeff Cui; Zichen Zhang; Zipeng Fu; Zipeng Lin
DexGraspNet: A Large-Scale Robotic Dexterous Grasp Dataset for General Objects Based on Simulation
Ruicheng Wang; Jialiang Zhang; Jiayi Chen; Yinzhen Xu; Puhao Li; Tengyu Liu; He Wang
BridgeData V2: A Dataset for Robot Learning at Scale
Homer Walke; Kevin Black; Abraham Lee; Moo Jin Kim; Max Du; Chongyi Zheng; Tony Zhao; Philippe Hansen-Estruch; Quan Vuong; Andre He; Vivek Myers; Kuan Fang; Chelsea Finn; Sergey Levine
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations
Ajay Mandlekar; Soroush Nasiriany; Bowen Wen; Iretiayo Akinola; Yashraj Narang; Linxi Fan; Yuke Zhu; Dieter Fox
Scaling Data Generation in Vision-and-Language Navigation
Zun Wang; Jialu Li; Yicong Hong; Yi Wang; Qi Wu; Mohit Bansal; Stephen Gould; Hao Tan; Yu Qiao
OpenScene: The Largest Up-to-Date 3D Occupancy Prediction Benchmark in Autonomous Driving
OpenScene Contributors
Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous driving
Mina Alibeigi; William Ljungbergh; Adam Tonderski; Georg Hess; Adam Lilja; Carl Lindström; Daria Motorniuk; Junsheng Fu; Jenny Widahl; Christoffer Petersson
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Kristen Grauman; Andrew Westbury; Eugene Byrne; Zachary Chavis; Antonino Furnari; Rohit Girdhar; Jackson Hamburger; Hao Jiang; Miao Liu; Xingyu Liu; Miguel Martin; Tushar Nagarajan; Ilija Radosavovic; Santhosh Kumar Ramakrishnan; Fiona Ryan; Jayant Sharma; Michael Wray; Mengmeng Xu; Eric Zhongcong Xu; Chen Zhao; Siddhant Bansal; Dhruv Batra; Vincent Cartillier; Sean Crane; Tien Do; Morrie Doulaty; Akshay Erapalli; Christoph Feichtenhofer; Adriano Fragomeni; Qichen Fu; Abrham Gebreselasie; Cristina González; James Hillis; Xuhua Huang; Yifei Huang; Wenqi Jia; Weslie Khoo; Jáchym Kolár; Satwik Kottur; Anurag Kumar; Federico Landini; Chao Li; Yanghao Li; Zhenqiang Li; Karttikeya Mangalam; Raghava Modhugu; Jonathan Munro; Tullie Murrell; Takumi Nishiyasu; Will Price; Paola Ruiz Puentes; Merey Ramazanova; Leda Sari; Kiran K. Somasundaram; Audrey Southerland; Yusuke Sugano; Ruijie Tao; Minh Vo; Yuchen Wang; Xindi Wu; Takuma Yagi; Ziwei Zhao; Yunyi Zhu; Pablo Arbeláez; David Crandall; Dima Damen; Giovanni Maria Farinella; Christian Fuegen; Bernard Ghanem; Vamsi Krishna Ithapu; C. V. Jawahar; Hanbyul Joo; Kris Kitani; Haizhou Li; Richard A. Newcombe; Aude Oliva; Hyun Soo Park; James M. Rehg; Yoichi Sato; Jianbo Shi; Mike Zheng Shou; Antonio Torralba; Lorenzo Torresani; Mingfei Yan; Jitendra Malik
Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation
Haresh Karnan; Anirudh Nair; Xuesu Xiao; Garrett Warnell; Sören Pirk; Alexander Toshev; Justin W. Hart; Joydeep Biswas; Peter Stone
TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models
Samuel Triest; Matthew Sivaprakasam; Sean J. Wang; Wenshan Wang; Aaron M. Johnson; Sebastian A. Scherer
Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding
Alexander Ku; Peter Anderson; Roma Patel; Eugene Ie; Jason Baldridge
nuScenes: A Multimodal Dataset for Autonomous Driving
Holger Caesar; Varun Bankiti; Alex H. Lang; Sourabh Vora; Venice Erin Liong; Qiang Xu; Anush Krishnan; Yu Pan; Giancarlo Baldan; Oscar Beijbom
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
Pei Sun; Henrik Kretzschmar; Xerxes Dotiwalla; Aurelien Chouard; Vijaysai Patnaik; Paul Tsui; James Guo; Yin Zhou; Yuning Chai; Benjamin Caine; Vijay Vasudevan; Wei Han; Jiquan Ngiam; Hang Zhao; Aleksei Timofeev; Scott Ettinger; Maxim Krivokon; Amy Gao; Aditya Joshi; Yu Zhang; Jonathon Shlens; Zhifeng Chen; Dragomir Anguelov
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
Fisher Yu; Haofeng Chen; Xin Wang; Wenqi Xian; Yingying Chen; Fangchen Liu; Vashisht Madhavan; Trevor Darrell
RoboNet: Large-Scale Multi-Robot Learning
Sudeep Dasari; Frederik Ebert; Stephen Tian; Suraj Nair; Bernadette Bucher; Karl Schmeckpeper; Siddharth Singh; Sergey Levine; Chelsea Finn
Argoverse: 3D Tracking and Forecasting With Rich Maps
Ming-Fang Chang; John Lambert; Patsorn Sangkloy; Jagjeet Singh; Slawomir Bak; Andrew Hartnett; De Wang; Peter Carr; Simon Lucey; Deva Ramanan; James Hays
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Antoine Miech; Dimitri Zhukov; Jean-Baptiste Alayrac; Makarand Tapaswi; Ivan Laptev; Josef Sivic
Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments
Peter Anderson; Qi Wu; Damien Teney; Jake Bruce; Mark Johnson; Niko Sünderhauf; Ian D. Reid; Stephen Gould; Anton van den Hengel
The Kinetics Human Action Video Dataset
Will Kay; Joao Carreira; Karen Simonyan; Brian Zhang; Chloe Hillier; Sudheendra Vijayanarasimhan; Fabio Viola; Tim Green; Trevor Back; Paul Natsev; Mustafa Suleyman; Andrew Zisserman
Matterport3D: Learning from RGB-D Data in Indoor Environments
Angel X. Chang; Angela Dai; Thomas A. Funkhouser; Maciej Halber; Matthias Nießner; Manolis Savva; Shuran Song; Andy Zeng; Yinda Zhang
The "Something Something" Video Database for Learning and Evaluating Visual Common Sense
Raghav Goyal; Samira Ebrahimi Kahou; Vincent Michalski; Joanna Materzynska; Susanne Westphal; Heuna Kim; Valentin Haenel; Ingo Fruend; Peter Yianilos; Moritz Mueller-Freitag; Florian Hoppe; Christian Thurau; Ingo Bax; Roland Memisevic
A benchmark for the evaluation of RGB-D SLAM systems
Jrgen Sturm; Nikolas Engelhard; Felix Endres; Wolfram Burgard; Daniel Cremers
Are we ready for autonomous driving? The KITTI vision benchmark suite
Andreas Geiger; Philip Lenz; Raquel Urtasun
Evaluation metrics
Measures and evaluation protocols
Subcategories 12
Beyond Task Success: Stage-Wise Reliability of World Model Planning under Sensing Degradation
Geonmyeong Lee; Byoung-Tak Zhang
Paired Exact-Reset Evaluation of a Prediction-Derived Medium-to-Full World-Model Cascade
Pastor, Malo de
Beyond Task Success: Behavioral and Representational Diagnostics for WAM and VLA
Mai, Hung; Zhu, Bin; Do, Tuan
Is the Future Compatible? Diagnosing Dynamic Consistency in World Action Models
Ruan, Bo-Kai; Hsiao, Teng-Fang; Lo, Ling; Shuai, Hong-Han
Deep Reinforcement Learning at the Edge of the Statistical Precipice
Rishabh Agarwal; Max Schwarzer; Pablo Samuel Castro; Aaron Courville; Marc Bellemare
General Evaluation for Instruction Conditioned Navigation using Dynamic Time Warping
Gabriel Ilharco; Vihan Jain; Alexander Ku; Eugene Ie; Jason Baldridge
Towards Accurate Generative Models of Video: A New Metric & Challenges
Thomas Unterthiner; Sjoerd van Steenkiste; Karol Kurach; Raphael Marinier; Marcin Michalski; Sylvain Gelly
On Evaluation of Embodied Navigation Agents
Peter Anderson; Angel Chang; Devendra Singh Chaplot; Alexey Dosovitskiy; Saurabh Gupta; Vladlen Koltun; Jana Kosecka; Jitendra Malik; Roozbeh Mottaghi; Manolis Savva; Amir R. Zamir
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Richard Zhang; Phillip Isola; Alexei A. Efros; Eli Shechtman; Oliver Wang
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Martin Heusel; Hubert Ramsauer; Thomas Unterthiner; Bernhard Nessler; Sepp Hochreiter
CIDEr: Consensus-Based Image Description Evaluation
Ramakrishna Vedantam; C. Lawrence Zitnick; Devi Parikh
METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments
Satanjeev Banerjee; Alon Lavie
Image Quality Assessment: From Error Visibility to Structural Similarity
Zhou Wang; Alan C. Bovik; Hamid R. Sheikh; Eero P. Simoncelli
ROUGE: A Package for Automatic Evaluation of Summaries
Chin-Yew Lin
Bleu: a Method for Automatic Evaluation of Machine Translation
Kishore Papineni; Salim Roukos; Todd Ward; Wei-Jing Zhu
Benchmarks & simulators
Tasks, environments, and simulators
Subcategories 27
Programmable World Model
Zheng-Hui Huang; Guixu Lin; Jiacheng Lin; Yi-Chuan Huang; Ruihan Yu; Muyao Niu; Siqi Yang; Yu-Lun Liu; Yung-Yu Chuang; Kaipeng Zhang; Zhixiang Wang
FolDeX: A Physical-World Benchmark for Long-Horizon Robotic Manipulation of Deformable Objects
Chenhuan Liu; Yi Xu; Feng Wu; Hanyang Wang; Wenxiao Kuai; Weihao Ding; Shan Wang; Yang Liu; Shuyong Gao; Wenqiang Zhang
TourPhysics: Bringing Physics to World Models for Exploration and Manipulation from a Single Image
Xin Zhang; Yabo Chen; Zixuan Duan; Haibin Huang; Chi Zhang; Feng Xu; Xuelong Li
RoboSynChallenge: Mastering Real-World Dexterity via Generalizing Synthesized Manipulation Skills
Zhao, Runyi; Wu, Ruixin; Li, Chengkun; Zhang, Hongrui; Li, Ang; Jin, Ruixing; Deng, Yueci; Guo, Yingying; Ding, Lihe; Dong, Shaocong; Xue, Tianfan; Gao, Yanjun; Luo, Yudong; Poupart, Pascal; Wu, Simo; Jia, Kui; Zheng, Wei-shi; Liu, Guiliang
EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPE
Yan, Zexuan; Wu, Yuzhou; Ma, Yue; He, Zonghang; Yin, Kaibo; Tu, Xiaobing; Wang, Yinggui; Ren, Jinkui; Zhang, Xiantao; Wang, Shijian; Liu, Jinghong; Zhang, Linfeng
RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies
Tianxing Chen; Yue Chen; Zixuan Li; Junyuan Tang; Kailun Su; Haoran Lu; Weijie Wan; Baijun Chen; Songling Liu; Haowen Yan; Honghao Su; Zhiyang Dou; Kaixuan Wang; Dandan Zhang; Yunze Liu; Yan Qin; Qiwei Liang; Qiwei Wu; Zijian Lin; Wenwei Lin; Yuran Wang; Minghua He; Tianshu Wu; Ruihai Wu; Jingquan Zhou; Kai-Chong Lei; Haibao Yu; Yuanfeng Ji; Weiyang Jin; Guanyu Lin; Xiaofan Li; Qi Xiong; Renjing Xu; Zhongyu Li; Wenhao Chai; Enze Xie; Ziwei Wang; Yao Mu; Hao Dong; Wojciech Matusik; Mingyu Ding; Wenbo Ding; Ping Luo; Masayoshi Tomizuka
PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation
Yuhang Huang; Xuan Lv; Junyan Xu; Zhiyuan Yu; Jiazhao Zhang; Ruizhen Hu; Wancheng Feng; Shilong Zou; Hewen Xiao; Ziqiao Zhou; Kaiyun Huang; Zhiyu Peng; Juzhan Xu; Hang Zhao; Chenyang Zhu; Renjiao Yi; Yifei Huang; Douhui Wu; Yan Zhang; Kexu Cheng; Chunhe Song; Yunzhi Xue; Xiuhong Zhang; Leitao Guo; Yunji Chen; Bin Wu; Haibin Yu; Kai Xu
Diffusion Transformer World-Action Model for AV Scene Prediction
Sharifullin, Ruslan; Jiang, Benjamin; Chew, Kai Xi
SIMPLE: Simulation-Based Policy Learning and Evaluation for Humanoid Loco-manipulation
Wei, Songlin; Ni, Zhenhao; Liu, Jie; Zhao, Zhenyu; Ye, Junjie; Jing, Hongyi; Xia, Junkai; Liu, Xiawei; Leong, Michael; Heng, Liang; Huang, Di; Wang, Yue
EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields
Yang, Zhaoyang; Jin, Yurun; Qi, Lizhe; Huang, Cong; Chen, Kai
RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation
Feng Jiang; Yang Chen; Kyle Xu; Yuchen Liu; Haifeng Wang; Zhenhao Shen; Jasper Lu; Shengze Huang; Yuanfei Wang; Chen Xie; Ruihai Wu
INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
InSpatio Team; Donghui Shen; Guofeng Zhang; Haomin Liu; Haoyu Ji; Hujun Bao; Hongjia Zhai; Jialin Liu; Jing Guo; Nan Wang; Siji Pan; Weihong Pan; Weijian Xie; Xianbin Liu; Xiaojun Xiang; Xiaoyu Zhang; Xinyu Chen; Yifu Wang; Yipeng Chen; Zhenzhou Fan; Zhewen Le; Zhichao Ye; Ziqiang Zhao
JailWAM: Jailbreaking World Action Models in Robot Control
Hanqing Liu; Songping Wang; Jiahuan Long; Jiacheng Hou; Jialiang Sun; Chao Li; Yang Yang; Wei Peng; Xu Liu; Tingsong Jiang; Yao Mu; Wen Yao
ManipArena: A Controlled Benchmark for Diagnosing Generalization in Real-Robot Manipulation
Yu Sun; Meng Cao; Yang Ping; Kaidong Zhang; Qingxuan Chen; Rongtao Xu; Liangwang Ruan; Xuecheng Chen; Dongxiu Liu; Yunxiao Yan; Zunnan Xu; Runze Xu; Charles Yang; Peilun Zhang; Xiaofan Li; Ruyi Gan; Liang Ma; Yuehao Yin; Jincheng Yu; Lufang Chen; Yuxin Liang; Peng Zhai; Hao Wang; Ivan Laptev; Ian Reid; Qian Wang; Xiaodan Liang
Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learning
Jai Bardhan; Patrik Drozdik; Josef Sivic; Vladimir Petrik
Toward Physically Consistent Driving Video World Models under Challenging Trajectories
Jiawei Zhou; Zhenxin Zhu; Lingyi Du; Linye Lyu; Lijun Zhou; Zhanqian Wu; Hongcheng Luo; Zhuotao Tian; Bing Wang; Guang Chen; Hangjun Ye; Haiyang Sun; Yu Li
Do World Action Models Generalize Better than VLAs? A Robustness Study
Zhanguang Zhang; Zhiyuan Li; Behnam Rahmati; Rui Heng Yang; Yintao Ma; Amir Rasouli; Sajjad Pakdamansavoji; Yangzheng Wu; Lingfeng Zhang; Tongtong Cao; Feng Wen; Xinyu Wang; Xingyue Quan; Yingxue Zhang
X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving
Chaoda Zheng; Sean Li; Jinhao Deng; Zhennan Wang; Shijia Chen; Liqiang Xiao; Ziheng Chi; Hongbin Lin; Kangjie Chen; Boyang Wang; Yu Zhang; Xianming Liu
WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models
Yu Shang; Zhuohang Li; Yiding Ma; Weikang Su; Xin Jin; Ziyou Wang; Lei Jin; Xin Zhang; Yinzhou Tang; Haisheng Su; Chen Gao; Wei Wu; Xihui Liu; Dhruv Shah; Zhaoxiang Zhang; Zhibo Chen; Jun Zhu; Yonghong Tian; Tat-Seng Chua; Wenwu Zhu; Yong Li
Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Test
Chun-Kai Fan; Xiaowei Chi; Xiaozhu Ju; Hao Li; Yong Bao; Yu-Kai Wang; Lizhang Chen; Zhiyuan Jiang; Kuangzhi Ge; Ying Li; Weishi Mi; Qingpo Wuwu; Peidong Jia; Yulin Luo; Kevin Zhang; Zhiyuan Qin; Yong Dai; Sirui Han; Yike Guo; Shanghang Zhang; Jian Tang
Ctrl-World: A Controllable Generative World Model for Robot Manipulation
Yanjiang Guo; Lucy Xiaoyang Shi; Jianyu Chen; Chelsea Finn
WorldGym: World Model as An Environment for Policy Evaluation
Julian Quevedo; Ansh Kumar Sharma; Yixiang Sun; Varad Suryavanshi; Percy Liang; Sherry Yang
DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
Yang Zhou; Hao Shao; Letian Wang; Zhuofan Zong; Hongsheng Li; Steven L. Waslander
Interactive World Simulator for Robot Policy Training and Evaluation
Yixuan Wang; Rhythm Syed; Fangyu Wu; Mengchao Zhang; Aykut Onol; Jose Barreiros; Hooshang Nayyeri; Tony Dear; Huan Zhang; Yunzhu Li
WorldLens: Full-Spectrum Evaluations of Driving World Models in Real World
Ao Liang; Lingdong Kong; Tianyi Yan; Hongsi Liu; Wesley Yang; Ziqi Huang; Wei Yin; Jialong Zuo; Yixuan Hu; Dekai Zhu; Dongyue Lu; Youquan Liu; Guangfeng Jiang; Linfeng Li; Xiangtai Li; Long Zhuo; Lai Xing Ng; Benoit R. Cottereau; Changxin Gao; Liang Pan; Wei Tsang Ooi; Ziwei Liu
MIND-V: Hierarchical World Model for Long-Horizon Robotic Manipulation with RL-based Physical Alignment
Ruicheng Zhang; Mingyang Zhang; Jun Zhou; Xiaofan Liu; Zunnan Xu; Zhizhou Zhong; Puxin Yan; Haocheng Luo; Xiu Li
Scalable Policy Evaluation with Video World Models
Wei-Cheng Tseng; Jinwei Gu; Qinsheng Zhang; Hanzi Mao; Ming-Yu Liu; Florian Shkurti; Lin Yen-Chen
PAN: A World Model for General, Interactable, and Long-Horizon World Simulation
PAN Team; Jiannan Xiang; Yi Gu; Zihan Liu; Zeyu Feng; Qiyue Gao; Yiyan Hu; Benhao Huang; Guangyi Liu; Yichi Yang; Kun Zhou; Davit Abrahamyan; Arif Ahmad; Ganesh Bannur; Junrong Chen; Kimi Chen; Mingkai Deng; Ruobing Han; Xinqi Huang; Haoqiang Kang; Zheqi Liu; Enze Ma; Hector Ren; Yashowardhan Shinde; Rohan Shingre; Ramsundar Tanikella; Kaiming Tao; Dequan Yang; Xinle Yu; Cong Zeng; Binglin Zhou; Zhengzhong Liu; Zhiting Hu; Eric P. Xing
LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models
Senyu Fei; Siyin Wang; Junhao Shi; Zihao Dai; Jikun Cai; Pengfang Qian; Li Ji; Xinzhe He; Shiduo Zhang; Zhaoye Fei; Jinlan Fu; Jingjing Gong; Xipeng Qiu
Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report
Riccardo Mereu; Aidan Scannell; Yuxin Hou; Yi Zhao; Aditya Jitta; Antonio Dominguez; Luigi Acerbi; Amos Storkey; Paul Chang
LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE
Yu Shang; Lei Jin; Yiding Ma; Xin Zhang; Chen Gao; Wei Wu; Yong Li
MoVieDrive: Urban Scene Synthesis with Multi-Modal Multi-View Video Diffusion Transformer
Guile Wu; David Huang; Dongfeng Bai; Bingbing Liu
RoboScape: Physics-informed Embodied World Model
Yu Shang; Xin Zhang; Yinzhou Tang; Lei Jin; Chen Gao; Wei Wu; Yong Li
WorldEval: World Model as Real-World Robot Policies Evaluator
Yaxuan Li; Yichen Zhu; Junjie Wen; Chaomin Shen; Yi Xu
EnerVerse-AC: Envisioning Embodied Environments with Action Condition
Yuxin Jiang; Shengcong Chen; Siyuan Huang; Liliang Chen; Pengfei Zhou; Yue Liao; Xindong He; Chiming Liu; Hongsheng Li; Maoqing Yao; Guanghui Ren
EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
Yue Hu; Siyuan Huang; Yue Liao; Shengcong Chen; Pengfei Zhou; Liliang Chen; Maoqing Yao; Guanghui Ren
GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving
Lloyd Russell; Anthony Hu; Lorenzo Bertoni; George Fedoseev; Jamie Shotton; Elahe Arani; Gianluca Corrado
DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation
Guosheng Zhao; Xiaofeng Wang; Zheng Zhu; Xinze Chen; Guan Huang; Xiaoyi Bao; Xingang Wang
Evaluating Real-World Robot Manipulation Policies in Simulation
Xuanlin Li; Kyle Hsu; Jiayuan Gu; Oier Mees; Karl Pertsch; Homer Rich Walke; Chuyuan Fu; Ishikaa Lunawat; Isabel Sieh; Sean Kirmani; Sergey Levine; Jiajun Wu; Chelsea Finn; Hao Su; Quan Vuong; Ted Xiao
RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins
Yao Mu; Tianxing Chen; Zanxin Chen; Shijia Peng; Zhiqian Lan; Zeyu Gao; Zhixuan Liang; Qiaojun Yu; Yude Zou; Mingkun Xu; Lunkai Lin; Zhiqiang Xie; Mingyu Ding; Ping Luo
FMB: A functional manipulation benchmark for generalizable robotic learning
Jianlan Luo; Charles Xu; Fangchen Liu; Liam Tan; Zipeng Lin; Jeffrey Wu; Pieter Abbeel; Sergey Levine
Pseudo-Simulation for Autonomous Driving
Wei Cao; Marcel Hallgarten; Tianyu Li; Daniel Dauner; Xunjiang Gu; Caojun Wang; Yakov Miron; Marco Aiello; Hongyang Li; Igor Gilitschenski; Boris Ivanovic; Marco Pavone; Andreas Geiger; Kashyap Chitta
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
Daniel Dauner; Marcel Hallgarten; Tianyu Li; Xinshuo Weng; Zhiyu Huang; Zetong Yang; Hongyang Li; Igor Gilitschenski; Boris Ivanovic; Marco Pavone; Andreas Geiger; Kashyap Chitta
Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving
Xiaosong Jia; Zhenjie Yang; Qifeng Li; Zhiyuan Zhang; Junchi Yan
RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
Soroush Nasiriany; Abhiram Maddukuri; Lance Zhang; Adeet Parikh; Aaron Lo; Abhishek Joshi; Ajay Mandlekar; Yuke Zhu
Genie: Generative Interactive Environments
Jake Bruce; Michael D Dennis; Ashley Edwards; Jack Parker-Holder; Yuge Shi; Edward Hughes; Matthew Lai; Aditi Mavalankar; Richie Steigerwald; Chris Apps; Yusuf Aytar; Sarah Maria Elisabeth Bechtle; Feryal Behbahani; Stephanie C.Y. Chan; Nicolas Heess; Lucy Gonzalez; Simon Osindero; Sherjil Ozair; Scott Reed; Jingwei Zhang; Konrad Zolna; Jeff Clune; Nando De Freitas; Satinder Singh; Tim Rocktäschel
Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning
Michael Matthews; Michael Beukman; Benjamin Ellis; Mikayel Samvelyan; Matthew Thomas Jackson; Samuel Coward; Jakob Nicolaus Foerster
ACT-Bench: Towards Action Controllable World Models for Autonomous Driving
Hidehisa Arai; Keishi Ishihara; Tsubasa Takahashi; Yu Yamaguchi
GAIA-1: A Generative World Model for Autonomous Driving
Anthony Hu; Lloyd Russell; Hudson Yeo; Zak Murez; George Fedoseev; Alex Kendall; Jamie Shotton; Gianluca Corrado
LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning
Bo Liu; Yifeng Zhu; Chongkai Gao; Yihao Feng; Qiang Liu; Yuke Zhu; Peter Stone
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
Oier Mees; Lukás Hermann; Erick Rosete-Beas; Wolfram Burgard
ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations
Tongzhou Mu; Zhan Ling; Fanbo Xiang; Derek Yang; Xuanlin Li; Stone Tao; Zhiao Huang; Zhiwei Jia; Hao Su
NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles
Holger Caesar; Juraj Kabzan; Kok Seang Tan; Whye Kit Fong; Eric Wolff; Alex Lang; Luke Fletcher; Oscar Beijbom; Sammy Omari
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
Jacob Krantz; Erik Wijmans; Arjun Majumdar; Dhruv Batra; Stefan Lee
RLBench: The Robot Learning Benchmark & Learning Environment
Stephen James; Zicong Ma; David Rovick Arrojo; Andrew J. Davison
Leveraging Procedural Generation to Benchmark Reinforcement Learning
Karl Cobbe; Christopher Hesse; Jacob Hilton; John Schulman
Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning
Tianhe Yu; Deirdre Quillen; Zhanpeng He; Ryan Julian; Karol Hausman; Chelsea Finn; Sergey Levine
Behaviour Suite for Reinforcement Learning
Ian Osband; Yotam Doron; Matteo Hessel; John Aslanides; Eren Sezener; Andre Saraiva; Katrina McKinney; Tor Lattimore; Csaba Szepesvári; Satinder Singh; Benjamin Van Roy; Richard S. Sutton; David Silver; Hado van Hasselt
A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark
Xiaohua Zhai; Joan Puigcerver; Alexander Kolesnikov; Pierre Ruyssen; Carlos Riquelme; Mario Lucic; Josip Djolonga; Andre Susano Pinto; Maxim Neumann; Alexey Dosovitskiy; Lucas Beyer; Olivier Bachem; Michael Tschannen; Marcin Michalski; Olivier Bousquet; Sylvain Gelly; Neil Houlsby
Habitat: A Platform for Embodied AI Research
Manolis Savva; Jitendra Malik; Devi Parikh; Dhruv Batra; Abhishek Kadian; Oleksandr Maksymets; Yili Zhao; Erik Wijmans; Bhavana Jain; Julian Straub; Jia Liu; Vladlen Koltun
DeepMind Control Suite
Yuval Tassa; Yotam Doron; Alistair Muldal; Tom Erez; Yazhe Li; Diego de Las Casas; David Budden; Abbas Abdolmaleki; Josh Merel; Andrew Lefrancq; Timothy Lillicrap; Martin Riedmiller
CARLA: An Open Urban Driving Simulator
Alexey Dosovitskiy; Germán Ros; Felipe Codevilla; Antonio M. López; Vladlen Koltun
The Arcade Learning Environment: An Evaluation Platform for General Agents
Marc G. Bellemare; Yavar Naddaf; Joel Veness; Michael Bowling
Components of WAMs
Reusable encoders, backbones, policies, and learning methods
Component areas 6
V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning
Lorenzo Mur-Labadia; Matthew Muckley; Amir Bar; Mido Assran; Koustuv Sinha; Mike Rabbat; Yann LeCun; Nicolas Ballas; Adrien Bardes
DINOv3
Oriane Siméoni; Huy V. Vo; Maximilian Seitzer; Federico Baldassarre; Maxime Oquab; Cijo Jose; Vasil Khalidov; Marc Szafraniec; Seungeun Yi; Michaël Ramamonjisoa; Francisco Massa; Daniel Haziza; Luca Wehrstedt; Jianyuan Wang; Timothée Darcet; Théo Moutakanni; Leonel Sentana; Claire Roberts; Andrea Vedaldi; Jamie Tolan; John Brandt; Camille Couprie; Julien Mairal; Hervé Jégou; Patrick Labatut; Piotr Bojanowski
World Simulation with Video Foundation Models for Physical AI
NVIDIA; :; Arslan Ali; Junjie Bai; Maciej Bala; Yogesh Balaji; Aaron Blakeman; Tiffany Cai; Jiaxin Cao; Tianshi Cao; Elizabeth Cha; Yu-Wei Chao; Prithvijit Chattopadhyay; Mike Chen; Yongxin Chen; Yu Chen; Shuai Cheng; Yin Cui; Jenna Diamond; Yifan Ding; Jiaojiao Fan; Linxi Fan; Liang Feng; Francesco Ferroni; Sanja Fidler; Xiao Fu; Ruiyuan Gao; Yunhao Ge; Jinwei Gu; Aryaman Gupta; Siddharth Gururani; Imad El Hanafi; Ali Hassani; Zekun Hao; Jacob Huffman; Joel Jang; Pooya Jannaty; Jan Kautz; Grace Lam; Xuan Li; Zhaoshuo Li; Maosheng Liao; Chen-Hsuan Lin; Tsung-Yi Lin; Yen-Chen Lin; Huan Ling; Ming-Yu Liu; Xian Liu; Yifan Lu; Alice Luo; Qianli Ma; Hanzi Mao; Kaichun Mo; Seungjun Nah; Yashraj Narang; Abhijeet Panaskar; Lindsey Pavao; Trung Pham; Morteza Ramezanali; Fitsum Reda; Scott Reed; Xuanchi Ren; Haonan Shao; Yue Shen; Stella Shi; Shuran Song; Bartosz Stefaniak; Shangkun Sun; Shitao Tang; Sameena Tasmeen; Lyne Tchapmi; Wei-Cheng Tseng; Jibin Varghese; Andrew Z. Wang; Hao Wang; Haoxiang Wang; Heng Wang; Ting-Chun Wang; Fangyin Wei; Jiashu Xu; Dinghao Yang; Xiaodong Yang; Haotian Ye; Seonghyeon Ye; Xiaohui Zeng; Jing Zhang; Qinsheng Zhang; Kaiwen Zheng; Andrew Zhu; Yuke Zhu
Seedance 1.0: Exploring the Boundaries of Video Generation Models
Yu Gao; Haoyuan Guo; Tuyen Hoang; Weilin Huang; Lu Jiang; Fangyuan Kong; Huixia Li; Jiashi Li; Liang Li; Xiaojie Li; Xunsong Li; Yifu Li; Shanchuan Lin; Zhijie Lin; Jiawei Liu; Shu Liu; Xiaonan Nie; Zhiwu Qing; Yuxi Ren; Li Sun; Zhi Tian; Rui Wang; Sen Wang; Guoqiang Wei; Guohong Wu; Jie Wu; Ruiqi Xia; Fei Xiao; Xuefeng Xiao; Jiangqiao Yan; Ceyuan Yang; Jianchao Yang; Runkai Yang; Tao Yang; Yihang Yang; Zilyu Ye; Xuejiao Zeng; Yan Zeng; Heng Zhang; Yang Zhao; Xiaozheng Zheng; Peihao Zhu; Jiaxin Zou; Feilong Zuo
Wan: Open and Advanced Large-Scale Video Generative Models
Team Wan; Ang Wang; Baole Ai; Bin Wen; Chaojie Mao; Chen-Wei Xie; Di Chen; Feiwu Yu; Haiming Zhao; Jianxiao Yang; Jianyuan Zeng; Jiayu Wang; Jingfeng Zhang; Jingren Zhou; Jinkai Wang; Jixuan Chen; Kai Zhu; Kang Zhao; Keyu Yan; Lianghua Huang; Mengyang Feng; Ningyi Zhang; Pandeng Li; Pingyu Wu; Ruihang Chu; Ruili Feng; Shiwei Zhang; Siyang Sun; Tao Fang; Tianxing Wang; Tianyi Gui; Tingyu Weng; Tong Shen; Wei Lin; Wei Wang; Wei Wang; Wenmeng Zhou; Wente Wang; Wenting Shen; Wenyuan Yu; Xianzhong Shi; Xiaoming Huang; Xin Xu; Yan Kou; Yangyu Lv; Yifei Li; Yijing Liu; Yiming Wang; Yingya Zhang; Yitong Huang; Yong Li; You Wu; Yu Liu; Yulin Pan; Yun Zheng; Yuntao Hong; Yupeng Shi; Yutong Feng; Zeyinzi Jiang; Zhen Han; Zhi-Fan Wu; Ziyu Liu
Gemma 3 Technical Report
Gemma Team; Aishwarya Kamath; Johan Ferret; Shreya Pathak; Nino Vieillard; Ramona Merhej; Sarah Perrin; Tatiana Matejovicova; Alexandre Ramé; Morgane Rivière; Louis Rouillard; Thomas Mesnard; Geoffrey Cideron; Jean-bastien Grill; Sabela Ramos; Edouard Yvinec; Michelle Casbon; Etienne Pot; Ivo Penchev; Gaël Liu; Francesco Visin; Kathleen Kenealy; Lucas Beyer; Xiaohai Zhai; Anton Tsitsulin; Robert Busa-Fekete; Alex Feng; Noveen Sachdeva; Benjamin Coleman; Yi Gao; Basil Mustafa; Iain Barr; Emilio Parisotto; David Tian; Matan Eyal; Colin Cherry; Jan-Thorsten Peter; Danila Sinopalnikov; Surya Bhupatiraju; Rishabh Agarwal; Mehran Kazemi; Dan Malkin; Ravin Kumar; David Vilar; Idan Brusilovsky; Jiaming Luo; Andreas Steiner; Abe Friesen; Abhanshu Sharma; Abheesht Sharma; Adi Mayrav Gilady; Adrian Goedeckemeyer; Alaa Saade; Alex Feng; Alexander Kolesnikov; Alexei Bendebury; Alvin Abdagic; Amit Vadi; András György; André Susano Pinto; Anil Das; Ankur Bapna; Antoine Miech; Antoine Yang; Antonia Paterson; Ashish Shenoy; Ayan Chakrabarti; Bilal Piot; Bo Wu; Bobak Shahriari; Bryce Petrini; Charlie Chen; Charline Le Lan; Christopher A. Choquette-Choo; CJ Carey; Cormac Brick; Daniel Deutsch; Danielle Eisenbud; Dee Cattle; Derek Cheng; Dimitris Paparas; Divyashree Shivakumar Sreepathihalli; Doug Reid; Dustin Tran; Dustin Zelle; Eric Noland; Erwin Huizenga; Eugene Kharitonov; Frederick Liu; Gagik Amirkhanyan; Glenn Cameron; Hadi Hashemi; Hanna Klimczak-Plucińska; Harman Singh; Harsh Mehta; Harshal Tushar Lehri; Hussein Hazimeh; Ian Ballantyne; Idan Szpektor; Ivan Nardini; Jean Pouget-Abadie; Jetha Chan; Joe Stanton; John Wieting; Jonathan Lai; Jordi Orbay; Joseph Fernandez; Josh Newlan; Ju-yeong Ji; Jyotinder Singh; Kat Black; Kathy Yu; Kevin Hui; Kiran Vodrahalli; Klaus Greff; Linhai Qiu; Marcella Valentine; Marina Coelho; Marvin Ritter; Matt Hoffman; Matthew Watson; Mayank Chaturvedi; Michael Moynihan; Min Ma; Nabila Babar; Natasha Noy; Nathan Byrd; Nick Roy; Nikola Momchev; Nilay Chauhan; Noveen Sachdeva; Oskar Bunyan; Pankil Botarda; Paul Caron; Paul Kishan Rubenstein; Phil Culliton; Philipp Schmid; Pier Giuseppe Sessa; Pingmei Xu; Piotr Stanczyk; Pouya Tafti; Rakesh Shivanna; Renjie Wu; Renke Pan; Reza Rokni; Rob Willoughby; Rohith Vallu; Ryan Mullins; Sammy Jerome; Sara Smoot; Sertan Girgin; Shariq Iqbal; Shashir Reddy; Shruti Sheth; Siim Põder; Sijal Bhatnagar; Sindhu Raghuram Panyam; Sivan Eiger; Susan Zhang; Tianqi Liu; Trevor Yacovone; Tyler Liechty; Uday Kalra; Utku Evci; Vedant Misra; Vincent Roseberry; Vlad Feinberg; Vlad Kolesnikov; Woohyun Han; Woosuk Kwon; Xi Chen; Yinlam Chow; Yuvein Zhu; Zichuan Wei; Zoltan Egyed; Victor Cotruta; Minh Giang; Phoebe Kirk; Anand Rao; Kat Black; Nabila Babar; Jessica Lo; Erica Moreira; Luiz Gustavo Martins; Omar Sanseviero; Lucas Gonzalez; Zach Gleicher; Tris Warkentin; Vahab Mirrokni; Evan Senter; Eli Collins; Joelle Barral; Zoubin Ghahramani; Raia Hadsell; Yossi Matias; D. Sculley; Slav Petrov; Noah Fiedel; Noam Shazeer; Oriol Vinyals; Jeff Dean; Demis Hassabis; Koray Kavukcuoglu; Clement Farabet; Elena Buchatskaya; Jean-Baptiste Alayrac; Rohan Anil; Dmitry; Lepikhin; Sebastian Borgeaud; Olivier Bachem; Armand Joulin; Alek Andreev; Cassidy Hardin; Robert Dadashi; Léonard Hussenot
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
NVIDIA; :; Hassan Abu Alhaija; Jose Alvarez; Maciej Bala; Tiffany Cai; Tianshi Cao; Liz Cha; Joshua Chen; Mike Chen; Francesco Ferroni; Sanja Fidler; Dieter Fox; Yunhao Ge; Jinwei Gu; Ali Hassani; Michael Isaev; Pooya Jannaty; Shiyi Lan; Tobias Lasser; Huan Ling; Ming-Yu Liu; Xian Liu; Yifan Lu; Alice Luo; Qianli Ma; Hanzi Mao; Fabio Ramos; Xuanchi Ren; Tianchang Shen; Xinglong Sun; Shitao Tang; Ting-Chun Wang; Jay Wu; Jiashu Xu; Stella Xu; Kevin Xie; Yuchong Ye; Xiaodong Yang; Xiaohui Zeng; Yu Zeng
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Karl Pertsch; Kyle Stachowicz; Brian Ichter; Danny Driess; Suraj Nair; Quan Vuong; Oier Mees; Chelsea Finn; Sergey Levine
Latent Action Pretraining from Videos
Seonghyeon Ye; Joel Jang; Byeongguk Jeon; Sejune Joo; Jianwei Yang; Baolin Peng; Ajay Mandlekar; Reuben Tan; Yu-Wei Chao; Bill Yuchen Lin; Lars Liden; Kimin Lee; Jianfeng Gao; Luke Zettlemoyer; Dieter Fox; Minjoon Seo
Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
Weixin Liang; Lili Yu; Liang Luo; Srinivasan Iyer; Ning Dong; Chunting Zhou; Gargi Ghosh; Mike Lewis; Wen-tau Yih; Luke Zettlemoyer; Xi Victoria Lin
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer
Zhuoyi Yang; Jiayan Teng; Wendi Zheng; Ming Ding; Shiyu Huang; Jiazheng Xu; Yuanming Yang; Wenyi Hong; Xiaohan Zhang; Guanyu Feng; Da Yin; Yuxuan Zhang; Weihan Wang; Yean Cheng; Bin Xu; Xiaotao Gu; Yuxiao Dong; Jie Tang
PaliGemma: A versatile 3B VLM for transfer
Lucas Beyer; Andreas Steiner; André Susano Pinto; Alexander Kolesnikov; Xiao Wang; Daniel Salz; Maxim Neumann; Ibrahim Alabdulmohsin; Michael Tschannen; Emanuele Bugliarello; Thomas Unterthiner; Daniel Keysers; Skanda Koppula; Fangyu Liu; Adam Grycner; Alexey Gritsenko; Neil Houlsby; Manoj Kumar; Keran Rong; Julian Eisenschlos; Rishabh Kabra; Matthias Bauer; Matko Bošnjak; Xi Chen; Matthias Minderer; Paul Voigtlaender; Ioana Bica; Ivana Balazevic; Joan Puigcerver; Pinelopi Papalampidi; Olivier Henaff; Xi Xiong; Radu Soricut; Jeremiah Harmsen; Xiaohua Zhai
DINOv2: Learning Robust Visual Features without Supervision
Maxime Oquab; Timothée Darcet; Théo Moutakanni; Huy V. Vo; Marc Szafraniec; Vasil Khalidov; Pierre Fernandez; Daniel Haziza; Francisco Massa; Alaaeldin El-Nouby; Mido Assran; Nicolas Ballas; Wojciech Galuba; Russell Howes; Po-Yao Huang; Shang-Wen Li; Ishan Misra; Michael Rabbat; Vasu Sharma; Gabriel Synnaeve; Hu Xu; Hervé Jégou; Julien Mairal; Patrick Labatut; Armand Joulin; Piotr Bojanowski
Video generation models as world simulators
OpenAI
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Siddharth Karamcheti; Suraj Nair; Ashwin Balakrishna; Percy Liang; Thomas Kollar; Dorsa Sadigh
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
Andreas Blattmann; Tim Dockhorn; Sumith Kulal; Daniel Mendelevitch; Maciej Kilian; Dominik Lorenz; Yam Levi; Zion English; Vikram Voleti; Adam Letts; Varun Jampani; Robin Rombach
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
Jinze Bai; Shuai Bai; Shusheng Yang; Shijie Wang; Sinan Tan; Peng Wang; Junyang Lin; Chang Zhou; Jingren Zhou
LLaMA: Open and Efficient Foundation Language Models
Hugo Touvron; Thibaut Lavril; Gautier Izacard; Xavier Martinet; Marie-Anne Lachaux; Timothée Lacroix; Baptiste Rozière; Naman Goyal; Eric Hambro; Faisal Azhar; Aurelien Rodriguez; Armand Joulin; Edouard Grave; Guillaume Lample
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Bernhard Kerbl; Georgios Kopanas; Thomas Leimkühler; George Drettakis
PaLM-E: An Embodied Multimodal Language Model
Danny Driess; Fei Xia; Mehdi S. M. Sajjadi; Corey Lynch; Aakanksha Chowdhery; Brian Ichter; Ayzaan Wahid; Jonathan Tompson; Quan Vuong; Tianhe Yu; Wenlong Huang; Yevgen Chebotar; Pierre Sermanet; Daniel Duckworth; Sergey Levine; Vincent Vanhoucke; Karol Hausman; Marc Toussaint; Klaus Greff; Andy Zeng; Igor Mordatch; Pete Florence
Scalable Diffusion Models with Transformers
William Peebles; Saining Xie
Sigmoid Loss for Language Image Pre-Training
Xiaohua Zhai; Basil Mustafa; Alexander Kolesnikov; Lucas Beyer
High-Resolution Image Synthesis With Latent Diffusion Models
Robin Rombach; Andreas Blattmann; Dominik Lorenz; Patrick Esser; Björn Ommer
BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View
Junjie Huang; Guan Huang; Zheng Zhu; Yun Ye; Dalong Du
Learning Transferable Visual Models From Natural Language Supervision
Alec Radford; Jong Wook Kim; Chris Hallacy; Aditya Ramesh; Gabriel Goh; Sandhini Agarwal; Girish Sastry; Amanda Askell; Pamela Mishkin; Jack Clark; Gretchen Krueger; Ilya Sutskever
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel; Noam Shazeer; Adam Roberts; Katherine Lee; Sharan Narang; Michael Matena; Yanqi Zhou; Wei Li; Peter J. Liu
Language Models are Few-Shot Learners
Tom Brown; Benjamin Mann; Nick Ryder; Melanie Subbiah; Jared D Kaplan; Prafulla Dhariwal; Arvind Neelakantan; Pranav Shyam; Girish Sastry; Amanda Askell; Sandhini Agarwal; Ariel Herbert-Voss; Gretchen Krueger; Tom Henighan; Rewon Child; Aditya Ramesh; Daniel Ziegler; Jeffrey Wu; Clemens Winter; Chris Hesse; Mark Chen; Eric Sigler; Mateusz Litwin; Scott Gray; Benjamin Chess; Jack Clark; Christopher Berner; Sam McCandlish; Alec Radford; Ilya Sutskever; Dario Amodei
Neural Discrete Representation Learning
Aaron van den Oord; Oriol Vinyals; Koray Kavukcuoglu
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles R. Qi; Hao Su; Kaichun Mo; Leonidas J. Guibas
Auto-Encoding Variational Bayes
Diederik P Kingma; Max Welling
Related resources
Subcategories 14
Valerant: An Automatic Navigable Game Map Generator via Action-Conditioned World Model Exploration
Yiran Qiao; Feng Wang; Jing Ma
Toward Unified Robot Learning: Bridging Representation, Vision-Language-Action, and World Models
Shaunak A. Mehta; Ananya Hazarika; Haochen Zhang; Fan Yang; Ryo Moriyama; Wenkai Li; Yash Patel; Kanata Suzuki
From World Models to World Action Models: A Concise Tutorial for Robotics
Zhang, Xiaoxiong; Zeng, Xiong; Zhang, Wei
On the Capability Separation Between World-Model Policy Learning and Imitated World-Action Models
Yu, Yang
Teach and Grow: An Agent-Centered Architecture for General Robot Learning
Nie, Chang; Liu, Zhe; Wang, Hesheng
PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
Chenghua Wang; Daliang Xu; Dongqi Cai; Duojin Sun; Hao Zhang; Haoze Qian; Huaiyuan Zhang; Jinshuo Cui; Junbo Cui; Kezhao Zhao; Longxi Gao; Mengwei Xu; Rongjie Yi; Ruixin Liu; Shangguang Wang; Tam Sikyuen; Tianyue Zhang; Weikai Xie; Xuanzhe Liu; Yingying Qin; Yiwen Lu; Yuan Yao; Yuezhi Zu; Yunhan Guo; Yuxin Zheng; Ziqi Guo
World Action Models in Real Time: An Empirical Study of Smooth Execution via Asynchronous Deployment
Team, Motubrain
Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots
Xu, Ling; Li, Borui; Wu, Hao; Han, Chuyu; Li, Xiangyu; Hua, Mohan; Jiang, Shiqi; Cao, Ting; Li, Chuanyou; Zhong, Sheng; Wang, Shuai
Data Pyramid for Embodied Manipulation: A Survey
Yifan Ye; Yankai Fu; Yaoxu Lv; Bohan Hou; Jun Cen; Lingdong Kong; Duo Zheng; Tianxing Chen; Jiaming Liu; Ziang Cao; Yunfan Lou; Wei Chow; Xian Sun; Yingshuo Wang; Kuangzhi Ge; Xiaowei Chi; Xidong Zhang; Zhibo Pang; Yiwu Zhong; Sirui Han; Zhihe Lu; Weihao Yuan; Qifeng Chen; Michael Yu Wang; Yao Mu; Ziwei Liu; Jianfei Yang; Ping Luo; Shanghang Zhang
RoboHarness: Memory-Driven Orchestration of Heterogeneous Robot Policies for Long-Horizon Planning
Huang, Jinbang; Hu, Yuanzhao; Li, Zhiyuan; Qi, Ran; Xiao, Yixin; Zhang, Zhanguang; Coates, Mark; Cao, Tongtong; Zhang, Yingxue
BadWAM: When World-Action Models Dream Right but Act Wrong
Li, Qi; Yang, Xingyi; Wang, Xinchao
From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence
Liang, Yuanzhi; Zhan, Xufeng; Huang, Haibin; Zhang, Chi; Li, Xuelong
Critique of Agent Model
Eric Xing; Mingkai Deng; Jinyu Hou
A Watermark for Vision-Language-Action and World Action Models
Liu, Yule; Liu, Shuai; Wei, Jiaheng; He, Xinlei
Attacking the Trusted Imagination: Oracle-Level Integrity Attacks on Imagine-then-Act World Models
Chen, Linghan; Ji, Kaiyan; Guo, Minyu
World Action Models: A Survey
Shen, Qiuhong; Zhang, Shihua; Liao, Yue; Li, Qi; Tan, Zhenxiong; Wang, Shizun; Yan, Shuicheng; Wang, Xinchao
World Action Models: The Next Frontier in Embodied AI
Wang, Siyin; Shi, Junhao; Fu, Zhaoyang; He, Xinzhe; Liu, Feihong; Yang, Chenchen; Zhou, Yikang; Fei, Zhaoye; Gong, Jingjing; Fu, Jinlan; Shou, Mike Zheng; Huang, Xuanjing; Qiu, Xipeng; Jiang, Yu-Gang
World Model for Robot Learning: A Comprehensive Survey
Bohan Hou; Gen Li; Jindou Jia; Tuo An; Xinying Guo; Sicong Leng; Haoran Geng; Yanjie Ze; Tatsuya Harada; Philip Torr; Oier Mees; Marc Pollefeys; Zhuang Liu; Jiajun Wu; Pieter Abbeel; Jitendra Malik; Yilun Du; Jianfei Yang
Factored Latent Action World Models
Zizhao Wang; Chang Shi; Jiaheng Hu; Kevin Rohling; Roberto Martín-Martín; Amy Zhang; Peter Stone
Towards Generalist Embodied AI: A Survey on World Models for VLA Agents
Wentao Tan; Lei Zhu; Bowen Wang; Enci Xie; Baixu Ji; Zengrong Lin; Wenjie Yang; Jingjing Li; Heng Tao Shen
Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrations
Irmak Guzey; Haozhi Qi; Julen Urain; Changhao Wang; Jessica Yin; Krishna Bodduluri; Mike Lambeta; Lerrel Pinto; Akshara Rai; Jitendra Malik; Tingfan Wu; Akash Sharma; Homanga Bharadhwaj
Learning Generalizable Robot Policy with Human Demonstration Video as a Prompt
Xiang Zhu; Yichen Liu; Hezhong Li; Jianyu Chen
CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrations
Anthony Liang; Pavel Czempin; Matthew M. Hong; Yutai Zhou; Jingzhen Wang; Erdem Bıyık; Stephen Tu
The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey
Sifan Tu; Xin Zhou; Dingkang Liang; Xingyu Jiang; Yumeng Zhang; Xiaofan Li; Xiang Bai
A review of learning-based dynamics models for robotic manipulation
Bo Ai; Stephen Tian; Haochen Shi; Yixuan Wang; Tobias Pfaff; Cheston Tan; Henrik I Christensen; Hao Su; Jiajun Wu; Yunzhu Li
FAST-LIVO2: Fast, Direct LiDAR-Inertial-Visual Odometry
Chunran Zheng; Wei Xu; Zuhao Zou; Tong Hua; Chongjian Yuan; Dongjiao He; Bingyang Zhou; Zheng Liu; Jiarong Lin; Fangcheng Zhu; Yunfan Ren; Rong Wang; Fanle Meng; Fu Zhang
EgoBridge: Domain Adaptation for Generalizable Imitation from Egocentric Human Data
Ryan Punamiya; Dhruv Patel; Patcharapong Aphiwetsa; Pranav Kuppili; Lawrence Zhu; Simar Kareer; Judy Hoffman; Danfei Xu
A Survey on Vision-Language-Action Models for Autonomous Driving
Sicong Jiang; Zilin Huang; Kangan Qian; Ziang Luo; Tianze Zhu; Yang Zhong; Yihong Tang; Menglin Kong; Yunlong Wang; Siwen Jiao; Hao Ye; Zihao Sheng; Xin Zhao; Tuopu Wen; Zheng Fu; Sikai Chen; Kun Jiang; Diange Yang; Seongjin Choi; Lijun Sun
DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control
Zichen Jeff Cui; Hengkai Pan; Aadhithya Iyer; Siddhant Haldar; Lerrel Pinto
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection
Hao Zhang; Feng Li; Shilong Liu; Lei Zhang; Hang Su; Jun Zhu; Lionel M. Ni; Heung-Yeung Shum
LIC-Fusion: LiDAR-Inertial-Camera Odometry
Xingxing Zuo; Patrick Geneva; Woosik Lee; Yong Liu; Guoquan Huang
Unassigned category
Major category not yet recorded
Subcategories 1
Diagnosing and Mitigating Perception-Decision Misalignment in Omni-LLMs via Modality Subspace Activation
Jiang, Hongbo; Li, Jie; Shen, Yunhang; Xie, Tianyu; Dai, Pingyang
BERT: A Review of Applications in Natural Language Processing and Understanding
M. V. Koroteev
Advances and applications of occupancy models
Larissa L Bailey; Darryl I MacKenzie; James D Nichols
Four quadrants
Rows show architecture; columns show prediction paradigm. Each cell previews four references. Open a count to see all.
One Model× Joint prediction
- JEPA Policy: Diffusion-Free Imitation Learning via Paired Action and Future Representation Prediction
- WorldAgen: Unified State-Action Prediction with Test-Time World Model Training
- SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
- Spatially Aware World Action Model via Geometric Latent Diffusion
One Model× IDM
- GlanceWAM: Sparse Test-Time Imagination for World-Action Models
- Towards Predictive, Aligned, and Scalable Robot Learning
- Geometric Action Model for Robot Policy Learning
- DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
Dual-system× Joint prediction
- OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
- Latent Action as Intention Enables Efficient Future Imagination for World Action Models
- Flex-ππ: A Multi-Stream World-Action Model with Compute Flexibility
- GameWAM: A World Action Model for Video Games
Dual-system× IDM
- GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
- ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training
- AcrossWAM1.0:A Modular Latent World-Action Stack for Compact Robot Policies
- Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization
Beyond the matrix
These references retain their separate classification status.
Outside quadrants
118Classified outside this four-quadrant framework.
Browse referencesNot applicable
270The source marks this framework as not applicable.
Browse referencesPending verification
4A quadrant assignment has not yet been verified.
Browse referencesUnassigned quadrant
23No quadrant assignment is recorded in the catalog.
Browse referencesResearch topics
Topics provide an additional classification alongside major categories and quadrants.
3D / 4D
Geometry, spatial structure, and dynamic scenes in action-conditioned world models.
Browse topicReinforcement learning
World models connected to policy learning, rewards, and decision-making.
Browse topicMemory
Persistent state and context for reasoning and acting across longer horizons.
Browse topicMultimodal / tactile
Combining vision, touch, and other signals to represent and interact with the world.
Browse topicReal-time / efficient
Efficient prediction, planning, and inference for practical control.
Browse topicLatent / representation
Learning compact representations and predicting dynamics in latent space.
Browse topicNavigation / domains
World-Action Models for navigation, driving, and specialized environments.
Browse topicEvaluation / theory
Benchmarks, analysis, surveys, and foundations for understanding world models.
Browse topicGeneral WAMs
General World-Action Models that connect prediction with action.
Browse topic174 references have an additional research topic; 390 have no research topic recorded. Major categories and quadrants are maintained separately.
Browse references without a research topic