World Model for Robot Learning: A Comprehensive Survey
1. Paper overview
In one sentence: The survey maps how prediction becomes useful for robot action, while showing why architectural integration and visually plausible futures do not alone establish reliable control. e02e05e11e19e23
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that reactive vision-language-action policies can struggle with long horizons and compounding errors because action prediction alone supplies insufficient foresight. They organize a fragmented literature around control-relevant prediction. Their definition includes both motor commands and semantic task instructions, while emphasizing that an actionable predictor must preserve consequences useful to robot decision making. e02e03 |
| Core mechanism | Source description A policy-centered taxonomy separates decoupled inverse dynamics, shared video–action backbones, interacting experts, unified VLAs and latent predictive policies, while tracking inference-time future generation. e05 |
| A key reported result | LIBERO standard four-suite manipulation (survey compilation): Cosmos Policy: 98.1/100.0/98.2/97.6, average 98.5. LingBot-VA: 98.5/99.6/97.2/98.5, average 98.5. Reported success rate (%) and directly reported four-suite average. Spatial/Object/Goal/Long protocol; numbers compiled from cited methods, with no common training-budget or seed specification. Say-Dream-ACT average 98.1; Motus 97.7; VLA-JEPA 97.2. The equal rounded averages conceal different suite profiles, and high values occur across several coupling styles. These are compiled results, not a matched architecture ablation. No uncertainty is supplied; unreported entries remain missing. e19e23 |
| Reading caution |
Core contributions
- Source description
A policy-centered taxonomy separates decoupled inverse dynamics, shared video–action backbones, interacting experts, unified VLAs and latent predictive policies, while tracking inference-time future generation. e05
Figure 3. Where prediction meets action: serial modules, shared parameters, or interacting experts. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel from the bottom upward. In (a), observations enter a video generator, whose predicted trajectory is passed to a distinct inverse dynamics model that produces actions. In (b), blue observation tokens and orange action tokens enter one shared backbone, with both modalities represented at its output. In (c), the colored video and action streams remain distinguishable while a joint-attention block connects them. The legend on the right separates actual modality icons from their token representations. Sections 3.2–3.4 and Equations (9)–(15) confirm that the decisive comparison concerns the interface and parameter sharing between prediction and action generation. e06e07e08e05
What it supports. The figure supports three distinct ways to couple predictive information to control. A shared training objective does not erase the difference between a downstream action decoder and repeatedly interacting experts. The survey uses this architectural distinction to organize the literature, without presenting a controlled experiment that selects a winner.
Where the evidence stops. This is a family-level schematic. It does not specify each method's attention mask, training schedule or active inference branches. Some jointly trained designs bypass visual generation at deployment, so the output icons should not be treated as mandatory online computation.
2. Motivation
2.1 The problem and the proposed response
The authors argue that reactive vision-language-action policies can struggle with long horizons and compounding errors because action prediction alone supplies insufficient foresight. They organize a fragmented literature around control-relevant prediction. Their definition includes both motor commands and semantic task instructions, while emphasizing that an actionable predictor must preserve consequences useful to robot decision making. e02e03
2.2 What this reading follows
A robot can use a predicted future in several different ways: as a plan to translate into actions, as an internal training target, or as a simulated consequence for choosing among actions. This survey's value is in keeping those uses distinguishable while connecting their shared predictive structure. Read the diagrams as a map of interfaces and responsibilities, then read the result tables as a compilation with uneven experimental controls. The tutorial follows that path from policy architecture to simulator reliability and benchmark interpretation. All numerical results are attributed to the survey's accounts of earlier work; this paper contributes synthesis rather than a new trained system. e02e05e11e19e23
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Related resources. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The catalog correctly treats this as foundational survey/technical-resource work. The paper organizes multiple architectures and predictive mechanisms rather than proposing one deployable architecture. Its own five-family taxonomy supports leaving architecture, prediction paradigm and quadrant not applicable. Joint predictive-control notation or co-training does not justify assigning the survey itself to a One Model quadrant. e02e04e05
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Ask which future–action query the system answers
Section 3.1 starts from a joint distribution over future observations and actions conditioned on the current observation and instruction. That creates a useful reading procedure: identify what is given and what must be predicted. A policy predicts actions; a passive predictor forecasts a task-consistent future; a controllable world model forecasts consequences of supplied actions; inverse dynamics recovers actions from transitions. Figure 3 then shows different implementation interfaces for these queries. A decoupled predictor can supply a visual plan to a separate policy, while a shared generator processes visual and action representations together. Expert architectures retain distinct parameters and exchange predictive information through attention. The survey presents this as an idealized probabilistic lens and architectural taxonomy, not proof that every implementation represents the same learned joint distribution. e04e06e07e08
Figure 4. Predictive knowledge can guide a policy through explicit outputs or compact internal representations. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the common inputs: observation and language enter both panels. The left panel presents a unified multimodal model with several possible outputs, including action and future-oriented visual prediction. The right panel replaces the explicit image output with a compact world representation, followed by an arrow to action. Read that arrow as the conceptual interface from predictive knowledge to control, not as proof of a separately implemented decoder in every listed system. Section 3.5 explains that future-image prediction may mainly supervise training, while Section 3.6 describes future embeddings or conditions that avoid decoding a full visual future. e09e10
What it supports. The organizing distinction is the form of predictive information available to action learning. World modeling need not mean generating a video at each control step. The survey includes both explicit auxiliary prediction and latent alignment as ways of introducing future structure into policies.
Where the evidence stops. The source's unified-VLA category includes specialized experts as well as shared models. Neither this diagram nor a joint-training label establishes full parameter sharing. The image also does not demonstrate that a latent representation preserves every physically relevant variable.
5.2 Trace prediction through training and deployment separately
Reader analysis: the most useful second pass is to draw two information flows, one for optimization and one for action execution. Figure 4 explains why they can differ. Future frames may supply supervised targets during learning without ever becoming observations available to the deployed policy. The survey's accounts of latent alignment and training-oriented future-image prediction make this distinction concrete. Likewise, shared video–action models can omit or reduce their visual branch at test time. Figure 5 adds another distinction: using imagined trajectories to update a policy differs from scoring candidate actions online. Consequently, evidence that predictive co-training improves representations would not by itself establish the benefit of online imagination. A discriminating experiment must control training while changing deployment computation, or control deployment while changing predictive supervision. e05e07e08e09e10e11
Figure 5. The same predictive environment can supply training experience or support action selection. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. In the left panel, follow the policy's action into the world model and its imagined observation back toward the policy. Reward feedback supplies the policy-update path. Equations (16)–(18) formalize this use of imagined transitions and optional rewards or termination signals for reinforcement learning. In the right panel, begin with the observation entering the policy at the bottom, follow the three candidate actions upward, and compare the world model's schematic scores. Section 4.2 explains how predicted consequences support ranking, rejection or revision before execution. The two panels distinguish an update to policy parameters from a decision among proposed behaviors. e11e12e13
What it supports. The left role seeks policy improvement through simulated interaction; the right role seeks better selection using predicted outcomes. Both rely on trustworthy action-conditioned dynamics. Section 4.1 further discusses repairing the simulator with policy failures or real interaction, because improved optimization cannot compensate for arbitrary simulation errors.
Where the evidence stops. The scores 0.6, 0.8 and 0.3 illustrate ranking; they are not measured success rates or calibrated probabilities. The diagram supplies no validation of simulator-to-real agreement. Hallucinated futures can corrupt rewards and action rankings.
5.3 Match the evaluation to the model's intended role
Reader analysis: choose an evaluation question before reading the benchmark table. A generator used for synthetic supervision needs action-relevant and executable trajectories. A policy evaluator needs rankings that agree with actual execution. A planner needs reliable consequences across repeated action and replanning steps. Sections 7.1 and 8.6 separate these requirements into open-loop quality, closed-loop utility and targeted diagnostics. Tables 5–6 then provide downstream context, but cannot replace those tests: equal LIBERO averages can conceal distinct suite profiles, while RoboTwin settings expose sensitivity to randomization. Their missing entries and protocol differences also prevent a clean pooled ranking. The survey therefore motivates a layered evidence standard: check the prediction, check its action dependence, and check whether the intended downstream decision actually improves. e13e17e19e20e23
Figure 6. From imagined demonstrations to action-sensitive, structured future prediction. Original paper, p. 19 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the arrows entering the robotic video world model: observations provide the current scene, language supplies semantic intent, actions specify interventions, and structure adds interaction or geometric constraints. The model's future observations leave to the left and feed imagination-driven supervision. The lower branches map onto different parts of Section 5, so the figure organizes capabilities rather than prescribing a single mandatory implementation. Section 5.3 makes action faithfulness central, and Section 5.4 discusses masks, geometry and multi-view cues. Section 5.5, described in the caption and text, adds the broader goal of reusing strong video backbones for interactive world modeling. e14e15e21
What it supports. The survey's progression changes what a generated future must preserve. Useful synthetic supervision motivates imagination, while simulator and planning roles demand faithful responses to actions and coherent interaction structure. Increasing visual quality alone does not establish these additional capabilities.
Where the evidence stops. The arrow from imagination-driven supervision toward the action branch is an organizing schematic, not a specified optimization algorithm. The 'causal alignment' and 'physics consistent' labels express intended capabilities; they do not establish causal identification or physical guarantees.
5.4 Training and inference
During training
There is no single survey training recipe. Decoupled predictors are typically pretrained and frozen or lightly adapted. Shared generators learn visual/action targets through diffusion, flow or discrete denoising objectives. Unified and latent policies can instead obtain predictive benefits through auxiliary supervision. The survey does not establish matched-data superiority of video pretraining over VLM pretraining. e06e07e09e10
Tables 3–4 compare action supervision, embodiment coverage, richer observations, language and contact signals, then their relevance to different modeling capabilities. These are qualitative, overlapping axes rather than measured dataset rankings. Section 7.2 highlights scarce failure-recovery and physically grounded supervision; it supplies no common data mixture or optimization budget. e18
During inference
The visual branch may be fully generated, partially used, or omitted. The survey describes Cosmos Policy's direct mode as requiring actions, with future states and values used for planning; Fast-WAM uses video co-training while skipping test-time video. Therefore, training with a world-model objective does not by itself imply online rollout. e05e07e08e21
For simulator-based selection, the policy proposes actions and the world model predicts their consequences before ranking or replanning. Executed actions must still interact with the actual environment. A model's imagined reward or success estimate is a prediction whose fidelity requires separate evaluation. e11e13e17
5.5 Implementation flow
- Choose the predictive query
Section 3.1 treats future observations and actions as an idealized joint distribution. Marginalizing futures yields a policy; marginalizing actions yields passive prediction. Action-conditioned prediction asks what a proposed action causes, while inverse dynamics asks which actions explain a state transition. This conceptual relationship does not prove that implementations share one network. e03e04
- Locate the prediction–action interface
In the decoupled family, a world model supplies future frames or compact visual plans to a separate policy. Shared-backbone methods instead jointly process visual and action representations. MoT designs preserve specialized streams coupled through attention. The relevant distinction is parameterization and information flow, not merely whether losses are jointly optimized. e06e07e08
- Identify internal predictive supervision
Unified VLAs add future images or structured predictive outputs to action learning. Latent variants replace image reconstruction with future embeddings or conditions. FLARE aligns action-network features with future embeddings; VLA-JEPA uses future frames as target supervision. Symbolic transition models provide a complementary interface to high-level planning. e09e10
- Separate simulation from action selection
A learned simulator predicts transitions and possibly rewards or termination for imagined policy optimization. An evaluator instead scores proposed behaviors or policy checkpoints. Co-evolving systems refine the simulator using policy failures or real data before generating further training experience. Reliability of the imagined consequences matters in either role. e11e12e13
- Track video-model capability
The video-generation taxonomy progresses from imagined supervision to explicit action conditioning, then interaction/geometry priors and reusable foundation backbones. These are overlapping capabilities. IRASim's frame-level action conditioning illustrates control alignment; multi-view and geometric structure address consistency. Neither a plausible video nor an architectural label establishes executability. e14e15
- Transfer the questions across domains
Navigation uses predictions of unseen views to support planning under partial observability. Driving adds multi-agent evolution and structured scene representations such as occupancy or latent states. These applications broaden the organizing framework without making manipulation benchmark scores evidence for navigation or driving success. e16
6. Experiments & results
This survey organizes robotic world models by how prediction connects to action: as part of a policy, as a learned simulator, or as a generator of useful visual futures. Its central reading lesson is to track both the predictive representation and when it is used. Joint training, online imagination and real action execution are different claims. The benchmark compilation illustrates competitive methods across several architectures without establishing a universally superior design.
This survey presents conceptual architecture/capability diagrams and compilations of earlier papers' results. It contains no new controlled ablation, original training run or empirical diagnostic plot isolating a mechanism. The six visuals therefore comprise four conceptual figures and two original survey tables; none is labeled an ablation. The table measurements remain attributed to the survey's cited works, and its schematic candidate scores are illustrative. e02e11e14e23
6.1 Read the original evidence
Table 5. Equal averages can conceal different LIBERO suite profiles. Original paper, p. 28 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row before comparing averages. The four named columns are LIBERO's Spatial, Object, Goal and Long suites; the final column is an average directly reported by the cited paper. Cosmos Policy and LingBot-VA both show 98.5 there, but their component values differ. Compare their Object entries, 100.0 and 99.6, with their Long entries, 97.6 and 98.5. Then use the group labels to see that these rows belong to different coupling families. Keep the footnotes in view: a dash means that a standard-protocol result was not directly reported, and missing averages are not filled in by this report. e19e23
What it supports. Both Cosmos Policy and LingBot-VA reach a reported 98.5% average, while Say-Dream-ACT reaches 98.1%, Motus 97.7% and VLA-JEPA 97.2%. The compilation supports the survey's observation that high benchmark performance appears across multiple architectural families; it does not isolate which architectural mechanism caused the scores.
Where the evidence stops. These are earlier papers' reported results as compiled by the survey, not common-budget reruns. Seeds, uncertainty and matched training conditions are absent here. A rounded tie is not a statistical equivalence result, and simulated manipulation performance does not establish physical deployment success.
Table 6. Protocol boundaries matter more than a single cross-benchmark leaderboard. Original paper, p. 29 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the footnotes. RT-A and RT-B distinguish non-randomized and randomized RoboTwin testing environments. Compare those two columns within one method before moving to another: LingBot-VA lists 92.9 and 91.6, Motus 88.7 and 87.0, and BagelVLA 75.3 and 20.9. The remaining columns refer to CALVIN and SIMPLER-style evaluations, so they are not additional observations of the same metric under the same setting. S-G, S-W and S-O distinguish robot setups. Slash-separated entries remain paired protocol variants, and dashes remain missing reports. This layout is useful for locating evidence, but its columns should not be averaged together. e20e23
What it supports. The selected RoboTwin rows show different performance profiles under randomized testing. LingBot-VA's reported values remain close across RT-A and RT-B, while BagelVLA's differ markedly. This makes setting sensitivity visible and motivates checking robustness under a fixed protocol, without treating the table as a causal test of architecture.
Where the evidence stops. The CALVIN footnote prints 'ABCD' and 'ABCDD' without train/test arrows; their exact split interpretation remains unresolved. Paired SIMPLER variants are also not identified individually. The survey explicitly cautions that these heterogeneous results are less suitable for strict ranking.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO standard four-suite manipulation (survey compilation) Spatial/Object/Goal/Long protocol; numbers compiled from cited methods, with no common training-budget or seed specification. | Cosmos Policy: 98.1/100.0/98.2/97.6, average 98.5. LingBot-VA: 98.5/99.6/97.2/98.5, average 98.5. Reported success rate (%) and directly reported four-suite average | Say-Dream-ACT average 98.1; Motus 97.7; VLA-JEPA 97.2. The equal rounded averages conceal different suite profiles, and high values occur across several coupling styles. These are compiled results, not a matched architecture ablation. No uncertainty is supplied; unreported entries remain missing. e19e23 |
| RoboTwin simple and randomized manipulation (survey compilation) Table 6 RT-A: non-randomized testing; RT-B: harder randomized environments. Section 7.3 discusses RoboTwin 2.0. | LingBot-VA: RT-A 92.9, RT-B 91.6. Compiled task success rate (%) | Motus: 88.7/87.0; BagelVLA: 75.3/20.9 in RT-A/RT-B order. The profiles vary strongly with setting. The survey does not supply matched training conditions or uncertainty, so the differences cannot isolate world-model architecture or prove transfer to physical robots. e20e23 |
6.3 Ablations and diagnostic examples
No ablation or diagnostic visual is included in this edition. The source-specific visual limitations are recorded above.
7. Analysis & limitations
7.1 What the evidence leaves open
The source is a narrative survey with representative comparisons, not a new controlled experiment. Tables 5–6 cannot isolate the causal benefit of world modeling from pretraining, model size, data or deployment differences. e02e23
Protocol reporting is incomplete: Table 6 prints CALVIN labels as 'ABCD' and 'ABCDD' without clarifying train/test arrows, and slash-separated values are only identified as original protocol variants. Missing entries are not zero; the compilation contains no common uncertainty estimates. e20e23
The authors identify action-conditioning gaps, expensive iterative denoising, sparse nonvisual physical feedback, asynchronous sensor fusion, control guarantees and grounded symbolic abstraction as unresolved issues. These are research challenges rather than quantified failure rates for one model. e21e22
Predictive realism, policy-ranking fidelity and executability are different evaluation targets. Good appearance cannot establish reliable planning, and latent predictors need not render realistic videos to be useful. e17e23
7.2 Questions for discussion
- When does predictive co-training improve control without an inference-time world model?
- Which action-sensitive diagnostic best predicts whether imagined policy rankings survive real execution?
- What comparison controls are needed before attributing benchmark gains to a coupling architecture?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful reconstruction of the survey's comparisons needs exact underlying paper versions, benchmark splits, embodiment/action conventions, training data, seeds and evaluation budgets. Tables 5–6 give useful row/column anchors but omit enough configuration that they are insufficient as standalone reproduction recipes. e19e20e23
Reader-proposed priorities are a controlled action-intervention test with fixed observations/instructions and a benchmark extraction audit preserving missing entries and protocol variants. These test the survey's causal-conditioning and comparison arguments; neither check was performed as an experiment here. e17e20e21
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Intervene on actions while holding intent fixed
Reader-proposed experiment, not performed: for held-out manipulation states, keep observation history, language, prediction horizon and sampling seeds fixed. Compare rollouts conditioned on the recorded action sequence, a zero-motion control and a deliberately altered valid sequence. Obtain corresponding environment transitions using resettable initial states. Measure action-sensitive object/robot transition error and whether a fixed rollout scorer ranks the candidates in the same order as executed outcomes. Include short and longer horizons. If the predictor produces nearly identical goal-completing futures despite interventions that change actual outcomes, it fails the causal-conditioning requirement even if its videos remain visually convincing. Data, checkpoints and resettable environments would need to be supplied for this check. e03e13e17e21
Check 2: Reconstruct the compiled comparisons without merging protocols
Reader-proposed evidence check, not performed: independently transcribe the Cosmos Policy, LingBot-VA and Motus LIBERO rows and the LingBot-VA, Motus and BagelVLA RoboTwin rows, keeping each suite/setting separate. With separately supplied original experiment reports, verify paper version, benchmark split, training data, action representation, evaluation count and reported uncertainty for every cell. Use Table 5's directly reported average as the reference; any recomputed mean is only a rounding check. Preserve Table 6's missing entries and paired variants. The falsifiable outcome is whether the apparent ordering survives restriction to genuinely matched protocols; unresolved settings must remain incomparable, rather than being silently pooled. e19e20e23
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page and every body page, including all six figures, all six tables, equations, method distinctions, evaluation definitions and limitations. Cross-checked the selected diagram flows against their captions and relevant equations/text. Inspected all six final original-PDF crops; table footnotes and diagram legends are retained. PDF pp. 32–43 were read in full as reference text but are outside the image pass. No appendix is present; separate supplement availability remains unverified.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors and abstract (p. 1)
- 1 Introduction (pp. 1–4)
- 2 Background, including 2.1.1–2.2.2 (pp. 4–7)
- 3 World Model for Policy, 3.1–3.6 (pp. 7–15)
- 4 World Model as Simulator, 4.1–4.2 (pp. 15–18)
- 5 World Model for Robotic Video Generation, 5.1–5.6 (pp. 19–23)
- 6 Navigation and Autonomous Driving, 6.1–6.2 (pp. 23–24)
- 7 Benchmarks, Datasets, and Results, including 7.1.3 diagnostics (pp. 24–30)
- 8 Challenges and Future Directions, 8.1–8.6 (pp. 30–31)
- References (pp. 32–43)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The verified edition is arXiv:2605.00080v1, dated 30 April 2026. Title and all 18 authors match the catalog; no different revision was supplied or compared.
- Text extraction does not reconstruct figure images; this limitation was addressed by inspecting the retained PDF's body pages, figures and tables.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Cited papers, linked repositories and resource websites were not independently inspected. Results below are verified as the survey's compilation, not independently verified against the cited experiments. No experiments were reproduced.
- Reference pages were read in full as text; visual inspection covered PDF pp. 1–31.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block and arXiv margin
The observed title and ordered 18-author list match the catalog. The PDF identifies arXiv:2605.00080v1, 30 April 2026, and lists nine institutions.
Go to primary source ↓e02PDF pp. 1–4, Section 1; Figures 1–2 and contribution list
The work surveys policy architectures, simulator applications and video-model capabilities. The timeline describes overlapping directions, not successive replacements.
Go to primary source ↓e03PDF pp. 4–5, Sections 2.1.1–2.1.2, Eqs. (1)–(2)
World modeling predicts future states under physical actions and/or high-level instructions; state may be visual, latent, structured or symbolic. Visual plausibility alone is insufficient.
Go to primary source ↓e04PDF pp. 6–7, Section 2.2, Eq. (3); Section 3.1, Eqs. (4)–(8)
The survey relates policy, passive prediction, controllable prediction and inverse dynamics through an idealized joint future-observation/action distribution. Here k is the action-chunk length.
Go to primary source ↓e05PDF p. 8, Table 1, all paradigm groups and inference column
Five policy families are compared by backbone, coupling and inference-time future generation. Fast-WAM skips test-time video; joint co-training and shared-backbone coupling are distinct labels.
Go to primary source ↓e06PDF pp. 9–10, Section 3.2, Eqs. (9)–(12); Figure 3(a) and caption
A separate predictor supplies future observations or representations to an action policy. The predictor is typically pretrained, then frozen or lightly adapted; plausible but uncontrollable futures can propagate error.
Go to primary source ↓e07PDF pp. 10–11, Section 3.3, Figure 3(b), Eqs. (13)–(14)
Shared backbones jointly model visual and action representations. Targets can be noise, velocity or masked tokens. Cosmos Policy separates direct action use from state/value-based planning; visual branches need not always run online.
Go to primary source ↓e08PDF pp. 10 and 12–13, Figure 3(c); Section 3.4, Eq. (15)
Expert architectures retain modality-specific parameterization with repeated attention-based interaction. Fast-WAM is discussed as emphasizing video co-training over explicit test-time imagination.
Go to primary source ↓e09PDF pp. 13–14, Figure 4 and caption; Section 3.5
Unified VLAs combine actions with future-oriented supervision; WorldVLA uses future-image prediction mainly during training. Some unified systems retain specialized experts.
Go to primary source ↓e10PDF pp. 14–15, Section 3.6
FLARE aligns action-network features with future embeddings; VLA-JEPA uses future frames only as supervision targets. Latent prediction avoids image decoding; symbolic transitions are a complementary planner interface.
Go to primary source ↓e11PDF pp. 15–16, Section 4 introduction; Figure 5 and caption; Section 4.1, Eqs. (16)–(18)
Learned simulators produce imagined transitions and optional reward/termination signals for policy optimization. Figure 5 contrasts policy updates with candidate scoring; its scores are schematic.
Go to primary source ↓e12PDF p. 17, Section 4.1, Eq. (19) and surrounding discussion
Policy failures or real interaction can refine a world model, whose improved imagined data then support policy updates; simulator reliability is part of the learning loop.
Go to primary source ↓e13PDF pp. 17–18, Section 4.2
World models rank or revise candidate actions, support planning, and compare policies/checkpoints. Hallucinations and long-horizon errors can corrupt evaluator signals.
Go to primary source ↓e14PDF pp. 19–20, Figure 6 and caption; Sections 5.1–5.2 and Table 2
Video-model capabilities span imagination-based supervision, action conditioning, structure-aware prediction and foundation-model reuse. The figure is an organizing schematic, not an experimentally validated pipeline.
Go to primary source ↓e15PDF pp. 21–23, Sections 5.3–5.6
Action alignment and interaction structure are emphasized. IRASim conditions individual frames on actions; structure includes masks, geometry and multi-view cues. Foundation backbones are adapted for interactive rollout and data generation.
Go to primary source ↓e16PDF pp. 23–24, Sections 6.1–6.2
Navigation uses predicted unseen views and planning states; driving adds multi-agent dynamics, geometry, occupancy/latent representations and predictive supervision.
Go to primary source ↓e17PDF pp. 24–26, Sections 7.1.1–7.1.3
Evaluation separates open-loop prediction, closed-loop control/policy ranking, and physical or executability diagnostics. WorldEval checks policy ordering; WorldSimBench includes inverse-dynamics recovery.
Go to primary source ↓e18PDF pp. 26–28, Tables 3–4 and Section 7.2
Datasets are compared on overlapping supervision and capability axes. Checkmarks, dashes and crosses are qualitative grades; failure recovery and dense physical supervision are described as scarce.
Go to primary source ↓e19PDF p. 28, Table 5, Cosmos Policy, LingBot-VA, Say-Dream-ACT, Motus and VLA-JEPA rows; header and footnotes
Compiled LIBERO averages are respectively 98.5, 98.5, 98.1, 97.7 and 97.2. Cosmos Policy has 98.1/100.0/98.2/97.6 across Spatial/Object/Goal/Long; LingBot-VA has 98.5/99.6/97.2/98.5. Dashes denote unreported standard-protocol results.
Go to primary source ↓e20PDF p. 29, Table 6, LingBot-VA, Motus and BagelVLA rows; all column footnotes; Section 7.3
RoboTwin RT-A/RT-B values are 92.9/91.6, 88.7/87.0 and 75.3/20.9 respectively. RT-A is non-randomized and RT-B randomized. CALVIN labels print ABCD/ABCDD; paired SIMPLER entries denote unspecified original protocol variants. Strict ranking is cautioned against.
Go to primary source ↓e21PDF p. 30, Sections 8.1–8.2
Weak conditioning may follow intent rather than pending actions. Video denoising creates latency; partial denoising, latent modeling and training-only prediction are discussed as efficiency routes.
Go to primary source ↓e22PDF pp. 30–31, Sections 8.3–8.5
Open issues include force/tactile sensing, asynchronous modality alignment, MPC cost, formal control guarantees and grounded symbolic abstractions.
Go to primary source ↓e23PDF pp. 28–31, Section 7.3 and Tables 5–6; Section 8.6
Results are compiled from other works without a common training-budget, seed or uncertainty specification. No new controlled ablation is presented; evaluation remains multidimensional and protocol-dependent.
Go to primary source ↓8.5 Primary sources
World Model for Robot Learning: A Comprehensive Survey ↗
PDF · 25,976 extracted words
Source fingerprint
1b035d79d5c135fd437e1bbe7625a7e284a405e0451a240e816ce5dcaf35f586