Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Xiaokang Liu; Zechen Bai; Hai Ci; Kevin Yuchen Ma; Mike Zheng Shou

Affiliations: Show Lab, National University of Singapore

Source: 2602.06508 ↗ · Catalog record

Reading: 265 / 558 · 6 original figures & tables · ~19 min ·

1. Paper overview

In one sentence: Learning subtle failures and a shared video–reward representation makes a video simulator more useful for VLA reinforcement learning, while policy-driven data refresh addresses new simulator exploits at the cost of further target-environment interaction. e-identitye-failuree-methode-loope-policy

At a glanceWhat to know
Research problem
Source description

A realistic video can still give the wrong answer about whether a grasp succeeds. The authors identify near-success errors—small pose mistakes that cause real failure but simulated success—as a threat to policy reinforcement learning. Physical rollouts and resets are expensive, yet inaccurate simulated observations and rewards can train exploitable behavior. e-probleme-failure

Core mechanism
Source description

SANS combines successful and nearly successful executions with action annotations and binary success labels, targeting the outcome boundary missing from success-oriented imitation data. e-data

A key reported resultPhysical cup placement and cube pushing: First RL round: Place Cup 36.7%; Push Cube 40.0%.

Physical task success rate. Franka setup; Table 3 reports 30 physical rollouts; object positions randomized.

SFT: 13.3% and 26.7%; gains 23.4 and 13.3 percentage points. These are executed robot outcomes. Figure 4's real-world-labeled training curves instead measure success inside the video simulator. e-policye-setup

Reading caution
Source description

Limited context and autoregressive drift constrain long-horizon use. Table 5 degrades from PSNR/SSIM 23.5890/0.7430 at 200 frames to 20.2351/0.6282 at 300. Section 4.3 defers LIBERO-100, while Appendix B.1 discusses LIBERO-Long; the report preserves these distinct labels. e-horizone-protocole-limitations

Core contributions

  • Source description

    SANS combines successful and nearly successful executions with action annotations and binary success labels, targeting the outcome boundary missing from success-oriented imitation data. e-data

  • Author claim

    A reward MLP shares denoised latents with video prediction. The authors argue that reward supervision improves action following as well as supplying an RL learning signal. e-methode-ablation

  • Source description

    An outer data-collection loop updates the simulator for mistakes made by an improved policy; an inner simulated rollout loop supplies observations and rewards for GRPO policy training. e-pipelinee-loope-iteration

Figure 3. Two feedback loops connect outcome prediction, policy optimization and fresh execution data. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start at the upper-left SANS box and follow the training-data arrow into the diffusion model. Observation and action conditioning enter the DiT; denoised latents feed both video decoding and reward prediction. The flames mark training of the generator and reward head. Now move to the lower-right box: the upper curved arrow carries action chunks from VLA to world model, while the lower arrow returns visual observations and rewards. Finally, the deployment arrow moves left to actual target-environment execution, whose trajectories return up to SANS. These directions agree with the Figure 3 caption and Sections 3.2–3.3. e-pipelinee-methode-loope-iteration

What it supports. There are two learned systems with different jobs. The simulator predicts the consequences of supplied actions and supplies the RL reward; the VLA chooses actions. Video and reward supervision share the simulator's latents, while real or physics-simulator executions refresh its training distribution between rounds.

Where the evidence stops. The loop diagram does not specify continuous checkpoint warm-starting. The reported iterative experiment resets world models to ManiSkill pretraining and policies to base SFT for each stage. Physical data collection remains necessary for the outer loop.

2. Motivation

2.1 The problem and the proposed response

Source description

A realistic video can still give the wrong answer about whether a grasp succeeds. The authors identify near-success errors—small pose mistakes that cause real failure but simulated success—as a threat to policy reinforcement learning. Physical rollouts and resets are expensive, yet inaccurate simulated observations and rewards can train exploitable behavior. e-probleme-failure

2.2 What this reading follows

Imagine a gripper missing a cup by a small offset while its video simulator confidently depicts a successful grasp. A policy trained against that simulator can learn the wrong lesson. World-VLA-Loop addresses this with successful and near-success demonstrations, a reward head trained on the generator's latents, and repeated collection of errors made by improved policies. This reading follows the supplied May 2026 v2 paper from action-conditioned generation to actual robot evaluation. Its central distinction is between predicting a successful outcome and executing one: the simulator's alignment scores, its training curves, and the physical robot's success rates answer different questions. e-identitye-failuree-methode-loope-policy

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryWAMs
ArchitectureDual-system
Prediction paradigmOther mechanisms
QuadrantOutside quadrants

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

The architecture supports Dual-system: a Cosmos-based action-conditioned simulator and an OpenVLA-OFT action policy remain distinct modules. Shared video/reward prediction happens inside the simulator, not as joint future/action prediction in a single policy. GRPO post-training through generated transitions supports Policy post-training & WM-RL and the recorded Other mechanisms / Outside quadrants classification. e-pipelinee-methode-loope-training

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • World model: observed frames and future action chunks containing 6-DoF end-effector poses plus gripper state (e-method).
  • VLA: visual observations and a task instruction; real tasks specify cup placement or cube pushing (e-loop, e-setup).
  • World model: autoregressive future video frames and scalar success-reward predictions (e-method).
  • VLA: action chunks executed in the learned simulator during RL and in the target environment during evaluation/data collection (e-loop).

4.2 Equations and their role

r^t=ϕ(zt),L=Lflow+λ∑t=1T∥r^t−rt∥2\hat r_t=\phi(z_t),\qquad \mathcal L=\mathcal L_{\mathrm{flow}}+\lambda\sum_{t=1}^{T}\lVert\hat r_t-r_t\rVert^2
The paper defines z_t as the final denoised latent for timestep t, phi as the reward MLP, r_t as the ground-truth reward and T as the action horizon. The flow-matching term trains generation; squared error supervises reward prediction. Lambda varies with sampled noise level following EDM, but its explicit schedule is absent. e-method

5. Method in detail

5.1 Teach the simulator the difference between a near grasp and a grasp

Reader analysis

SANS is motivated by errors that look almost correct. In ManiSkill, the authors can use object poses to generate successful actions and perturb those poses to produce nearby failures. Policy rollouts add naturally occurring mistakes; real-world collection also uses teleoperation. Each trajectory contains video, actions and binary success labels. This matters because action-conditioned generation must respond to the action actually supplied, even when its pretrained visual prior favors a familiar successful manipulation. The source's Figure 2 demonstrates the problematic alternative: an offset grasp fails in execution but appears successful in generated video. The resulting target is outcome-sensitive simulation. Reader interpretation: this resembles learning a local decision boundary around contact success, although the paper does not measure that boundary geometrically or compare all possible failure-sampling strategies. e-datae-failuree-method

Figure 5. The policy's new mistakes become data for the next simulator-training round. Original paper, p. 9 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow the blue arrows from the upper-left SFT rollout downward. Its imprecise pushing direction supplies first-round training data. The orange-labeled row is world-model generation; the blue-labeled row beneath it is the trained policy's execution, annotated as reward hacking and unstable behavior. The long arrow carries those collected rollouts into the second iteration on the right. That simulator now depicts both successful and failed trajectories, and the final blue row shows the second policy's more stable push. The source caption and Section 4.5 confirm the generation/execution distinction; the arrows connect data collection to training rather than showing gradients through the robot. e-qualitativee-iteration

What it supports. The example explains why an initially useful simulator may become inadequate after policy optimization. The policy encounters or exploits behavior absent from earlier data. Augmenting SANS with that behavior lets the next world model represent previously missed failures, supporting another round of policy improvement.

Where the evidence stops. These selected frames illustrate the authors' account of reward hacking; they do not measure its prevalence or isolate simulator refresh from added data. Section 4.5 refers to improved grasping poses, whereas this figure's labels and visible task concern pushing.

5.2 Use one latent representation for video and success prediction

Source description

The action embedder maps the future control sequence into features added to diffusion timestamp embeddings. The DiT then generates denoised video latents under observation and action conditioning. A decoder maps those latents to visible frames, and a lightweight MLP maps them to scalar rewards. The loss combines flow matching with squared error against the binary success labels, weighted according to sampled noise level. Thus a binary target does not imply a binary-valued raw head: the generated scalar is subsequently thresholded, with 0.9 specified for reward-alignment evaluation. During training, reward supervision updates the generator alongside the head. During simulated RL rollouts, the head supplies the learning signal. Table 4 is the mechanism-relevant test because removing reward supervision also changes visual outcome agreement. e-methode-alignmente-ablatione-loop

5.3 Separate simulated feedback from the outer execution loop

Reader analysis

Inside policy RL, an original initial image starts the episode, after which the video model supplies observations and rewards in response to VLA action chunks. GRPO compares groups of eight rollouts; this feedback can improve the policy without executing every training episode on the robot. However, the updated policy changes which actions the simulator must handle. The outer loop therefore collects actual target-environment executions and augments SANS before another simulator-training round. The reported experiment initializes each policy stage from the same base SFT version and each world model from ManiSkill pretraining. Reader interpretation: the improvement tests a refreshed training-data pipeline more directly than uninterrupted policy self-improvement. Table 3 verifies physical gains after the first round, while Figure 5 illustrates the new failures that motivate another round. e-protocole-traininge-iteratione-policye-qualitative

5.4 Training and inference

During training

Source description

World-model training jointly optimizes flow matching and squared reward error. Full-parameter tuning uses FusedAdam, learning rates 1e−3 for ManiSkill pretraining and 1e−5 for downstream adaptation. The experimental protocol specifies 80–100 mixed trajectories for simulator adaptation and approximately 50 successful trajectories for policy SFT. e-methode-traininge-protocol

Source description

The policy pipeline follows SimpleVLA-RL, with GRPO group size 8 and chunk size 24. A request-response server allocates action chunks to world-model workers and returns observations and rewards. Other RL hyperparameters are referred to SimpleVLA-RL rather than enumerated here. e-training

During inference

Source description

At simulator evaluation, an initial image and absolute end-effector action sequence generate the corresponding video. Reward alignment classifies success when the predicted reward exceeds 0.9. Physical evaluation deploys the trained VLA; no online search through imagined candidate actions is specified. e-evaluatione-alignmente-policy

5.5 Implementation flow

  1. Collect outcome-sensitive data

    ManiSkill successes use object-pose control, with pose perturbations and policy rollouts supplying failures. LIBERO uses OpenVLA-OFT rollouts; real data combines teleoperation and policy executions. Pretraining contains 35k video-action pairs over 23 tasks. Target adaptation uses roughly 50 successes and 50 failures per task. e-data

  2. Predict consequences of supplied actions

    Cosmos-Predict 2 provides the diffusion-transformer backbone. An MLP embeds actions and adds them to diffusion timestamp embeddings. Observed images condition denoising; a decoder produces video, while a lightweight reward head reads denoised latents. The simulator predicts outcomes of actions supplied by the policy; it does not infer actions from desired video. e-methode-pipeline

  3. Use the simulator for policy optimization

    Only the initial image comes from the original dataset during RL. Subsequent observations are generated recursively from policy actions. Thresholded success rewards yield group-relative advantages for GRPO. The simulator is an inference-time environment during policy training, rather than an auxiliary prediction loss attached to the VLA. e-loope-protocol

  4. Refresh the outcome distribution

    Deploy the refined policy in its target environment and append new success/failure executions. In the demonstrated two-stage experiment, each world model starts from the ManiSkill-pretrained checkpoint and each policy RL stage starts from the base SFT checkpoint; this is not simply continuous checkpoint-to-checkpoint policy training. e-iteration

6. Experiments & results

World-VLA-Loop trains an action-conditioned video simulator to predict both future images and success rewards, then uses that simulator to improve a separate OpenVLA-OFT policy. Success and near-success demonstrations teach the simulator about subtle failures; fresh policy executions expand that data between training rounds. Two physical tasks improve, but simulator success remains distinct from execution success, and long rollouts degrade (e-method, e-loop, e-policy, e-iteration, e-horizon).

6.1 Read the original evidence

Table 2. Outcome agreement tests whether a predicted success or failure matches the execution label. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read each column vertically before comparing tasks. Visual Alignment classifies the final generated frames as success or failure and compares that judgment with ground truth. Reward Alignment instead uses the simulator's scalar reward, with success defined by exceeding 0.9. The caption reports 50 samples for each task. Across the eight columns, the averages are 88.5% for visual agreement and 87.25% for reward agreement. The real-world columns are especially useful: the reward head agrees more often than visual judgment on Place Cup, but less often on Push Cube. Neither row reports how frequently the robot succeeds. e-alignmente-evaluation

What it supports. The head usually agrees with execution outcomes, but error rates remain task dependent. For Push Cube, reward agreement is 78%, compared with 94% for Place Cup. A simulator can therefore support useful learning without being a uniformly reliable success oracle across tasks.

Where the evidence stops. The caption says samples are evaluated within set; it does not establish a held-out split or class balance. Similar aggregate row values do not reveal whether the two predictors fail on the same samples, and no uncertainty intervals are shown.

Table 3. The first RL round improves executed performance, with separate evaluation scopes for LIBERO and physical tasks. Original paper, p. 8 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Compare SFT Base with RL Post-Training (Ours) within a column. The highlighted delta row uses percent signs, but subtraction of success rates gives percentage-point gains. The caption specifies 500 rollouts for the LIBERO suites and 30 physical rollouts for the real-world experiments. The two rightmost columns therefore provide direct robot-execution evidence: Place Cup improves by 23.4 points and Push Cube by 13.3 points. Read the oracle row separately: its repeated entries are averages over entire task suites, obtained with ground-truth simulator observations and rewards, rather than the individual-task protocol used above. e-policye-iteration

What it supports. Training in the learned simulator transfers to better performance in the target environments. Nevertheless, the first-round physical rates are 36.7% and 40.0%, leaving substantial failure rates. Figure 1(b) and Section 4.3 later report 50.0% and 53.3% after the augmented-data iteration; those later values are outside this table.

Where the evidence stops. Figure 4's real-world-labeled curves on the same page are evaluated inside the learned simulator, unlike these physical results. No seed dispersion or confidence intervals are supplied, and the oracle row cannot establish a matched per-task superiority claim.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
World-model outcome agreement

Six LIBERO tasks and two real-world tasks, 50 samples per task; the caption says within set without establishing a held-out split.

88.5% / 87.25% average

Visual alignment / reward alignment

Place Cup: 90% / 94%; Push Cube: 84% / 78%.

Agreement with ground-truth success/failure is distinct from policy success or complete trajectory accuracy. Visual outcomes are judged from final generated frames. e-evaluatione-alignment

LIBERO policy post-training

Two reported tasks each from Object, Goal and Spatial; Table 3 describes 500 rollouts for LIBERO suites.

Object: 97.9%, 91.9%; Goal: 100%, 96.2%; Spatial: 93.9%, 94.0%.

Task success rate

SFT: 73.9%, 73.9%; 91.9%, 86.1%; 83.9%, 87.9%, respectively.

The six gains average approximately 12.7 percentage points. Oracle results are entire-suite averages, not matched per-task comparators; task identities and run uncertainty are unspecified. e-policy

Physical cup placement and cube pushing

Franka setup; Table 3 reports 30 physical rollouts; object positions randomized.

First RL round: Place Cup 36.7%; Push Cube 40.0%.

Physical task success rate

SFT: 13.3% and 26.7%; gains 23.4 and 13.3 percentage points.

These are executed robot outcomes. Figure 4's real-world-labeled training curves instead measure success inside the video simulator. e-policye-setup

Physical performance after iterative refinement

Two outer-loop RL stages with augmented rollout data and the checkpoint resets described in Section 4.3.

Place Cup 50.0%; Push Cube 53.3%.

Physical task success rate

First-round 36.7% and 40.0%; approximately 13.3 percentage points further improvement on each task.

Supports the complete augmented-data pipeline. Additional data and simulator refresh are not separated by a matched-budget control; stage-specific trial details are not separately tabulated. e-iteratione-policy

Near-success data and reward-head ablations

Table 4: LIBERO-Object Task 1, Task 2, and an unnamed real-world column.

Full model: 92%, 90%, 90%.

Visual outcome alignment

Without near-success data: 60%, 66%, 54%; without reward head: 68%, 70%, 80%.

Reward-head removal reduces alignment by 24, 20 and 10 percentage points. These exact differences qualify the prose's approximately 30% description. Matched data volume and seed variability are not reported. e-ablation

External VLM reward replacement

Appendix B.2, LIBERO-Object; generated frames judged by the model named Qwen3.5-VL-Instruct 8B.

Integrated reward: 97.9%, 91.9%.

Policy success rate

VLM reward: 93.9%, 88.0%.

A downstream reward-source comparison. Table 4 instead names Qwen3-VL-8B-Instruct and measures reward alignment; these are different reported comparisons. e-vlme-ablation

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 4. The two proposed ingredients both affect outcome agreement in the reported ablations. Original paper, p. 8 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Treat the upper and lower blocks as different comparisons. The upper block changes world-model training: remove near-success data, remove the reward head, or retain both. In Object Task 1, alignment moves from 60% without near-success data and 68% without the head to 92% with the full method. The lower block instead compares reward prediction sources: Qwen3-VL-8B-Instruct judges the generated video, while the internal head predicts from its latents. Section 4.4 says the VLM receives the generated frames associated with those latent predictions. The rightmost column is labeled only Real-World; no task name is given. e-ablatione-methode-vlm

What it supports. Reward supervision appears to change what the simulator depicts, not merely how its outputs are scored. Removing the head reduces visual alignment by 24, 20 and 10 percentage points across the columns. Removing near-success data produces still larger losses in these tests, supporting attention to the training outcome distribution.

Where the evidence stops. The prose's approximately 30% reduction is broader than the exact row differences. Data-size matching and seed variability are unspecified. Appendix B.2 uses a differently named Qwen3.5 model for downstream RL, so it should not be merged with this alignment comparison.

Table 5. Longer autoregressive rollouts progressively lose visual quality in the appendix diagnostic. Original paper, p. 14 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read downward along either metric while increasing the frame count. Both PSNR and SSIM decline at each reported length, so the table indicates gradual degradation within the measured range. Appendix B.1 connects this issue to longer manipulation episodes: it describes the usual Object, Spatial and Goal tasks as around 220 frames and LIBERO-Long as 300–400 frames. Because generated frames become subsequent policy observations, this diagnostic matters for control even though its measurements concern video quality. The table does not show policy rewards or physical success, and it does not report a row beyond 300 frames. e-horizone-limitationse-protocole-loop

What it supports. The approach's demonstrated utility has a temporal boundary. PSNR falls from 23.5890 at 200 frames to 20.2351 at 300, while SSIM falls from 0.7430 to 0.6282. These values support the reported drift limitation, rather than unrestricted long-horizon simulation quality.

Where the evidence stops. The accompanying text says severe degradation typically occurs after 300 frames, but these rows alone cannot establish a sharp threshold. The appendix does not specify this diagnostic's sample count or split; Section 4.3 separately defers LIBERO-100 evaluation.

7. Analysis & limitations

7.1 What the evidence leaves open

Source description

Limited context and autoregressive drift constrain long-horizon use. Table 5 degrades from PSNR/SSIM 23.5890/0.7430 at 200 frames to 20.2351/0.6282 at 300. Section 4.3 defers LIBERO-100, while Appendix B.1 discusses LIBERO-Long; the report preserves these distinct labels. e-horizone-protocole-limitations

Source description

Binary success supervision is sparse despite step-wise reward outputs; intermediate sub-goal rewards are future work. Figure 5 documents reward hacking after the first RL round, so a shared latent reward head does not eliminate simulator exploitation. e-limitationse-qualitative

Reader analysis

Two physical tasks, small evaluation samples and absent uncertainty estimates limit generalization claims. Appendix C's unseen-action examples are qualitative; they do not quantify out-of-distribution execution success. Data-collection costs remain part of the outer loop. e-policye-iteratione-unseen

7.2 Questions for discussion

  1. Does a matched-size mixture of ordinary failures perform as well as near-success failures?
  2. Does simulator refresh still help when additional data volume, policy initialization and physical interaction budget are controlled?
  3. How do false-success rewards change as rollout horizon grows?

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Reported costs: 24 generated frames take about 6 seconds on a single H100 node; task RL takes 20–40 optimization steps and 25–35 hours. Adaptation with 100 trajectories takes about 4 hours on 8 H100 GPUs. Inference batch size and node GPU count are unspecified. e-compute

Reader analysis

Reproduction needs synchronized SANS videos/actions/rewards, exact target-task splits, checkpoints and the referenced RL configuration. The PDF omits the reward-head dimensions, explicit noise-weight schedule, detailed denoising settings, exact LIBERO task IDs and VLM prompts. Section 4.3 calls the initial dataset SUPA once; its relationship to SANS is not explained. e-datae-methode-traininge-policye-iteratione-vlm

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Separate near-success sampling from reward supervision

Reader-proposed check, not performed: on one identified LIBERO-Object task, cross two equally sized failure-data mixtures—near-success failures versus other policy failures—with joint reward/video training enabled or disabled. Hold the success data, checkpoint, tuning budget and test episodes fixed. Evaluate final-frame outcome agreement and false-success frequency on held-out small pose perturbations; where a reward head is trained, also evaluate its greater-than-0.9 decision. Report multiple seeds. If matched near-success data yields no advantage over other failures, or reward supervision fails to improve visual agreement, the corresponding mechanism claim would be weakened without confounding it with data volume. e-datae-methode-ablatione-alignmente-training

Check 2: Test whether new-policy data specifically repairs simulator exploits

Reader-proposed check, not performed: compare a fixed simulator, a simulator retrained on an equal number of new base-policy executions, and one retrained on executions from the first RL policy. Match training steps, success/failure counts and target-environment interaction budget; reset each next-stage VLA to the same SFT checkpoint as in the paper. Measure outcome agreement on held-out first-RL failure cases, predicted-versus-executed success, and final physical success under matched initial states. A specific benefit from new-policy data should reduce false simulated successes on those failures and improve executed performance; improvement only inside simulation would not validate the proposed repair. e-iteratione-qualitativee-policye-setupe-alignment

8.3 Reading coverage

Visual audit: Visually inspected the title/authors/version on PDF page 1; all Figures 1–9 and Tables 1–6; method and loss on pages 4–6; evaluation definitions and protocol on pages 6–8; iterative refinement and qualitative claims on pages 8–9; limitations on page 10; implementation, hardware, task setup and cost on pages 13–14; and appendix generation examples on pages 15–16. All six final original crops were viewed, including the corrected architecture crop. Figure 3's action, observation/reward and rollout arrows were checked against its caption and Sections 3.2–3.3. Figure 5's generation/execution labels were checked against its caption and Section 4.5. The report preserves Table 4's exact reductions, the differing VLM names, simulator-versus-physical evaluation scopes, and the pushing/grasping wording discrepancy. Reference-only pages 11–12 were read in the complete text but not visually inspected. Separate supplements and code are outside this review.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Abstract
  • 1 Introduction
  • 2 Related Work
  • 3 Method
  • 3.1 Success and Near-Success Dataset
  • 3.2 State-aware Video World Simulator
  • 3.3 World Simulator for Policy RL Post-Training
  • 4 Experiments
  • 4.1 Experiment Settings
  • 4.2 World Model Evaluation
  • 4.3 World Simulator for VLA Post-training
  • 4.4 Ablation Study
  • 4.5 Qualitative Results
  • 5 Conclusion, including Limitations and Future Work
  • References
  • A.1 World Model Training
  • A.2 VLA Deployment in World Model
  • A.3 Real-World Experimental Settings
  • A.4 Computation Cost
  • B.1 Quality Analysis on Long-Horizon Tasks
  • B.2 VLM Reward for RL
  • C.1 Additional World Model Generation Results
  • C.2 World Model in Unseen Cases

Outside the original text pass

  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • The supplied artifact is arXiv:2602.06508v2 [cs.RO], dated 25 May 2026. Its title and all five authors match the catalog. The catalog submission date is 6 February 2026; the original version was not supplied, so changes between versions were not verified.
  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by visually inspecting the supplied PDF, including every figure and table.
  • Separate supplemental material availability has not been fully verified.
  • All five supplied text chunks were read in full, including references and Appendices A–C. Reference-only PDF pages 11–12 were read as text but not rendered; all other PDF pages were visually inspected. Appendix printed page numbering restarts at 1 on PDF page 13.
  • Project links, code, datasets and separate supplements were not inspected; no experiments were reproduced.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e-identityPDF p. 1, title block and arXiv margin stampInspect

Exact catalog title and five authors; Show Lab, National University of Singapore; arXiv:2602.06508v2 [cs.RO], 25 May 2026. Xiaokang Liu and Zechen Bai have equal-contribution markers; Mike Zheng Shou has the correspondence marker.

Go to primary source ↓
e-problemPDF pp. 1–2, Abstract and Section 1Inspect

Near-success action errors, unreliable video-derived rewards, physical RL costs and policy-induced distribution shift motivate the method.

Go to primary source ↓
e-failurePDF p. 3, Figure 2 and captionInspect

Illustrated generated grasp success contrasts with executed failure under a gripper offset; transparent overlays denote ground-truth gripper trajectories.

Go to primary source ↓
e-pipelinePDF p. 4, Figure 3 and captionInspect

Four phases: SANS collection, video/reward world-model training, simulated GRPO policy training, and target-environment deployment feeding new video-action pairs back to SANS. Arrows send policy actions to the world model and observations/rewards back.

Go to primary source ↓
e-dataPDF pp. 4–5, Section 3.1Inspect

SANS mixes successful and near-success trajectories; ManiSkill pretraining has 35k video-action pairs across 23 tasks; target datasets have roughly 50 success and 50 failure trajectories per task with step-wise binary success labels.

Go to primary source ↓
e-methodPDF p. 5, Section 3.2, action embedder, reward-head definition and unnumbered lossInspect

Cosmos-Predict 2 DiT conditions on observed frames and end-effector/gripper actions, adds action embeddings to timestamp embeddings, and predicts rewards with an MLP from denoised latents. Joint flow-matching and squared reward error use a noise-modulated lambda.

Go to primary source ↓
e-loopPDF p. 6, Section 3.3Inspect

OpenVLA-OFT is optimized using GRPO and generated observations/rewards; target-environment executions of updated policies augment subsequent simulator-training data.

Go to primary source ↓
e-evaluationPDF p. 6, Sections 4, 4.1 and 4.2Inspect

Evaluation uses a Franka setup, fixed third-person RealSense D435 observation and chunk size 24. Initial-image/absolute-action input drives video evaluation. The experiment overview reports average visual alignment 88.5% and reward alignment 87.25%.

Go to primary source ↓
e-videoPDF p. 7, Table 1Inspect

LIBERO SSIM/PSNR/LPIPS/MSE are 0.90/26.57/0.031/0.0024; real-world values are 0.91/29.61/0.059/0.0019. These describe video reconstruction quality.

Go to primary source ↓
e-alignmentPDF p. 7, Table 2 and Section 4.2 Generation AccuracyInspect

Fifty samples per task; visual alignment is final-frame outcome agreement, reward alignment thresholds at greater than 0.9. Visual percentages are 92,90,94,78,86,94,90,84; reward percentages are 88,90,90,76,88,94,94,78.

Go to primary source ↓
e-protocolPDF p. 7, Section 4.3 Basic RL TrainingInspect

Only the initial frame is sourced from original data in RL; later observations are generated. Simulator adaptation uses 80–100 mixed trajectories and policy SFT about 50 successes. LIBERO-100 is deferred because of long-horizon video drift.

Go to primary source ↓
e-policyPDF p. 8, Table 3 including caption; Figure 4 and following clarificationInspect

Table 3 gives six task-specific LIBERO results and two physical-task results before/after RL, describing 500 LIBERO rollouts and 30 physical rollouts. Oracle values are suite averages. Figure 4 real-world curves are evaluated in the learned simulator; Table 3 physical results are executed.

Go to primary source ↓
e-iterationPDF p. 1, Figure 1(b); PDF p. 8, Section 4.3 Iterative Refinement through Policy RolloutsInspect

SFT/first RL/iterative physical success: cup 13.3/36.7/50.0%, cube 26.7/40.0/53.3%. New first-round policy rollouts augment training. Both world models initialize from ManiSkill pretraining and each RL policy from base SFT. Initial data is called SUPA in this paragraph.

Go to primary source ↓
e-ablationPDF p. 8, Table 4 and Section 4.4; PDF p. 9, Section 4.4 continuationInspect

Visual alignment across Object-1/Object-2/Real-World is 60/66/54 without near-success, 68/70/80 without reward head, and 92/90/90 for full model. Reward alignment is 84/58/84 for Qwen3-VL-8B-Instruct and 88/90/94 internally. The VLM judges generated frames; prose describes an approximately 30% drop without reward supervision.

Go to primary source ↓
e-qualitativePDF p. 9, Figure 5, caption and Section 4.5Inspect

A pushing example contrasts SFT pose errors, first-round reward hacking, and second-round improvement after collecting and learning the newly encountered failures. The figure distinguishes generated rows from policy execution rows.

Go to primary source ↓
e-limitationsPDF p. 10, Limitations and Future WorkInspect

Authors identify limited context memory and quality drift beyond 300 frames, and propose intermediate sub-goal rewards beyond sparse final-state supervision.

Go to primary source ↓
e-trainingPDF p. 13 (appendix p. 1), Sections A.1–A.2Inspect

Action-conditioned Cosmos Predict 2 checkpoint; full-parameter tuning; FusedAdam; learning rates 1e−3 then 1e−5. GRPO groups contain 8 rollouts, chunk size is 24, other hyperparameters refer to SimpleVLA-RL. World-model workers serve observation/reward requests.

Go to primary source ↓
e-setupPDF p. 13 (appendix p. 1), Section A.3 and Figure 6Inspect

Real prompts ask to place the cup on the green plate and push the cube into the dustpan. Cup, cube and plate positions are randomized; precise grasp/push contact is needed.

Go to primary source ↓
e-computePDF pp. 13–14 (appendix pp. 1–2), Section A.4Inspect

Batched 24-frame generation takes about 6 seconds on a single NVIDIA H100 node; task RL typically takes 20–40 optimization steps and 25–35 hours. Fine-tuning with 100 trajectories takes about 4 hours on 8 H100 GPUs.

Go to primary source ↓
e-horizonPDF p. 14 (appendix p. 2), Section B.1 and Table 5Inspect

At 200/250/300 frames, PSNR is 23.5890/22.2148/20.2351 and SSIM is 0.7430/0.6731/0.6282. Text attributes long-task difficulty to drift, describes common suites as around 220 frames and LIBERO-Long as 300–400 frames.

Go to primary source ↓
e-vlmPDF p. 14 (appendix p. 2), Section B.2 and Table 6Inspect

VLM-reward RL names Qwen3.5-VL-Instruct 8B and yields 93.9/88.0% on Object-1/Object-2, versus integrated reward 97.9/91.9%. Generated frames are prompted for binary success. No comparative timing table is supplied.

Go to primary source ↓
e-unseenPDF pp. 14–16 (appendix pp. 2–4), Sections C.1–C.2 and Figures 7–9Inspect

Figures show generated failure and success trajectories and examples described as unseen target-task action sequences. Authors attribute novel-action adherence to ManiSkill pretraining; these pages do not report quantitative unseen-action execution evaluation.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.