VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model
1. Paper overview
In one sentence: VLAW turns real policy failures into better visual dynamics, then uses filtered imagined successes to improve a separate robot policy, with reliability limited by both model fidelity and reward selection. e-probleme-loope-filter-policye-wm-metricse-policy-resultse-limits
| At a glance | What to know |
|---|---|
| Research problem | Source description Real robot rollouts require resets and supervision. A simulator could multiply this experience, but demonstration-trained video models can depict success even when an action fails. VLAW targets this mismatch on contact-rich and deformable-object manipulation, where small physical errors can corrupt policy supervision. e-problem |
| Core mechanism | Source description An alternating procedure grounds a pretrained action-conditioned world model in successful and failed online rollouts, then uses its synthetic trajectories to post-train a separate VLA policy. e-loope-world-training |
| A key reported result | Real-robot success across five manipulation categories: Ours-2: stacking 92%, wiping 86%, open book 86%, scooping 92%, drawing 78%; mean 86.8%. Ours-1 mean: 74.4%. Task success rate and unweighted mean. DROID Franka Panda with Robotiq gripper; multitask training for two iterations. Appendix B states 50 evaluations per task, except DSRL with 10. No separately specified held-out evaluation split or repeated-seed uncertainty. Base 46.0%; DSRL 50.0%; Filtered BC-1 64.8%; Filtered BC-2 75.2%. Ours-2 exceeds Filtered BC-2 by 11.6 percentage points. These are real execution results. Table 2 implies a 40.8-point gain over base; the abstract instead claims 39.2 points, an unresolved discrepancy. Section 5.3 claims matched online rollout budgets, but Appendix B explicitly gives fewer DSRL evaluations. e-setupe-policy-resultse-identity |
| Reading caution | Source description Evaluation covers five task categories and world-model adaptation uses online data from those downstream tasks. Broader task coverage is explicitly future work. e-limits |
Core contributions
- Source description
An alternating procedure grounds a pretrained action-conditioned world model in successful and failed online rollouts, then uses its synthetic trajectories to post-train a separate VLA policy. e-loope-world-training
- Source description
Successful real and synthetic trajectories train the policy through flow matching; a fine-tuned video-language reward model filters imagined data. This avoids explicit action likelihoods and policy-gradient optimization. e-filter-policye-rl
- Reader analysis
The experiments connect improved interaction-outcome prediction to real-robot policy gains, with drawing-data and reward-threshold diagnostics. They do not establish general-purpose simulator accuracy. e-wm-metricse-policy-resultse-drawinge-reward
Figure 3. Two separate learned models exchange training data through an iterative loop. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin at the blue VLA block on the left. Its upper arrow collects real trajectories; the green arrow uses that dataset to fine-tune the world model. The lower arrow leads to synthetic trajectories. The caption and Section 3 specify the closed-loop interaction omitted from this compact arrow: the policy acts on predicted observations and the world model supplies the next observation. On the right, both datasets point toward a new policy update. Read the label “Advantage Weighted Regression” together with Equation (4): the implemented weights are binary trajectory-success indicators, with successful real and synthetic samples entering the flow-matching loss. e-loope-world-traininge-filter-policye-rl
What it supports. The important asymmetry is which data each learner receives. The world model learns from both success and failure, because it must predict either outcome. The policy is fitted only to successful trajectories. Iteration changes both the simulated experience available for training and the policy that collects the next real batch.
Where the evidence stops. This is a training-flow schematic, not a deployment planner or one unified predictor. Its advantage-weighted label refers to the approximate interpretation in Appendix A; it does not establish explicit advantage estimation or exact KL-regularized optimization.
2. Motivation
2.1 The problem and the proposed response
Real robot rollouts require resets and supervision. A simulator could multiply this experience, but demonstration-trained video models can depict success even when an action fails. VLAW targets this mismatch on contact-rich and deformable-object manipulation, where small physical errors can corrupt policy supervision. e-problem
2.2 What this reading follows
A robot can learn little from an imaginary success if its simulator has quietly removed the physical difficulty. VLAW addresses this by adapting a video world model to the policy’s own successful and failed attempts before generating more training experience. The policy then learns from successful real trajectories and imagined trajectories accepted by a video-language reward model. Read the evidence in three stages: whether the simulator predicts interaction outcomes more faithfully, whether the trained policy improves on the physical robot, and what the data and reward ablations reveal. The paper supports this loop on five manipulation categories, while leaving broader generalization and several evaluation details unresolved. e-probleme-loope-filter-policye-wm-metricse-policy-resultse-limits
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The separate VLA policy and action-conditioned video world model support Dual-system; the reward model is an additional evaluator. The mechanism is model-based policy post-training through filtered imagined experience. It is neither joint future/action prediction within one model nor inverse dynamics from a predicted future, supporting Other mechanisms and Outside quadrants. e-loope-world-traininge-filter-policy
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why failures belong in the simulator’s training set
Consider a scoop that never catches the intended contents. If a world model has mostly seen successful demonstrations, it may continue the action sequence with a successful transfer anyway. VLAW’s first intervention is to collect the current policy’s real attempts and preserve both outcomes in world-model training. Ctrl-World predicts future observations under the recorded action chunk, while DROID co-training regularizes adaptation to the smaller online dataset. Failure trajectories therefore teach the simulator what should remain unsuccessful; they are not discarded at this stage. Table 1 tests the resulting prediction change using identical recorded actions, and Figure 6 supplies selected visual comparisons. The method aims to make imagined supervision more faithful around the policy’s encountered states and actions. e-probleme-world-traininge-wm-metricse-replay
5.2 How imagined experience becomes a policy update
After adaptation, the policy and world model alternate from a real initial observation: propose an action, predict its visual consequence, then propose the next action from that prediction. The resulting trajectory is a candidate training example, not an action sequence already validated on hardware. Qwen3-VL-4B-Instruct scores the video against the instruction, and a strict yes-probability threshold selects successes. The policy then learns through ordinary flow-matching regression on accepted synthetic and successful real trajectories. Appendix A relates this procedure to advantage-weighted policy improvement using an approximate projection, but the implemented selector is binary. Reader interpretation: the simplicity moves much of the burden from optimization into the quality and coverage of accepted trajectories. That is why both world-model fidelity and reward errors matter. e-loope-filter-policye-rle-reward
5.3 Separate simulator evidence from physical improvement
The paper’s strongest argument requires two different measurements. Five-second action replay establishes whether predicted observations and interaction outcomes agree with reality under recorded actions. Table 2 then measures the trained policy’s physical task success, reaching an 86.8% mean after two rounds compared with 75.2% for Filtered BC-2. Neither measurement alone answers the other question. The drawing ablation adds evidence that both real successes and a larger synthetic pool help in the first round, while the reward matrices expose the cost of conservative filtering. Reader interpretation: these results support the overall data-generation loop, but do not isolate which improvement in prediction causes which gain in control. Unequal DSRL evaluation counts, unspecified splits, and the abstract/table discrepancy further limit precision. e-wm-metricse-policy-resultse-drawinge-rewarde-identity
5.4 Training and inference
During training
Warm-start π0.5 with 25 expert demonstrations per task category. Each of two iterations collects 50 real trajectories per category, fine-tunes the world model for 50,000 steps, generates 500 synthetic trajectories per task, and updates the policy for 2,000 steps at batch size 256. e-schedule
Qwen3-VL-4B-Instruct is fine-tuned on first-iteration labeled rollouts. Each reward input is a 16-frame video; training uses 200 steps and batch size 128. Synthetic success requires the probability of the yes token to exceed 0.8. e-filter-policye-schedulee-reward
The regularized-RL interpretation is approximate: Appendix A substitutes flow-matching regression for a likelihood-based projection, then motivates binary success weights using a discount approaching one and strongly penalized failures. The implementation does not estimate explicit per-transition advantages. e-rle-filter-policy
During inference
During synthetic collection, policy and world model interact autoregressively. During real rollouts, the policy’s actions execute on the robot and receive real observations. The described method uses imagination for post-training; it does not specify deployment-time world-model planning or inverse-dynamics action extraction. e-loope-filter-policy
5.5 Implementation flow
- Collect the policy’s own experience
Execute the current policy in the real environment. Label success when resetting the robot and append trajectories to a growing dataset. Failures remain useful world-model training examples, even though policy regression subsequently excludes them. e-loope-world-traininge-filter-policy
- Fit action-conditioned visual dynamics
Fine-tune Ctrl-World to denoise future observations conditioned on the current observation and action chunk. Mix online experience with original DROID data to regularize the update; the source describes visual prediction rather than an explicit geometric or force state. e-world-training
- Generate closed-loop imagination
Start from an observation sampled from a real trajectory. The policy samples an action using the instruction and predicted observation, the world model predicts the next observation, and that prediction feeds the next policy step. This produces complete synthetic trajectories. e-loop
- Filter and refit the policy
The reward model scores each trajectory against its instruction. Retain predicted successes and combine them with successful real trajectories for policy training. Repeat real collection, world-model updating, imagination, and policy updating. e-filter-policye-schedule
6. Experiments & results
VLAW improves a robot policy by first teaching its video world model what the policy actually does, including failures, then learning from reward-filtered imagined successes. Separate π0.5, Ctrl-World, and reward models form an iterative training pipeline. Real-robot results support useful synthetic supervision on five task categories, with unresolved reporting and evaluation details (e-loop, e-policy-results, e-identity).
6.1 Read the original evidence
Table 1. Online experience improves visual prediction and reduces falsely predicted interaction success. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read downward from pretrained Ctrl-World to expert-only adaptation and then adaptation with expert plus online rollouts. The first five columns evaluate 256 replayed clips, each five seconds long, using wrist-camera images. Arrows indicate that PSNR and SSIM should increase while LPIPS, FID and FVD should decrease. The right-hand group instead counts interaction outcomes in 50 clips: true positives, false negatives, true negatives and false positives. These are different evaluations, so do not combine their denominators. The final column is especially relevant to VLAW’s motivation: it counts imagined successful interactions where the real interaction failed. e-wm-metrics
What it supports. Relative to expert-only tuning, online adaptation changes FVD from 99.98 to 64.12 and false-positive interaction counts from 11 to 1. The improvement is not uniform across event categories: true positives fall from 28 to 26 and false negatives rise from 2 to 4. The evidence supports reduced optimism, with a small additional failure-detection tradeoff.
Where the evidence stops. Recorded actions are replayed here; this is not an executed-policy success benchmark. The pretrained row omits event counts, and the source does not identify a held-out split or uncertainty for these metrics.
Table 2. After two iterations, VLAW reaches 86.8% mean real-robot success. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The five task columns report success fractions, while Mean gives their unweighted average. Compare Ours-1 with Filtered BC-1, then Ours-2 with Filtered BC-2, to keep the iteration count aligned. Filtered BC uses successful real rollouts; VLAW also uses filtered synthetic experience. The table’s Wiping task corresponds to erasing marker strokes. Stacking is tied at 0.80 in the first round, so the gains are not strictly positive in every matched cell. Appendix B states 50 evaluations per task except 10 for DSRL; Section 5.3 separately states that online collection budgets are controlled. e-policy-resultse-setupe-filter-policye-identity
What it supports. Ours-2 reaches 0.868 mean success against 0.752 for Filtered BC-2, a difference of 11.6 percentage points. The detailed task rates are 0.92, 0.86, 0.86, 0.92 and 0.78. This is the key evidence that synthetic supervision can improve physical execution under the reported setting.
Where the evidence stops. The table implies a 40.8-point gain over the 0.460 base, whereas the abstract reports 39.2 points. This discrepancy remains unresolved. Evaluation/training separation and repeated-seed uncertainty are also unspecified.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Real-robot success across five manipulation categories DROID Franka Panda with Robotiq gripper; multitask training for two iterations. Appendix B states 50 evaluations per task, except DSRL with 10. No separately specified held-out evaluation split or repeated-seed uncertainty. | Ours-2: stacking 92%, wiping 86%, open book 86%, scooping 92%, drawing 78%; mean 86.8%. Ours-1 mean: 74.4%. Task success rate and unweighted mean | Base 46.0%; DSRL 50.0%; Filtered BC-1 64.8%; Filtered BC-2 75.2%. Ours-2 exceeds Filtered BC-2 by 11.6 percentage points. These are real execution results. Table 2 implies a 40.8-point gain over base; the abstract instead claims 39.2 points, an unresolved discrepancy. Section 5.3 claims matched online rollout budgets, but Appendix B explicitly gives fewer DSRL evaluations. e-setupe-policy-resultse-identity |
| World-model action replay fidelity 256 five-second wrist-view clips; 50 interaction clips for event counts. Recorded real actions are replayed from matching starting frames. Train/test separation is not specified. | Expert-plus-online: 21.77, 0.784, 0.136, 9.58, 64.12; TP/FN/TN/FP = 26/4/19/1. PSNR ↑; SSIM ↑; LPIPS ↓; FID ↓; FVD ↓; event confusion counts | Expert-only: 19.87, 0.748, 0.189, 12.76, 99.98; 28/2/9/11. Pretrained: 16.32, 0.634, 0.347, 41.03, 225.13; event counts unreported. Online data reduces false optimism substantially, but increases false negatives from 2 to 4. Video fidelity and event agreement measure model prediction, not robot success. e-wm-metrics |
| First-round drawing data ablation Drawing task; 50 real rollouts and either 500 or 250 synthetic rollouts before successful-trajectory filtering. | Real plus 500 synthetic: 68%; real plus 250 synthetic: 60%. Success rate | Synthetic only: 50%; real only: 46%; base: 22%. Combining the sources works best here. Accepted sample counts and matched optimization exposure are not reported, so this is not a pure test of data diversity at fixed training-data size. e-drawinge-filter-policy |
| Reward-model success filtering 40 manually labeled trajectories; direct yes/no versus probability threshold greater than 0.8. | Threshold: 10/12/2/16. TP/FN/FP/TN counts | Direct output: 15/7/8/10. Fewer false acceptances come with fewer retained true successes. The diagnostic does not measure the downstream policy effect of changing the threshold. e-reward |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 6. Matched actions expose outcome differences that visual realism alone can hide. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each task band begins with an initial wrist image and its instruction. Use border colors rather than reading every row as one long sequence: blue denotes the real outcome, dark gray the pretrained world model, yellow expert-only adaptation, and green online adaptation. Within each task, the compared predictions use the same initial frame and five identical recorded action chunks. Inspect whether the object configuration or drawn trace agrees with the real sequence. The authors use these examples to illustrate the overly optimistic outcomes of expert-only training and the closer alignment obtained from online policy experience. e-replaye-wm-metrics
What it supports. The figure makes the diagnostic question concrete: did a given action actually accomplish the intended interaction? A visually plausible block, scoop or circle can still imply the wrong outcome. The selected comparisons illustrate the failure mode measured more systematically by Table 1’s event-confusion counts.
Where the evidence stops. These are selected wrist-view examples, not a frequency estimate. The colors identify world-model training variants, not different robot policies. The caption’s physical-fidelity conclusion should be read alongside the quantitative replay protocol.
Figure 9. Drawing benefits from both real successes and a larger synthetic rollout pool. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. This is a first-round drawing experiment, not the five-task average or the final policy. Match the five colors to the legend on the left. The vertical scale is a success fraction; the labels above bars express percentages. The caption specifies a real rollout pool of 50 trajectories and a synthetic pool reduced from 500 to 250 in the half-synthetic condition. Equation (4) subsequently filters trajectories for policy training, so these pool sizes should not be read as the number of successful training examples. Compare green with purple for synthetic quantity and green with pink for retaining real data. e-drawinge-filter-policy
What it supports. The full mixture reaches 68%, versus 60% with half the synthetic pool and 50% with synthetic data alone. Real-only training reaches 46%, above the 22% base. Together these comparisons support using both sources and generating more synthetic candidates on this task, rather than assuming imagined trajectories can replace all real policy supervision.
Where the evidence stops. Only drawing in the first round is ablated. The figure gives neither accepted-success counts nor uncertainty, and changing the pool changes available training data; it does not isolate diversity at fixed accepted sample volume.
Table 3. A conservative reward threshold removes false successes but discards additional true successes. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Rows indicate human-labeled ground truth and columns indicate model predictions. The upper matrix uses direct yes/no output. The lower matrix accepts a trajectory only when the yes-token probability is strictly greater than 0.8. Read the bottom-left cell of each matrix to count failures that would incorrectly enter the successful training set: 8 above and 2 below. Then inspect the top-right cells, which count real successes rejected by the filter: 7 and 12. Both matrices cover the same stated total of 40 manually labeled trajectories, allowing the two kinds of error to be considered together. e-rewarde-filter-policy
What it supports. Thresholding makes the accepted dataset more conservative, but it does not improve every error category. True-positive acceptances decrease from 15 to 10 while false-positive acceptances decrease from 8 to 2. This supports the authors’ concern about contaminated success labels and reveals an accompanying loss of useful successful experience.
Where the evidence stops. The small diagnostic is not a calibration curve or a downstream policy ablation over thresholds. The source does not specify the subset’s task composition or its separation from reward-model training data.
7. Analysis & limitations
7.1 What the evidence leaves open
Evaluation covers five task categories and world-model adaptation uses online data from those downstream tasks. Broader task coverage is explicitly future work. e-limits
Long-horizon examples show imagined trajectories up to 20 seconds, but quantitative replay uses five-second clips. Selected plausible videos do not establish long-horizon physical correctness or transfer to unseen tasks. e-long-rolloutse-wm-metrics
No uncertainty bars, repeated-seed analysis, or explicit held-out split accompanies the main results. Appendix B relates evaluation counts to collected online rollouts, leaving evaluation/training separation unclear. e-policy-results
7.2 Questions for discussion
- Does better failure-event calibration predict physical policy gains when synthetic training volume is held fixed (e-wm-metrics, e-drawing)?
- How should reward precision and successful-trajectory coverage be traded off across tasks (e-reward)?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the π0.5, Ctrl-World and Qwen3-VL-4B-Instruct initializations, original DROID data, a three-camera Franka/Robotiq setup, expert demonstrations and success-labeled online interactions. The described schedules alone do not specify learning rates, optimizer settings, λ, frozen modules, accelerator counts, wall-clock cost, software versions or complete action/state encoding. e-setupe-world-traininge-schedulee-reward
Operationalize the outcome criteria before evaluation: Appendix B leaves the stack holding duration, book-opening angle, minimum scooped amount and drawing gap tolerance qualitative. Preserve the explicitly specified reward threshold of 0.8 rather than confusing it with those missing task thresholds. e-setupe-reward
Proposed checks: hold accepted training volume fixed when comparing real/synthetic mixtures, and independently audit reward thresholds for both false acceptance and lost true successes before measuring physical policy outcomes. These are proposed experiments, not reproduced results. e-drawinge-reward
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the drawing gain survive matched training exposure?
Reader-proposed experiment: repeat the first-round drawing comparison from the same warm-start policy and real rollout pool. Compare real-only training, real plus 250 synthetic candidates, and real plus 500 candidates. Record acceptance counts, then add controls that equalize accepted transitions, real/synthetic sampling ratios where applicable, update steps and batch size; oversample real successes for a matched-exposure real-only control. Evaluate independent reset trials with a preregistered circle-closure tolerance and repeated seeds. A remaining mixture advantage would support useful additional synthetic coverage. If the gain disappears after exposure is matched, the original result would be explained more narrowly by training quantity or sampling. e-drawinge-filter-policye-schedulee-setup
Check 2: Does conservative reward filtering improve physical learning?
Reader-proposed experiment: freeze one world model and candidate rollout pool, then compare direct yes/no filtering with strict probability thresholds of 0.5, 0.8 and 0.9. Use a separate human-labeled set, stratified by task and excluded from reward-model fitting, to measure false acceptances and rejected true successes. Train matched policy copies with both natural accepted-set sizes and a size-matched control, keeping update exposure fixed. Evaluate real-robot success on independent trials. The mechanism is supported if reduced label contamination improves control without unacceptable loss of successful coverage; it is weakened if stricter filters merely reduce false positives while yielding equal or worse physical success. e-rewarde-filter-policye-policy-results
8.3 Reading coverage
Visual audit: Visually inspected the title, authors, affiliations and both date markings on page 1; overview and pipeline figures on pages 1–3; method equations and Algorithm 1 on page 4; setup and regularized-RL formulation on page 5; world-model table and rollout figure on page 6; replay and policy plots on page 7; candidate rollouts, drawing ablation and limitations on page 8; derivation and task criteria on page 13; and evaluation counts, detailed policy results, reward configuration and confusion matrices on page 14. All six final original crops were inspected. Figure 3 was cross-checked against its caption, Equations (3)–(4), Algorithm 1 and Appendix A: its compact synthetic-data arrow omits the explicitly described policy/world-model alternation, and its advantage-weighted label is interpreted through the implemented binary filtering. The strict reward inequality and all table orientations were checked. Pages 9–12 contain reference material read as text; external videos and separate supplements were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 13, 14. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; 1. Introduction (pp. 1–2)
- 2. Related Works, including 2.1–2.2 (pp. 2–3)
- 3. Preliminaries (p. 3)
- 4. Co-Improvement of VLA and World Model, including 4.1–4.3 and Algorithm 1 (pp. 3–5)
- 5. Experiments, including 5.1–5.3 (pp. 5–8)
- 6. Conclusions and discussions (p. 8)
- Impact Statement; Acknowledgment; References (pp. 9–12)
- Appendix A. Relation to Regularized Reinforcement Learning (p. 13)
- Appendix B. Task Details (pp. 13–14)
- Appendix C. Reward Model Details (p. 14)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All five supplied text chunks were read, covering the complete 14-page PDF. Text extraction does not reconstruct figure images; this gap was addressed by inspecting PDF pages 1–8 and 13–14 and all final crops. Reference-only pages 9–12 were read as text.
- Separate supplemental material availability has not been fully verified. No separate supplements, linked videos, code, or checkpoints were inspected, and no experiments were reproduced.
- Identity and edition: the observed title and all six authors match the catalog. The title page identifies arXiv:2602.12063v2 [cs.RO], 15 February 2026, while its footer says Preprint, February 17, 2026. Both dates are preserved without reconciliation. The catalog lists ICML 2026; a proceedings edition and other revisions were not supplied or compared.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title/author block, arXiv margin, preprint footer and Abstract
Title and six authors match; affiliations are Stanford University and Tsinghua University. Margin identifies v2, 15 February 2026; footer dates the preprint February 17. Abstract claims 39.2% absolute improvement over base and 11.6% from synthetic rollouts.
Go to primary source ↓e-problemPDF pp. 1–3, Introduction and Section 4.1 opening
Real collection is costly; demonstration-heavy world models can be optimistic and miss contact/deformation details. Online policy experience supplies successes and failures.
Go to primary source ↓e-loopPDF p. 3, Figure 3 and caption; Sections 3–4
Separate policy and world model interact from real initial observations to generate imagined trajectories. Real rollouts ground the world model; real and synthetic data feed policy learning.
Go to primary source ↓e-world-trainingPDF p. 4, Section 4.1, Eqs. (1)–(2); Algorithm 1 lines 4–7
Growing real rollout data, including failures, fine-tunes pretrained Ctrl-World through future-observation denoising; DROID co-training regularizes it with λ.
Go to primary source ↓e-filter-policyPDF p. 4, reward-model paragraph, Eqs. (3)–(4); Algorithm 1 lines 9–12
First-iteration labels fine-tune the reward model. A strict yes-probability threshold filters synthetic success. Binary trajectory weights retain only successful real and synthetic samples for flow matching.
Go to primary source ↓e-rlPDF p. 5, Section 4.3, Eqs. (5)–(7); p. 13, Appendix A, Eqs. (8)–(11)
The authors motivate weighted flow matching using a regularized policy-improvement projection and offline approximation; binary filtering is related through limiting discount and failure-penalty assumptions.
Go to primary source ↓e-setupPDF p. 5, Figure 4 and Section 5.1; pp. 13–14, Appendix B, Success Criteria
Franka Panda, Robotiq gripper, two third-person cameras and one wrist camera serve stacking, opening books, wiping, scooping and drawing. Several success tolerances remain qualitative.
Go to primary source ↓e-schedulePDF pp. 5–6, Section 5.1, Base Models and Hyperparameters
π0.5 and Ctrl-World are the initial models. Twenty-five expert demonstrations warm-start each category; two iterations use 50 real rollouts/category, 50K world-model updates, 500 synthetic rollouts/task, and 2K policy updates at batch 256.
Go to primary source ↓e-wm-metricsPDF p. 6, Table 1, all rows/columns and caption; Section 5.2
Metrics use 256 five-second wrist-view clips and event counts use 50 interaction clips. Online adaptation improves all reported video metrics and changes expert-only event TP/FN/TN/FP from 28/2/9/11 to 26/4/19/1.
Go to primary source ↓e-replayPDF p. 7, Figure 6 and caption
Colored borders compare real trajectories, pretrained predictions, expert-only predictions and online-adapted predictions using identical starting frames and five recorded action chunks; only wrist views are shown.
Go to primary source ↓e-long-rolloutsPDF p. 6, Figure 5 and caption; p. 7, Section 5.2, Policy-in-the-loop rollout; p. 8, Figure 8 and caption
Selected policy-in-the-loop examples run for 20 iterations/seconds. Figure 8 displays diverse imagined candidates from real initial frames, with real failures and some imagined successes.
Go to primary source ↓e-policy-resultsPDF p. 14, Appendix B evaluation paragraph and Table 2, all rows/columns; p. 7, Figure 7 and Section 5.3 baselines
Table 2 means are 0.460, 0.500, 0.648, 0.752, 0.744 and 0.868. Appendix B states 50 evaluations per task except 10 for DSRL; Section 5.3 states equal online rollout budgets. No repeated-seed uncertainty is given.
Go to primary source ↓e-drawingPDF p. 8, Figure 9, legend, bars and caption; Section 5.3 Ablations
First-round drawing rates are 22, 46, 50, 60 and 68 percent for base, real only, synthetic only, real plus half synthetic and full mixture. Caption specifies 500 versus 250 synthetic and 50 real rollouts.
Go to primary source ↓e-rewardPDF p. 14, Appendix C and Table 3, both confusion matrices
Reward inputs contain 16 frames; Qwen3-VL-4B-Instruct trains for 200 steps at batch 128. On 40 labeled trajectories, strict p(yes)>0.8 changes TP/FN/FP/TN from 15/7/8/10 to 10/12/2/16.
Go to primary source ↓e-limitsPDF p. 8, Section 6, Conclusions and discussions
The authors explicitly limit evaluation to five categories and identify broader online task coverage as future work.
Go to primary source ↓8.5 Primary sources
VLAW: Iterative Co-Improvement of Vision-Language-Action Policy and World Model ↗
PDF · 8,054 extracted words
Source fingerprint
34ba87492a0fca9d382c366a80884eef99bc1101d973bbbb64125ae78bedffb8