WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models
1. Paper overview
In one sentence: WISE spends world-model computation on selected interaction states to improve a VLA action head, trading exhaustive imagination for dependence on learned state selection and reliable trajectory scoring. e-probleme-contexte-world-modele-updatee-scheduling-ablatione-reward-ablatione-real
| At a glance | What to know |
|---|---|
| Research problem | Author claim Demonstration-based adaptation is costly, while physical trial-and-error incurs interaction costs. The authors argue that uniform imagination also wastes computation and that long rollouts accumulate prediction errors. WISE therefore asks where to imagine, how far to predict, and which imagined comparisons can safely inform policy refinement. e-problem |
| Core mechanism | Source description A coordinated post-training loop combines visual state selection, bounded multi-view prediction, reliability-filtered trajectory comparisons, and action-head refinement grounded in real observations. e-contexte-architecturee-updatee-training |
| A key reported result | Imagination scheduling on Stack, Threading, and Mug Cleanup: WISE: 68.3; 28; 2.61. Mean success (%); selected states; reported GPU hours. Three-task ablation; selected-state counts averaged across tasks with fixed per-state candidate evaluation. Full imagination: 60.4; 138; 11.45. Uniform: 61.1; 28; 2.90. Random: 59.1; 28; 2.89. Reported savings are approximately 80% of selected states and 77% of GPU time versus full imagination, with +7.9 success points. Equal-state-budget comparisons support selection quality; these are not total training-cost savings. e-scheduling-ablatione-efficiency-scopee-training |
| Reading caution | Source description The authors identify fixed imagination horizons and coarse stage supervision as limitations, and leave broader embodiments, longer tasks, and more backbones to future work. e-limitations |
Core contributions
- Source description
A coordinated post-training loop combines visual state selection, bounded multi-view prediction, reliability-filtered trajectory comparisons, and action-head refinement grounded in real observations. e-contexte-architecturee-updatee-training
- Source description
Matched-budget scheduling and reward-component ablations test the allocation of imagination and the construction of its learning signal, alongside simulated and physical policy evaluations. e-simulatione-reward-ablatione-scheduling-ablatione-real
Figure 2. A post-training loop uses imagined futures to evaluate actions anchored in real observations. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the observation and instruction in the upper left. The policy supplies candidate action sequences, and the scheduler's cross branch proceeds to the next state while its check branch invokes the world model. Follow the imagined videos into the reward model, then the filter, group advantage, buffer, and update arrows. This is a schematic: Sections 3.2–3.4 establish that continuation actions can depend on predicted images, but supervision applies only to the initial action chunk from the real context. The diagram's language label L should not be confused with the rollout length L in the equations, where language is written ℓ. e-architecturee-contexte-world-modele-schedulere-rewarde-updatee-training
What it supports. The architecture separates selecting a useful context from judging the quality of candidate futures. Its feedback loop changes the VLA action head during post-training; it does not show a single network jointly predicting actions and observations. A faithful reading therefore tracks which module predicts images, which produces actions, and where the update is applied.
Where the evidence stops. The filter box uses ρ>0.5, σ_R<0.2 and ΔR>0.02. Section 3.4 instead specifies ρ_rank≥0.5, σ_WM≤0.2 and ΔR≥0.02. This edition follows the explicit equations; implemented boundary behavior and the notation difference remain unresolved.
2. Motivation
2.1 The problem and the proposed response
Demonstration-based adaptation is costly, while physical trial-and-error incurs interaction costs. The authors argue that uniform imagination also wastes computation and that long rollouts accumulate prediction errors. WISE therefore asks where to imagine, how far to predict, and which imagined comparisons can safely inform policy refinement. e-problem
2.2 What this reading follows
A robot policy may move through open space competently yet struggle when it must grasp, insert, or manipulate an articulated object. WISE turns that uneven difficulty into a post-training strategy: select a real observation, imagine several candidate continuations, and use their relative quality to refine the action that started each continuation. The key distinction is where supervision lands. Predicted images help evaluate alternatives, but only actions originating from real contexts train the policy. Read the architecture first, then compare the scheduling and reward ablations with the physical execution results. The evidence supports this coordinated pipeline while leaving several implementation choices unresolved. e-probleme-contexte-world-modele-updatee-scheduling-ablatione-reward-ablatione-real
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The snapshot is unassigned, so there is no existing quadrant judgment to confirm. Architecture evidence supports separate policy and forward world-model modules plus separate scheduler/evaluator networks. Actions condition future-image prediction; no inverse-dynamics action extraction or single-model joint future/action prediction is described. This is world-model-guided policy post-training; catalog assignment remains an editorial decision. e-contexte-world-modele-updatee-training
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Use interaction labels as a proxy for where computation helps
The scheduler answers a narrower question than the policy. The policy must choose actions from images, proprioception, and an instruction; the scheduler predicts whether the current images look interaction-relevant. Offline gripper and grasp states supply weak labels for grasping or active manipulation, and a DINOv2-based classifier learns to recognize those contexts visually. Once frozen, it no longer needs those execution signals. The reader's interpretation is that this converts an expensive question—whether extra imagination will improve learning—into a cheaper classification proxy. That proxy is useful only if interaction relevance tracks informative candidate differences. Table 3 tests this indirectly: learned selection beats uniform and random selection at the same average state count. It does not measure the causal value of imagination at each selected state. e-contexte-schedulere-scheduler-traininge-scheduling-ablatione-efficiency-scope
5.2 Let synthetic continuations judge a real-context action
At a selected real context, the policy samples several possible first action chunks. Each starts a bounded continuation: the world model predicts multi-view observations, a deterministic rule updates proprioception, and the policy chooses the next chunk from those imagined inputs. Repeating for L chunks gives LH future frames per candidate. The evaluator reduces that continuation to a reward, and two imaginations per candidate provide the repeated predictions used by the reliability filter. Accepted rewards become normalized group advantages. The update then returns to the action that began the continuation at the real observation. It does not imitate every generated continuation action. This separation is central to WISE's training design: imagined states supply comparative evidence, while the action-model loss is evaluated on the first chunk and its real context. e-contexte-world-modele-rewarde-update
5.3 Read reward shaping as a set of assumptions to test
WISE's forward term credits positive progress changes, weighted by confidence and clipped per frame. Its backward term penalizes progress reversals, and its completion term takes the maximum completion score along the rollout. These choices make candidate evaluation sensitive to intermediate advancement as well as eventual task completion. The authors' ablation shows better policy success with the full combination; Figure 6 explains why progress and completion are not interchangeable, since some tasks receive completion evidence only near the end. A reader-side concern is that bounded imagination and repeated predictions cannot by themselves rule out a shared evaluator error. A transient false completion peak can affect the maximum term even if later frames regress. The report's proposed ranking check therefore tests these signals against executable simulator outcomes rather than assuming that a high learned reward is correct. e-rewarde-reward-traininge-reward-ablatione-diagnosticse-world-modele-update
5.4 Training and inference
During training
World-model training uses rectified flow on future VAE latents, with perturbed observed latents as conditioning. It first uses full DROID, then the same task demonstrations as the VLA: 300 per simulated task and 100 per real task. Both VLA backbones are task-fine-tuned before WISE. Post-training accumulates 256 rollouts per selected context/update and changes only the action head; the world model, VLA backbone, scheduler, and evaluator are frozen. e-world-modele-training
The scheduler trains its head and final DINOv2 block with class-weighted binary cross-entropy. The evaluator trains its heads and final four blocks for 15 epochs using AdamW, combining temporal ranking, successful/failed hard negatives, absolute progress, confidence, completion, and augmentation consistency. e-scheduler-traininge-reward-training
During inference
During post-training, the policy repeatedly acts inside bounded imagined continuations, while real contexts anchor learning. Physical evaluation measures execution by the resulting policies. The paper does not specify a separate deployment-time search algorithm or demonstrate a runtime planning-speed improvement. e-world-modele-updatee-protocol
5.5 Implementation flow
- Select a real context
A view-shared DINOv2 encoder and MLP estimate visual interaction relevance. Imagination occurs only when the score exceeds the scheduling threshold. Gripper/grasp signals provide offline weak labels, but the trained scheduler uses images alone. e-schedulere-scheduler-training
- Branch into bounded counterfactuals
The policy samples M action chunks. A frozen Open-Sora-derived model predicts wrist and third-person futures. Subsequent chunks use imagined images and proprioception updated deterministically by F_s. L chunks each predict H frames, giving LH imagined frames; these continuations evaluate the initial candidate. e-contexte-world-model
- Score progress and completion
A separately trained DINOv2 evaluator predicts progress p, confidence c, and signed completion q. Reward combines confidence-weighted forward progress, a backward-progress penalty, and the maximum completion score. Confidence is not a probability of task success. e-rewarde-reward-traininge-diagnostics
- Filter and refine
Each candidate is imagined twice. Group acceptance requires reward separation at least 0.02, ranking consistency at least 0.5, and reward variation at most 0.2. Normalized signed advantages weight action-model gradients. Only the first action from the real context is supervised; generated state–action pairs are not training targets. e-update
6. Experiments & results
WISE post-trains a VLA action head by selectively imagining alternative behaviors at visually identified interaction states. A separate frozen world model predicts bounded futures; a frozen evaluator ranks them; updates supervise only the first action chunk from a real context. The strongest controlled evidence is the scheduling ablation: higher task success with substantially less imagination computation. The method, results, and reproduction boundaries below trace this conclusion to the primary text.
6.1 Read the original evidence
Table 1. Both backbones improve across all five simulated tasks, with different taskwise strengths. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each gray WISE row against its matching base-policy row, then use the blue delta row to inspect the taskwise change. All main cells are success percentages; delta entries express percentage-point gains. The final column averages Stack, Coffee, Square, Threading, and Mug Cleanup. Appendix D specifies MimicGen D0 environments and 96 episodes per task/policy, with shared evaluation seeds for base and post-trained policies. Compare π₀ + WISE with π₀ + DPO to assess the strongest listed same-backbone baseline. Avoid treating π₀.₅ as uniformly stronger: its starting success varies substantially by task. e-simulatione-environmentse-protocole-summary-mismatch
What it supports. π₀ + WISE reaches 59.5% average success, versus 49.7% for its base and 53.8% for DPO. π₀.₅ rises from 52.3% to 60.2%. WISE improves every task relative to its corresponding base, although the best final backbone differs across tasks; π₀ + WISE is stronger on Coffee and Threading.
Where the evidence stops. Figure 1's π₀ summary chart prints 59.8%, whereas this table and Section 4.2 print 59.5%; the detailed result is retained here. No policy-success confidence intervals or repeated-training variability accompany the table.
Table 4. Controlled physical trials show larger average gains under the tested distribution shifts. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each task has a Std. column for nominal conditions and a Gen. column averaging controlled variations. Read those pairs separately before comparing the final four-task averages. Open requires only the opening phase; Open & Close requires both stages within one execution. The physical protocol counts partial completion as failure and permits neither retries nor manual intervention after execution begins. The generalization factors change interaction height, illumination, distractors, or instruction wording independently. Blue rows give percentage-point changes from the matching backbone. Appendix D states 20 trials per task/policy, but does not resolve how those trials are allocated across shifts. e-reale-environmentse-protocole-shifts
What it supports. Under generalization, π₀ improves from 42.5% to 68.8%, and π₀.₅ improves from 50.0% to 71.3%. Both WISE variants reach 77.5% standard-setting average success. The gains represent physical task execution rather than generated-video quality, while the Open & Close columns show that substantial failure remains in the multi-stage task.
Where the evidence stops. The average hides variation among shift types: separate height, lighting, distractor, and language scores are not supplied. These controlled laboratory trials therefore support bounded robustness claims, without establishing arbitrary scene changes or statistically quantified reliability.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Five-task MimicGen D0 manipulation Stack, Coffee, Square, Threading, Mug Cleanup; 96 episodes per task/policy, paired evaluation seeds; within-backbone initialization matched. | π₀ + WISE: 59.5; π₀.₅ + WISE: 60.2. Mean episode success rate (%) | Base policies: 49.7 and 52.3; gains of 9.8 and 7.9 percentage points. π₀ + DPO: 53.8. Every task improves relative to its corresponding base. Figure 1 instead prints 59.8 for π₀ + WISE; Table 1 and Section 4.2 consistently give 59.5, used here. e-simulatione-environmentse-protocole-summary-mismatch |
| Imagination scheduling on Stack, Threading, and Mug Cleanup Three-task ablation; selected-state counts averaged across tasks with fixed per-state candidate evaluation. | WISE: 68.3; 28; 2.61. Mean success (%); selected states; reported GPU hours | Full imagination: 60.4; 138; 11.45. Uniform: 61.1; 28; 2.90. Random: 59.1; 28; 2.89. Reported savings are approximately 80% of selected states and 77% of GPU time versus full imagination, with +7.9 success points. Equal-state-budget comparisons support selection quality; these are not total training-cost savings. e-scheduling-ablatione-efficiency-scopee-training |
| Trajectory reward ablation on Stack, Threading, and Mug Cleanup Three-task policy-success evaluation with reduced reward formulations. | Full reward: 68.3. Mean success rate (%) | Progress only: 57.3; completion only: 57.3; without confidence: 58.0; without backward penalty: 59.4. The combined evaluator outperforms each removal. This demonstrates downstream usefulness, without independently proving confidence calibration or disentangling all component interactions. e-reward-ablation |
| Real-world manipulation under controlled distribution shifts Galaxea R1 Lite; Pick-and-Place, Two-Cube Stacking, Open, Open-and-Close. Generalization averages independently varied height, lighting, distractors, and instruction paraphrases. Protocol states 20 trials per task/policy. | π₀ + WISE: standard 77.5, generalization 68.8. π₀.₅ + WISE: standard 77.5, generalization 71.3. Four-task mean success rate (%) | π₀: 60.0/42.5; π₀.₅: 61.3/50.0. Generalization gains: 26.3 and 21.3 percentage points. These are physical execution results with complete-task success and no intervention or retries. The allocation of the stated 20 trials across individual shifts is unclear; pooled generalization does not establish performance for every shift. e-reale-environmentse-protocole-shifts |
| Reward-model outcome discrimination Five simulated tasks; 20 successful and 20 failed videos per task, 64 sampled frames per video. | Square 0.993; Coffee 0.938; Stack 1.000; Threading 1.000; Mug Cleanup 0.973. Final-reward AUROC | Source defines 0.5 as chance discrimination. The plotted sample separates final outcomes well, but its train/test separation is not specified; this is not a measured counterfactual-ranking accuracy on unseen imagined videos. e-diagnostics |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. The combined reward supports better downstream policy success than any listed reduced formulation. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The rows alter the trajectory evaluator used to rank imagined candidate actions. Progress supplies intermediate changes, completion supplies an outcome signal, confidence weights forward increments, and the backward penalty charges for regression. Read down each task column before using Avg.; this table covers only Stack, Threading, and Mug Cleanup, so its average cannot be directly compared with the five-task average in Table 1. The full formulation's gray row is the reference for each removal. These cells measure task success after policy refinement, not prediction accuracy, reward magnitude, or confidence calibration. e-rewarde-reward-traininge-reward-ablation
What it supports. The full reward achieves 68.3% mean success, compared with 57.3% for either progress or completion alone. Removing confidence yields 58.0%, and removing the backward penalty yields 59.4%. All three tasks favor the full formulation, supporting the usefulness of combining intermediate progress, terminal information, and safeguards against misleading progress.
Where the evidence stops. These are component-removal comparisons, not a complete factorial study. They do not establish independent additive effects or calibrated confidence, and the paper supplies no uncertainty estimates for these success-rate differences.
Table 3. At the same selected-state budget, learned scheduling gives the highest reported success. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across each row from policy success to computation. The three task columns and Avg. measure execution success; the final two columns count selected states and report GPU hours. Appendix C clarifies that state counts are averaged across tasks and that each selected state receives the same candidate-generation and world-model evaluation procedure. Full Imagination tests the cost of invoking imagination broadly. Uniform and Random are the more direct allocation controls because they select the same number of states as WISE. Equal state count is a useful budget match, although measured GPU time differs slightly. e-scheduling-ablatione-efficiency-scopee-training
What it supports. WISE achieves 68.3% success using 28 selected states and 2.61 GPU hours. Full imagination gives 60.4% using 138 states and 11.45 hours: the paper reports about 80% fewer states and 77% less GPU time. At 28 states, WISE exceeds uniform and random scheduling by 7.2 and 9.2 percentage points.
Where the evidence stops. These savings concern the reported imagination/post-training comparison, not the complete pipeline. Table 5 separately lists expensive world-model pretraining and task adaptation. The table does not establish deployment latency or a total training-cost reduction.
Figure 6. Reward components separate final outcomes, while high confidence can also occur in failed videos. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read one task row from left to right: progress p(t), completion q(t), confidence c(t), and cumulative reward R(t). The legend separates successful and failed videos, with 20 of each per task. Solid curves are means; shaded bands are interquartile ranges, not confidence intervals. Time is normalized to trajectory completion, and Appendix B states that 64 frames are sampled per video. Watch Threading and Mug Cleanup: completion rises very late in successful trajectories. Then inspect confidence, which can remain high even when execution fails. The rightmost annotations evaluate final-reward discrimination rather than framewise calibration. e-diagnosticse-rewarde-reward-training
What it supports. Final-reward AUROC ranges from 0.938 for Coffee to 1.000 for Stack and Threading; Square is 0.993 and Mug Cleanup is 0.973. These plots explain why completion and progress contribute different temporal signals. They also directly caution against interpreting the confidence head as a success predictor: confidence can stay high for the failed-video group.
Where the evidence stops. The source does not state train/test separation for these videos or establish their representativeness of unseen counterfactual rollouts. Strong final-outcome AUROC on this sample does not prove accurate ranking among closely competing imagined action candidates.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify fixed imagination horizons and coarse stage supervision as limitations, and leave broader embodiments, longer tasks, and more backbones to future work. e-limitations
Figure 2 uses strict filter inequalities and σ_R, whereas Section 3.4 uses inclusive bounds and σ_WM. The report follows the explicit equations; exact implemented boundary behavior remains unresolved. e-architecturee-update
Policy-success tables omit confidence intervals and training-run variability. The small physical protocol and aggregate shift scores limit claims about statistical reliability or broad robustness. e-simulatione-reale-protocole-shifts
7.2 Questions for discussion
- Does interaction relevance predict the benefit of imagination beyond merely detecting contact stages?
- Do repeated imagined outcomes remain reliably ranked when the policy departs from the task demonstrations?
8. Reproducibility audit
8.1 Requirements and known gaps
Budget for the full pipeline: Table 5 reports world-model pretraining on 16 NVIDIA A800 GPUs for about seven days, task adaptation on eight A800s for about ten hours per task, and VLA adaptation taking about twelve hours in simulation or eight hours for real tasks. The latter stage's GPU count is not specified. e-training
A faithful implementation still needs numerical H, L, M and κ, the relationship between M and 256 accumulated rollouts, the deterministic proprioception update, exact ranking/variation statistics, the reference loss and coefficient, and optimizer schedules. Appendix A omits these despite main-text cross-references; Appendix B gives only coarse label construction and partial evaluator-loss definitions. e-world-modele-schedulere-updatee-traininge-scheduler-traininge-reward-training
The reproducibility statement claims an anonymous repository and a future public release. The supplied PDF provides no repository address there; this is an author availability statement, not verified code access. e-code
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Match the budget, then test whether selected states are more informative
Reader-proposed check, not performed: use the same task-adapted π₀, world model, evaluator, real-context pool, and policy-update count on Stack, Threading, and Mug Cleanup. Compare learned, uniform, and random state selection with the same average 28-state budget and identical candidates, rollout horizons, repeated imaginations, and acceptance rules. Record actual world-model calls, GPU time, accepted-group rate, and reward separation. Evaluate each resulting policy on the same 96 initial-condition seeds and repeat training seeds to estimate variability. The selection hypothesis weakens if WISE's success advantage disappears after these controls, or if it mainly reflects unequal accepted-update counts. Declare currently unspecified rollout and optimizer settings before running. e-scheduling-ablatione-efficiency-scopee-protocole-updatee-training
Check 2: Check imagined rankings against executable outcomes as horizons grow
Reader-proposed check, not performed: save held-out simulator states and sample an identical first-action candidate set at each state. Generate repeated imagined continuations at several declared rollout lengths while freezing all models. Execute matching first chunks from restored simulator states, then continue with the same policy under actual observations to obtain complete-task outcomes. Compare ranking agreement and wrong-selection rates before and after WISE's reliability filter, separately for each horizon. Inspect cases with high peak completion followed by regression; as a diagnostic control, compare peak versus final-frame completion scoring. If accepted groups retain stable imagined rankings but consistently choose worse executable candidates, repeated-imagination agreement is insufficient evidence of trustworthy supervision. e-world-modele-rewarde-updatee-diagnosticse-protocole-limitations
8.3 Reading coverage
Visual audit: All six text chunks were read individually, covering the complete 26-page supplied PDF including references and Appendices A–D. Nineteen PDF pages were rendered and actually viewed: title/version/affiliations on p. 1; Figure 1 and its numerical discrepancy on p. 2; method, training and update equations on pp. 4–7; Tables 1–4 on pp. 8–10; task photos and generalization/failure examples on pp. 10–11; limitations and code-availability statement on pp. 11–12; compute, frozen modules and component training on pp. 19–21; Figure 6 and scheduling analysis on pp. 22–23; environment/platform, task trajectories and evaluation protocol on pp. 23–25; independent distribution shifts on pp. 25–26. All nine figures and five tables were visually inspected. Every final crop was separately viewed, with complete graphical content, headers, labels and relevant legends retained. Figure 2's inequalities were checked against Section 3.4; their discrepancy is disclosed. Related-work/reference-only pp. 3 and 13–18 were read as text, not rendered. External code and supplemental media remain outside this reading.
PDF pages inspected for this edition: 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 19, 20, 21, 22, 23, 24, 25, 26. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Work; 2.1–2.3
- 3. Method; 3.1 Problem Formulation; 3.2 Bounded Counterfactual Imagination; 3.3 Imagination Scheduling; 3.4 Counterfactual Trajectory Evaluation and Policy Refinement
- 4. Experiments; 4.1–4.4
- 5. Conclusion
- AI use, ethics and reproducibility statements
- References
- Appendix A. Implementation Details
- Appendix B.1. Imagination Scheduler Training
- Appendix B.2. Reward Model Training
- Appendix C.1–C.2. Further Analysis of Ablation Results
- Appendix D.1–D.4. Detailed Experimental Configurations
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The retained PDF was additionally inspected for all nine figures, all five tables, the title block, and every page supporting retained scientific details.
- Separate supplemental material availability has not been fully verified.
- Code, external references, and external supplementary media were not inspected; no experiments were reproduced.
- Identity/version: the exact title and five-author byline match the catalog. The PDF bears arXiv:2609.03681v1 [cs.RO], 3 Sep 2026, but its title-page dateline reads 2026/9/5. This date difference is preserved; no separate revision history was supplied and no later edition was substituted.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, affiliations, dateline, and arXiv margin stamp
Exact WISE title; Chenhao Zhang, Hanyu Zhao, Hang Cheng, Tengfei Pan, Long Zeng; Tsinghua University and Beijing Academy of Artificial Intelligence (BAAI). The stamp says v1, 3 Sep 2026; the dateline says 2026/9/5.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Introduction
Motivates reducing demonstration/exploration costs while avoiding uniform imagination and compounded rollout error; emphasizes real-context policy supervision.
Go to primary source ↓e-contextPDF p. 4, Sections 3.1–3.2, context and action-conditioned prediction equations
Context contains observations, robot state, instruction; the policy samples M action chunks and a separate frozen Open-Sora-adapted model predicts multi-view futures.
Go to primary source ↓e-architecturePDF p. 5, Figure 2, scheduling branches, reliability filter and update arrows
Diagram connects policy actions, scheduled rollouts, reward scoring, filtering, group advantages and policy updates. Its filter uses ρ>0.5, σ_R<0.2, and ΔR>0.02.
Go to primary source ↓e-world-modelPDF pp. 5–6, Section 3.2, rectified-flow loss and recursive rollout equations
Future VAE latents receive rectified-flow supervision; observed latents condition prediction. Policy actions and deterministic F_s propagate imagined continuations over L chunks of H frames.
Go to primary source ↓e-schedulerPDF p. 6, Section 3.3, DINOv2 scoring and m_t indicator equations
View-shared image encoding feeds an interaction-relevance score; s_t>κ triggers imagination. Execution signals supply weak training supervision, not scheduler inputs at use.
Go to primary source ↓e-rewardPDF pp. 6–7, Section 3.4, evaluator heads and G_i, B_i, T_i, R_i equations
Sigmoid progress/confidence and tanh completion produce clipped forward reward, backward penalty and maximum completion; progress starts from the real observation.
Go to primary source ↓e-updatePDF p. 7, Section 3.4, repeated-imagination filter, A_i and g_WISE equations and final paragraph
Two imaginations per candidate; inclusive thresholds ΔR≥0.02, ρ_rank≥0.5, σ_WM≤0.2. Signed stopped advantages weight flow-matching gradients plus a reference term. Only the first real-context action chunk is supervised.
Go to primary source ↓e-simulationPDF p. 8, Sections 4.1–4.2 and Table 1, all rows and Avg. column
Matched environments, evaluation and within-backbone initialization; π₀/π₀.₅ averages rise 49.7→59.5 and 52.3→60.2. π₀+GRPO, DSRL, DPO average 51.5, 50.0, 53.8 respectively; taskwise values are preserved in the original crop.
Go to primary source ↓e-summary-mismatchPDF p. 2, Figure 1(b), simulation π₀ chart; p. 8, Section 4.2 and Table 1
Figure 1 shows 59.8% for WISE on π₀ but labels the gain +9.8; the detailed section and table give 59.5% and +9.8 percentage points.
Go to primary source ↓e-reward-ablationPDF p. 9, Table 2, all variants and Avg. column
Full reward averages 68.3%, compared with 57.3 progress-only, 57.3 completion-only, 58.0 without confidence and 59.4 without backward penalty on Stack/Threading/Mug Cleanup.
Go to primary source ↓e-scheduling-ablationPDF p. 9, Table 3 and preceding scheduling paragraph
Full/Uniform/Random/WISE give average success 60.4/61.1/59.1/68.3, selected states 138/28/28/28, and GPU hours 11.45/2.90/2.89/2.61. Text reports about 80% fewer states and 77% less GPU time.
Go to primary source ↓e-realPDF pp. 9–10, Section 4.4 and Table 4, all task columns and Avg. Std./Gen.
Galaxea R1 Lite tests four tasks, with Open a subtask of Open-and-Close. WISE averages 77.5/68.8 for π₀ and 77.5/71.3 for π₀.₅ versus base 60.0/42.5 and 61.3/50.0; DPO on π₀ gives 55.0/45.0.
Go to primary source ↓e-limitationsPDF p. 11, Section 5, final paragraph
Authors explicitly describe fixed horizons and coarse scheduler supervision and propose adaptive horizons, finer scheduling, and broader embodiment/task/backbone evaluation.
Go to primary source ↓e-codePDF p. 12, Reproducibility statement
Claims an anonymous repository and public availability upon publication; this statement contains no repository URL.
Go to primary source ↓e-trainingPDF p. 19, Appendix A, Table 5 and Appendix B opening
Full DROID pretraining (citation rendered [?]), task demonstrations, 16/8 A800 GPU counts and approximate stage times, task-adapted VLAs, 256 rollouts per selected context/update, action-head-only optimization, and frozen auxiliary modules. No reference-loss definition appears in Appendix A.
Go to primary source ↓e-scheduler-trainingPDF pp. 19–20, Appendix B.1, architecture, weak supervision and optimization
DINOv2 ViT-L/14, two 256-wide hidden layers with LayerNorm/GELU/dropout 0.1; train head and final encoder block. Gripper/grasp-derived labels and weighted binary cross-entropy supervise visual interaction relevance.
Go to primary source ↓e-reward-trainingPDF pp. 20–21, Appendix B.2, architecture, training objective and reward computation
Separate DINOv2 ViT-L/14 with three heads; temporal ranking and failure hard negatives, composite loss weights, 15 AdamW epochs updating last four encoder blocks and heads. Reward uses α=0.2, β=2.0, δ=0.2 and remains frozen during policy optimization.
Go to primary source ↓e-diagnosticsPDF p. 21, Appendix B.2 qualitative visualization paragraphs; p. 22, Figure 6, all rows/columns and caption
Per task: 20 successful/20 failed videos and 64 frames each, normalized time. Lines are means and shading interquartile ranges. AUROCs are Square .993, Coffee .938, Stack 1.000, Threading 1.000, Mug Cleanup .973. Confidence can remain high for failure; video train/test separation is not described.
Go to primary source ↓e-efficiency-scopePDF pp. 22–23, Appendix C.2, scheduling interpretation and state-count definition
Selected states are averaged across tasks; each receives the same candidate-generation/world-model evaluation procedure, making state count a budget proxy under fixed per-state configuration.
Go to primary source ↓e-environmentsPDF p. 23, Appendix D.1–D.2 and Figure 7
Lists five MimicGen D0 tasks and synchronized wrist/third-person RGB, proprioception and instruction on Galaxea R1 Lite; compared methods share nominal platform, cameras, workspace and task setup.
Go to primary source ↓e-protocolPDF pp. 24–25, Appendix D.3, Evaluation Protocol
Complete success within a maximum horizon; partial completion fails. Native simulated success criteria, 96 episodes per task/policy and paired seeds. Real evaluation states 20 independent trials per task/policy, resets, no retries/intervention; four-task arithmetic means.
Go to primary source ↓e-shiftsPDF p. 9, Section 4.4; p. 11, Figure 4; pp. 25–26, Appendix D.4 and Figure 9
Generalization averages height, lighting, novel distractor and equivalent-language changes, each varied independently with identical configurations across policies. Height is adjusted, illumination switched off, targets retained and task meaning preserved; per-factor trial allocation and quantitative per-factor results are not specified.
Go to primary source ↓8.5 Primary sources
WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models ↗
PDF · 8,738 extracted words
Source fingerprint
19b00210ebf92358ee23ea74bc58acbdc93968bb7420184f9ae7628114038735