Awesome-WAMs
GitHub
PAPER REPORTENAll readings ↗

DreamWAM: Beyond RGB Future Prediction for World Action Models

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Shanglin Yuan; Weiheng Zhao; Xin Shi; Haoyi Jiang; Xianda Guo; Liu Liu; Wenyu Liu; Wei Sui; Xinggang Wang

Affiliations: Huazhong University of Science and Technology; D-Robotics; Wuhan University; Horizon Robotics

Source: 2608.04996 ↗ · Project page ↗ · Catalog record

Reading: 92 / 558 · 5 original figures & tables · ~18 min ·

1. Paper overview

In one sentence: DreamWAM improves robustness by learning several views of the future through representation-specific training routes, while keeping auxiliary teachers and predictions out of RGB-only deployment. e-probleme-viewse-routinge-zeroe-pluse-ablation

At a glanceWhat to know
Research problem
Author claim

RGB prediction can entangle task-relevant changes with texture, illumination and background. The authors propose richer future targets to preserve movement, spatial relationships and object identity under shifts that leave the required manipulation unchanged. e-problem

Core mechanism
Source description

Combine RGB–motion latent denoising with gated residual prediction of geometry and semantics in a coupled VideoDiT–ActionDiT policy. e-viewse-routinge-coupling

A key reported resultLIBERO-Plus matched robustness: DreamWAM-uncond 63.44; DreamWAM joint 75.47.

Unweighted mean of seven dimension success rates (%). Seven unseen perturbation dimensions; 10,030 episodes/seed, two seeds; no perturbation training data.

Matched Fast-WAM 51.36 and Fast-WAM-Joint 69.16. All seven columns improve in each pair. From printed averages, gains are 12.08 and 6.31 percentage points. Prose gives 12.09 for no-rollout; underlying rounding is unresolved. e-sim-protocole-plus

Reading caution
Source description

These targets are correlated projections, not complete physical state: motion is colorized 2D flow; depth is compressed; semantics are features, not labels. Metric 3D and contact-aware targets remain future work. e-targetse-boundaries

Core contributions

  • Source description

    Combine RGB–motion latent denoising with gated residual prediction of geometry and semantics in a coupled VideoDiT–ActionDiT policy. e-viewse-routinge-coupling

  • Reader analysis

    Matched no-rollout and joint evaluations separate benefits learned during training from benefits of online RGB imagination; routing controls test whether extra supervision alone suffices. e-pluse-ablation

Figure 2. Different future targets reach the action learner through different training routes. Original paper, p. 4 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Start at the lower-left world-state panel. RGB and colorized optical flow pass through the Wan VAE and channel concatenation into VideoDiT; channel splitting separates their predictions. Depth and DINO have their own prediction heads. The right panel expands the video block: mixed attention, text cross-attention and a feed-forward stage precede feature-specific residuals and gates. Read the plus nodes together with Eq. (3), which preserves the original block output and adds gated corrections. The adjoining ActionDiT is a separately parameterized expert coupled through shared attention. This is a training diagram: Appendix A.2 removes auxiliary computation at deployment. e-routinge-couplinge-viewse-objectivee-zeroe-targets

What it supports. The architectural idea is to match each target to a compatible representation pathway. Motion already occupies the video VAE grid, while depth and DINO features arise from external encoders. The authors use controlled residual corrections for those heterogeneous features so they can shape the shared representation used by action denoising.

Where the evidence stops. Panel (b) draws separate feature-output additions, whereas Eq. (3) writes both predictors as P_j(h_L); exact head wiring remains unresolved. The figure's five-loss summary also omits the gate penalty included in Eq. (4). This crop should not be read as the deployment graph.

2. Motivation

2.1 The problem and the proposed response

Author claim

RGB prediction can entangle task-relevant changes with texture, illumination and background. The authors propose richer future targets to preserve movement, spatial relationships and object identity under shifts that leave the required manipulation unchanged. e-problem

2.2 What this reading follows

A robot placing a mug must preserve which object it is moving, how it moves and where it should end up, even when the background or lighting changes. DreamWAM turns those requirements into complementary future-learning targets: RGB appearance, optical-flow motion, depth-derived geometry and DINO semantics. The policy still receives RGB observations at deployment. The central question is therefore whether richer training targets improve action learning, and whether their route into a pretrained video model matters. Read the architecture alongside the matched Fast-WAM comparisons: the largest gains appear under unseen perturbations, while the ablation shows that adding every target through denoising can hurt. e-probleme-viewse-routinge-zeroe-pluse-ablation

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimensionRecorded classification
Major categoryWAMs
ArchitectureDual-system
Prediction paradigmJoint prediction
QuadrantQ3 · Dual-system × Joint prediction

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

Separate VideoDiT and ActionDiT parameters support Dual-system; shared attention and concurrent video/action denoising support Joint prediction and Q3, rather than inverse-dynamics action extraction. Geometry and multiview preprocessing justify a qualified spatial tag, not a claim of metric 3D world reconstruction. No-rollout deployment does not erase joint predictive training. e-couplinge-viewse-targetse-sim-protocol

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

InputsOutputs
  • Language instruction and current RGB observation at deployment
  • Training demonstrations with future RGB clips and action chunks; offline teacher-derived future targets
  • Denoised action chunk for execution
  • Future RGB latents in joint inference; motion and feature predictions only during training

4.2 Equations and their role

xtq=(1−t)zq+tϵq,q∈{rgb,mot};xtjoint=Concat⁡C[xtrgb,xtmot]x_t^q=(1-t)z^q+t\epsilon^q,\quad q\in\{\mathrm{rgb},\mathrm{mot}\};\qquad x_t^{\mathrm{joint}}=\operatorname{Concat}_C[x_t^{\mathrm{rgb}},x_t^{\mathrm{mot}}]
Eq. (2): z is the target latent, x its perturbed version, t the shared flow-matching timestep, and epsilon the perturbation term. Concat_C joins channels. The source does not specify the perturbation distribution here. e-views
hˉℓ=Bℓ(hℓ−1),hℓ=hˉℓ+∑j∈{geo,sem}gℓj(hˉℓ,c)⊙Rℓj(hˉℓ),f^j=Pj(hL)\bar h_\ell=B_\ell(h_{\ell-1}),\qquad h_\ell=\bar h_\ell+\sum_{j\in\{\mathrm{geo},\mathrm{sem}\}}g_\ell^j(\bar h_\ell,c)\odot R_\ell^j(\bar h_\ell),\qquad \hat f^j=P_j(h_L)
Eq. (3): B is the original block; bar h its output; R a view-specific residual; g its gate conditioned on the state and instruction c. Elementwise gated corrections produce h. P predicts aligned features from final state h_L. e-routing
L=λrgbLrgbFM+λmotLmotFM+λgeoLgeopred+λsemLsempred+λactLactFM+λgLgate\mathcal L=\lambda_{\mathrm{rgb}}\mathcal L_{\mathrm{rgb}}^{\mathrm{FM}}+\lambda_{\mathrm{mot}}\mathcal L_{\mathrm{mot}}^{\mathrm{FM}}+\lambda_{\mathrm{geo}}\mathcal L_{\mathrm{geo}}^{\mathrm{pred}}+\lambda_{\mathrm{sem}}\mathcal L_{\mathrm{sem}}^{\mathrm{pred}}+\lambda_{\mathrm{act}}\mathcal L_{\mathrm{act}}^{\mathrm{FM}}+\lambda_g\mathcal L_{\mathrm{gate}}
Eq. (4): lambda terms weight flow-matching, feature-prediction and gate losses; the gate term penalizes gate activations with an L1 norm. Numerical weights and exact feature-loss formulas are unspecified. e-objectivee-gaps
Wxinfer+b=WRGBzRGB0+Wflow0flow+b=WRGBzRGB0+bW x_{\mathrm{infer}}+b=W_{\mathrm{RGB}}z_{\mathrm{RGB}}^0+W_{\mathrm{flow}}0_{\mathrm{flow}}+b=W_{\mathrm{RGB}}z_{\mathrm{RGB}}^0+b
Appendix Eqs. (8)–(10): the input projection W partitions into RGB and flow groups; b is its bias and z_RGB^0 encodes the current observation. Zero flow contributes no input signal, although learned shared parameters retain training effects. e-zero

5. Method in detail

5.1 1. Build future targets without introducing new deployment sensors

Source description

The four future views are computed from the same training videos, so the additional information enters as supervision rather than new robot observations. Appendix A makes the distinction concrete. A nine-frame clip produces eight adjacent optical flows; their color visualizations repeat the first transition before the video VAE encodes the sequence. Depth is computed per camera view, then restored to the concatenated layout and compressed with training-only PCA. DINO patch features undergo their own PCA and alignment. Both feature targets aggregate frames into three temporal groups matching the video latent grid. This construction explains why motion shares the RGB denoising space while geometry and semantics need separate treatment. It also limits the interpretation: the targets are compressed image-derived views, not a complete metric physical state. e-targetse-viewse-boundaries

5.2 2. Follow the training signal across the two experts

Source description

VideoDiT and ActionDiT keep separate parameters, but their tokens participate in shared attention at paired layers. The richer video representation can therefore influence action denoising without ActionDiT directly reading a depth map or DINO teacher feature. For motion, the paper concatenates noisy VAE latents with RGB at the same flow-matching timestep. For geometry and semantics, Eq. (3) preserves each original video block output and adds learned, gated residuals; the objective also penalizes gate activations. At deployment those branches and teachers are inactive. Appendix A.2 shows that zero-filled flow channels make no contribution through the input projection. The shared backbone, however, retains parameters learned under the auxiliary objectives. This provides a mechanism for training-time supervision to affect later RGB-conditioned action generation. e-couplinge-viewse-routinge-objectivee-zero

5.3 3. Separate three conclusions that the experiments support differently

Reader analysis

Reader analysis: begin with whether structured training helps at all. The matched no-rollout pair improves on LIBERO-Plus even though neither model generates future video online. Next ask whether online imagination still helps: DreamWAM's joint variant scores higher than its no-rollout variant, so these results support complementary benefits. Finally ask why the training modification works. Table 3 provides a useful but narrower answer: keeping all targets while switching from all-denoise to hybrid routing changes the standard LIBERO outcome, and motion removal causes the largest omission penalty. That does not prove a particular geometric or semantic robustness mechanism, because component ablations are not reported under the perturbations. The real robot corroborates improved execution under visual changes, but its robustness evidence concerns only Strawberry Selection, with thirty trials per condition. e-pluse-sim-protocole-ablatione-real-resultse-real-datae-real-tasks

5.4 Training and inference

During training

Source description

The loss combines RGB/motion/action flow matching, geometry/semantic prediction and L1 gate regularization. Auxiliary objectives update the shared video backbone; teacher encoders and the target VAE are frozen. e-objectivee-targets

Source description

Matched simulation variants use Wan2.2-5B, eight NVIDIA H20 GPUs, bfloat16, learning rate 1×10⁻⁵ and batch size 16. Only original LIBERO demonstrations are used. Real-world models share 1,600 pooled demonstrations in LeRobot v2.1 format and train for five epochs. e-sim-protocole-real-data

During inference

Source description

DreamWAM jointly denoises future RGB and actions; DreamWAM-uncond keeps only the first-frame video input and denoises actions without video rollout. The 32-step action policy executes ten steps before replanning, using ten denoising steps. e-sim-protocole-objective

Source description

Flow input channels are zero-filled; RAFT, DA3, DINOv2, residual branches and prediction heads are not executed. Training also zeros conditioning-frame flow, but applies flow loss to future steps. The control output remains the action chunk. e-zeroe-objective

5.5 Implementation flow

  1. Construct aligned future targets

    Frozen RAFT estimates eight adjacent flows from nine frames. Colorized flow duplicates its first transition to form nine frames; the frozen Wan2.2 VAE encodes it as 48×3×14×28. These are image-space motion features, not raw metric vectors. e-targets

  2. Compress geometry and semantics

    DA3-Base processes individual camera views before depth maps are reassembled, log-transformed and reduced by rank-8 PCA. Frozen DINOv2 ViT-B/14 with registers supplies normalized 768-dimensional patch features, also reduced to eight dimensions. Both use training-only PCA, a 14×28 grid and temporal groups: frame 1, mean of 2–5, mean of 6–9. DINO features receive channel normalization. e-targets

  3. Route according to representation

    RGB and VAE-encoded flow enter channel-concatenated denoising. Geometry and semantics instead produce gated additive corrections at selected video layers, preserving the original block output. Their teachers supply targets rather than policy observations. e-viewse-routing

  4. Transfer future structure into actions

    Separate video and action experts contribute tokens to shared attention, then apply their own projections and feed-forward transformations. ActionDiT denoises actions using this shared representation without directly consuming teacher features. e-coupling

6. Experiments & results

DreamWAM trains coupled video and action experts to anticipate appearance, motion, geometry and semantics, while retaining RGB-only deployment. Its strongest evidence is improved executed-task success under unseen perturbations, including when future video rollout is disabled. Routing heterogeneous targets matters as much as adding them.

6.1 Read the original evidence

Table 1. Standard LIBERO gains are modest, and individual suites can move in opposite directions. Original paper, p. 5 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Read the bottom four rows as two comparisons. Fast-WAM and DreamWAM-uncond use only the first-frame video input without future video rollout; Fast-WAM-Joint and DreamWAM jointly denoise future RGB and actions. The four suite columns each summarize ten tasks, with fifty rollouts per task and two seeds for the paper's implementations. Underlining marks the better no-rollout score, while bold marks the better joint score. The other methods provide benchmark context, but their backbones and pretraining differ. Start with Avg., then inspect Goal and Object to see where average improvements hide small suite-level reversals. e-liberoe-sim-protocol

What it supports. Mean success increases from 97.30% to 98.40% without rollout and from 98.00% to 98.90% with joint inference. The latter includes a Long-suite increase from 95.60% to 97.60%. These results show that the modified training remains effective on the original tasks, while leaving little room for large average gains.

Where the evidence stops. The joint Goal score decreases from 98.80% to 98.60%, and the no-rollout Object score decreases from 99.80% to 99.60%. Two-seed means without dispersion do not establish whether these small differences are statistically reliable.

Table 2. Structured future supervision improves every reported perturbation dimension in both matched inference modes. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Follow each matched pair across the seven perturbation columns before reading Avg. Cam. changes viewpoint, Robot changes robot appearance, Lang. changes instructions, and the remaining columns cover light, background, image noise and object layout. Avg. gives each dimension equal weight rather than pooling all episodes. Each seed has 10,030 evaluation episodes, and training uses only original LIBERO demonstrations. The Emb. PT column distinguishes embodied pretraining; the matched WAM rows are marked absent. Those shared-backbone comparisons carry more evidential weight than ranking heterogeneous VLA and WAM systems. Bold and underlining mark the overall best and second-best entries. e-pluse-sim-protocol

What it supports. Without video rollout, the mean rises from 51.36% to 63.44%; with joint RGB–action inference, it rises from 69.16% to 75.47%. The benefit therefore survives removal of online future video generation. Joint inference still scores higher within DreamWAM, supporting a complementary role for learned supervision and test-time imagination.

Where the evidence stops. Perturbations preserve the underlying manipulation skills. These results do not test learning new skills. Printed averages imply a 12.08-point no-rollout gain, while the prose says 12.09; the source does not resolve the rounding difference.

Figure 3. Real-robot improvements extend to unseen visual changes on Strawberry Selection. Original paper, p. 7 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Use the numeric labels above the gray and purple bars to compare methods; the plot provides no numbered vertical ticks. The left group covers dish stacking, two-object picking, strawberry selection and block stacking. The right group changes only Strawberry Selection: lighting, colored background patches or irrelevant distractor objects, labeled Layout. Each setting has thirty trials per model. Below, the photographs show task configurations and the types of perturbation; the final image illustrates a baseline gripper-to-plate alignment error. Appendix B defines success as completing the required sequence with stable final object placements, and computes standard-task and perturbation averages separately. e-real-resultse-real-datae-real-tasks

What it supports. DreamWAM improves all seven settings. Its standard-task average is reported as 96.7%, versus 90.8% for Fast-WAM-Joint; the perturbation averages are 74.4% and 55.6%. Under the Layout/distractor condition, the displayed scores are 76.7% and 50.0%. These are executed manipulation outcomes on an AgileX PiPER platform.

Where the evidence stops. The perturbations are tested separately on one task, with no additional adaptation. Layout here means added irrelevant objects, not a documented rearrangement of target fruits. Thirty trials and a single failure photograph cannot isolate which auxiliary target caused the improvement.

6.2 Results and evaluation conditions

Task & protocolReported resultComparison & interpretation
LIBERO matched policy success

Four ten-task suites; 50 rollouts/task, 2,000/seed; two seeds; original demonstrations only.

DreamWAM-uncond 98.40; DreamWAM joint 98.90.

Mean success rate (%)

Fast-WAM no-rollout 97.30; Fast-WAM-Joint 98.00.

Small mean gains near ceiling; joint Goal falls from 98.80 to 98.60, so improvement is not universal across suites. e-sim-protocole-libero

LIBERO-Plus matched robustness

Seven unseen perturbation dimensions; 10,030 episodes/seed, two seeds; no perturbation training data.

DreamWAM-uncond 63.44; DreamWAM joint 75.47.

Unweighted mean of seven dimension success rates (%)

Matched Fast-WAM 51.36 and Fast-WAM-Joint 69.16.

All seven columns improve in each pair. From printed averages, gains are 12.08 and 6.31 percentage points. Prose gives 12.09 for no-rollout; underlying rounding is unresolved. e-sim-protocole-plus

Real-world standard manipulation

AgileX PiPER; four tasks, 30 trials each; matched joint models trained on the same demonstrations.

DreamWAM 96.7.

Arithmetic mean task success (%)

Fast-WAM-Joint 90.8.

All four plotted task scores improve; success requires the complete sequence and stable final placements. e-real-resultse-real-datae-real-tasks

Real-world Strawberry Selection robustness

Lighting, background and distractor/layout changes separately; 30 trials each; no adaptation or perturbed demonstrations.

DreamWAM 74.4.

Arithmetic mean perturbation success (%)

Fast-WAM-Joint 55.6.

Evidence concerns three visual shifts on one trained task, not new skills or combined perturbations. e-real-resultse-real-datae-real-tasks

LIBERO component and routing ablation

Matched joint training; all variants retain RGB; two seeds under the standard simulation protocol.

Hybrid D/R/R 98.90.

Mean success rate (%)

RGB only 98.00; all-denoise D/D/D 97.85; without motion 97.50, geometry 98.25, semantics 98.60.

Compatible routing matters; adding all targets through denoising underperforms RGB only. Motion removal hurts most, but these near-ceiling results do not establish perturbation-specific causal mechanisms. e-ablatione-sim-protocol

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Table 3. The same auxiliary targets can help or hurt depending on how they enter VideoDiT. Original paper, p. 6 ↗

Excerpt from the authors’ paper; cropped without altering the figure or table.

How to read it. Decode the route columns first: Mot., Geo. and Sem. mean motion, geometry and semantics; a dash means absent, D means full denoising through channel concatenation, and R means gated residual injection. Every row retains RGB prediction. The paired single-view rows hold the target fixed while changing its route. The leave-one-view-out rows test the complete model's ingredients. Finally, compare Denoise with Hybrid: both use all three targets, but route them as D/D/D versus D/R/R. Read suite scores after the average, since the best overall route does not win every column. Bold and underlining indicate best and second-best values, including ties. e-ablatione-sim-protocole-routing

What it supports. Motion alone favors D over R, scoring 98.50% versus 97.85%; geometry and semantics favor residual routing. With all targets present, D/R/R reaches 98.90%, compared with 97.85% for D/D/D and 98.00% for RGB only. The measured advantage cannot be explained simply by counting auxiliary targets.

Where the evidence stops. These are standard LIBERO ablations, not LIBERO-Plus or real-robot component studies. Routing also changes how the backbone is optimized. Without dispersion or direct representation diagnostics, the table supports a design choice more strongly than a specific causal account of robustness.

7. Analysis & limitations

7.1 What the evidence leaves open

Source description

These targets are correlated projections, not complete physical state: motion is colorized 2D flow; depth is compressed; semantics are features, not labels. Metric 3D and contact-aware targets remain future work. e-targetse-boundaries

Reader analysis

Simulation tables give two-seed means without dispersion; real settings have 30 trials and no confidence intervals. Heterogeneous external baselines are contextual, while matched Fast-WAM comparisons provide stronger controls. The single pictured failure cannot isolate causation. e-sim-protocole-liberoe-pluse-real-results

Reader analysis

Figure 2 shows separate feature-output additions, whereas Eq. (3) writes both predictors from h_L; exact head wiring remains unclear. Its five-loss summary also omits the gate penalty explicitly included in Eq. (4). e-routinge-objective

7.2 Questions for discussion

  1. Does the routing advantage persist under unseen perturbations with more seeds?
  2. How much robustness survives temporal misalignment of motion targets?

8. Reproducibility audit

8.1 Requirements and known gaps

Open question

Recover loss weights, feature-loss definitions, selected residual layers, gate initialization and simulation training duration before claiming a faithful reproduction. The source also lacks a complete optimizer/scheduler specification and measured deployment latency. e-gaps

Reader analysis

Proposed checks: repeat D/R/R versus D/D/D with identical targets and seeds under LIBERO-Plus; separately compare both inference modes with auxiliary computation disabled. Report task-level outcomes and uncertainty, not only averages. e-ablatione-pluse-zero

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Does hybrid routing improve robustness when supervision is held fixed?

Reader-proposed experiment, not performed: train RGB-only, D/D/D and D/R/R variants with identical original LIBERO demonstrations, target caches, initialization seeds and action settings. Recover the unspecified loss weights and branch configuration first, and record any unavoidable parameter or compute differences. Use at least five paired seeds, then evaluate both standard LIBERO and all seven LIBERO-Plus dimensions using the same episodes. Report task-level success and paired uncertainty. If D/R/R only improves near-ceiling standard scores, or its perturbation advantage vanishes across seeds, the interpretation that routing explains robust action learning would be weakened. e-ablatione-sim-protocole-pluse-gapse-targets

Check 2: Does the learned benefit survive removal of every auxiliary runtime path?

Reader-proposed experiment, not performed: evaluate matched RGB-only and DreamWAM checkpoints in both no-rollout and joint inference, holding initial states, action horizon, executed steps and denoising steps fixed. Assert that RAFT, DA3, DINOv2 and residual heads are never called and that flow channels remain zero. Numerically compare the zero-padded input projection against its RGB-only partition, then record success, latency and memory for each mode. The claim predicts a DreamWAM advantage even without future video rollout. A gain that requires live teacher features would contradict the stated deployment mechanism; joint-only gains would weaken the claim of internalized benefits. e-zeroe-objectivee-sim-protocole-plus

8.3 Reading coverage

Visual audit: Visually inspected the title/authors/version on p. 1; Figure 1 on p. 2; method and equations on pp. 3–5; Tables 1–3 on pp. 5–6; Figure 3 and ablation discussion on p. 7; target construction, inference algebra and training/evaluation protocol on p. 10; and task definitions, perturbations and success criterion on p. 11. All five final original crops were viewed and checked for complete labels, headers and legends. Figure 2 was cross-checked against Eqs. (3)–(4): the separate feature-output additions and omitted schematic gate-loss term are disclosed. Tables have no separate footnotes; caption-defined notation and protocols are explained in the guides. All five text chunks, including references and appendices, were read; reference pages 8–9 were not visually inspected. Separate supplements were not supplied or verified; no code or experiments were run.

PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 10, 11. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions
  • Abstract and Introduction (pp. 1–2)
  • Related Work (pp. 2–3)
  • Method, including training and inference (pp. 3–5)
  • Experimental Setup and Results (pp. 5–6)
  • Ablation Study and Conclusion (pp. 6–7)
  • References (pp. 8–9)
  • Appendix A.1–A.2: target construction and inference (p. 10)
  • Appendix B.1–B.4: real-world data, tasks, perturbations and success (pp. 10–11)

Outside the original text pass

  • Verified edition: arXiv:2608.04996v1, 5 August 2026. Title and all nine authors agree with the catalog. No alternative revision was supplied or compared.
  • Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
  • The extraction limitation was addressed by visually inspecting Figures 1–3, Tables 1–3, and all pages supporting retained scientific claims. Reference pages 8–9 were read as text only.
  • Separate supplemental material availability has not been fully verified.
  • Code, linked project resources and models were not inspected; no experiments were reproduced.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e-identityPDF p. 1, title block and arXiv margin stampInspect

The title and all nine authors match the catalog; the supplied edition is arXiv:2608.04996v1, 5 August 2026. Four institutions are credited.

Go to primary source ↓
e-problemPDF pp. 1–2, Introduction and Figure 1/captionInspect

The authors motivate appearance, motion, geometry and semantics using manipulation under visual shifts; the pictured rollouts are qualitative examples.

Go to primary source ↓
e-viewsPDF p. 3, Method: Preliminaries, Structured Future Views, Eqs. (1)–(2)Inspect

Language and observations condition coupled video/action learning. RGB and motion latents share a denoising timestep and are concatenated by channel; geometry and semantics are training targets.

Go to primary source ↓
e-routingPDF p. 4, Figure 2/caption and Gated residual modeling, Eq. (3)Inspect

Gated view-specific residuals add to the original block output; the equation predicts both feature views from the final shared state. The schematic draws separate feature-output addition nodes.

Go to primary source ↓
e-couplingPDF p. 4, World-Action CouplingInspect

VideoDiT and ActionDiT retain separate parameters, sharing attention context at paired layers before separate projections and feed-forward transformations.

Go to primary source ↓
e-objectivePDF p. 5, Training Objective and Inference, Eq. (4)Inspect

The objective weights RGB, motion, geometry, semantics, action and an L1 gate-activation penalty. Non-RGB branches and teachers are inactive at deployment.

Go to primary source ↓
e-sim-protocolPDF p. 5, Experimental Setup: SimulationInspect

Original LIBERO training only; four ten-task suites, 500 demonstrations per suite, two seeds, 2,000 standard and 10,030 perturbed episodes per seed. Matched models use Wan2.2-5B and the same action policy and training setup.

Go to primary source ↓
e-liberoPDF p. 5, Table 1, matched four rows, suite and Avg. columnsInspect

No-rollout averages are 97.30/98.40 and joint averages 98.00/98.90 for Fast-WAM/DreamWAM; some individual suites do not improve.

Go to primary source ↓
e-plusPDF p. 6, Table 2, matched WAM rows, seven perturbation columns and Avg.; LIBERO-Plus discussionInspect

Matched no-rollout averages are 51.36/63.44 and joint averages 69.16/75.47. Both pairs improve in all seven columns; prose reports a 12.09-point no-rollout gain.

Go to primary source ↓
e-ablationPDF p. 6, Table 3, all rows; p. 7, Ablation Study continuationInspect

D denotes full denoising and R gated residual injection. RGB only scores 98.00, D/D/D 97.85 and D/R/R 98.90; paired single-view and omission controls test routing and complementarity.

Go to primary source ↓
e-real-resultsPDF p. 7, Figure 3/caption; p. 6, Real-World Robot resultsInspect

Four standard-task bars and three perturbation bars compare the two joint models. Reported means are 90.8/96.7 and 55.6/74.4; a qualitative failure concerns gripper-to-plate alignment.

Go to primary source ↓
e-targetsPDF p. 10, Appendix A.1, Eqs. (5)–(7)Inspect

Frozen RAFT and Wan VAE encode colorized flow as 48×3×14×28. DA3-Base depth and DINOv2 ViT-B/14 with registers are reduced with training-only PCA and aligned as 8×3×14×28.

Go to primary source ↓
e-zeroPDF p. 10, Appendix A.2, Eqs. (8)–(10)Inspect

Zero flow channels contribute nothing through the input projection. Conditioning-frame flow is also zero during training; future latent steps receive the flow objective. Teachers, residual branches and heads are not executed at deployment.

Go to primary source ↓
e-real-dataPDF p. 10, Appendix B.1, Eq. (11); p. 5, Experimental Setup: Real robotInspect

AgileX PiPER dual-arm evaluation uses 400 demonstrations per task, 1,600 total, LeRobot v2.1 and five epochs. Each model receives 30 trials per setting, 210 overall; standard and perturbation means are separate.

Go to primary source ↓
e-real-tasksPDF pp. 10–11, Appendix B.2–B.4Inspect

Four tasks specify ordered placement/stacking. Strawberry perturbations separately change colored background patches, fill lighting and irrelevant distractor objects. Complete sequences and stable final placements are required; there is no partial credit.

Go to primary source ↓
e-boundariesPDF p. 3, Structured Future Views; p. 7, ConclusionInspect

Future views are neither a disentangled nor complete physical state. Metric 3D and contact-aware targets are proposed future work.

Go to primary source ↓
e-gapsPDF pp. 4–5, Method and Experimental Setup; p. 10, Appendix AInspect

The supplied implementation description gives losses, target construction and basic training settings but does not specify loss weights, exact residual-layer placement, gate initialization, feature-loss formulas or simulation training duration.

Go to primary source ↓

8.5 Primary sources

Scroll across the image to inspect details. Press Esc to close.