Vid2WAM: Distilling Video Diffusion Priors into World Action Models
1. Paper overview
In one sentence: Vid2WAM moves video generation and inverse dynamics into offline supervision, improving a student's action policy while leaving difficult held-out tasks far from solved. e02e03e05e07e11e18
| At a glance | What to know |
|---|---|
| Research problem | Author claim Future-supervised robot policies usually need costly paired robot videos and actions. The authors ask whether generated futures can replace some target-task expert trajectories without imposing video-generation latency during control. Their concern is that transferring only recovered actions discards predictive visual information and inherits generation and inverse-dynamics errors. e02 |
| Core mechanism | Source description Two complementary offline targets connect a large video foundation model to a smaller WAM: student-compatible future latents and embodiment-specific pseudo-actions. e03 |
| A key reported result | RoboTwin held-out-task manipulation: 54.7 clean; 55.3 randomized Mean task success (%). Novel regime: 35 seen/15 held-out tasks; 100 trials per task per clean/randomized condition. Target instructions and initial observations seed synthetic training. Fast-WAM: 45.0/42.8; Motus: 48.9/51.5; π0.5: 46.5/46.3. Gains over Fast-WAM are 9.7/12.5 percentage points. These are the held-out subset; the corresponding 50-task averages are 78.3/78.5. Equal real demonstration budgets do not mean equal synthetic-data exposure. e06e07e15e21 |
| Reading caution | Reader analysis Generated motion and IDM labels can be physically inconsistent. IDM validation randomly splits frame-level samples and counts a sample correct only when every target dimension is within 0.05. Correlated windows may make validation optimistic; these scores do not measure pseudo-label accuracy on unseen generated videos. e02e14 |
Core contributions
- Source description
Two complementary offline targets connect a large video foundation model to a smaller WAM: student-compatible future latents and embodiment-specific pseudo-actions. e03
Figure 2. Generated futures supply video targets and recovered action targets before the teacher is removed. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin at the lower left: a current or generated initial frame and the language instruction condition the frozen video teacher. Its rollout feeds the IDM for pseudo-actions and supplies the visual-target route toward the student. Follow the four loss labels: real and pseudo supervision each have a video term and an action term. Blue tokens represent video information, yellow tokens actions, and gray tokens noise. The flame and snowflake legend distinguishes training from freezing. At the right, only current-frame video features and an action model remain. Use the caption for this inference interpretation; the full action-adapter routing is specified in Eq. (3) and Table S7. e03e04e05e14e17
What it supports. The teacher contributes task-progress supervision without sitting in the deployed control loop. The student learns from both generated futures and recovered actions, while real demonstrations anchor action learning. Source-specific residual paths are small corrections around shared action computation, and the real-domain path remains available during inference.
Where the evidence stops. The frozen visual-side 'adapter' is not fully defined by the diagram. Appendix A describes RGB decode/re-encoding; Eq. (3) concerns trainable action adapters. The inference panel also omits the real adapters retained in Table S7. Treat these as schematic omissions, not evidence that every adapter is frozen or removed.
2. Motivation
2.1 The problem and the proposed response
Future-supervised robot policies usually need costly paired robot videos and actions. The authors ask whether generated futures can replace some target-task expert trajectories without imposing video-generation latency during control. Their concern is that transferring only recovered actions discards predictive visual information and inherits generation and inverse-dynamics errors. e02
2.2 What this reading follows
A video model can imagine a manipulation sequence without knowing the robot commands that execute it. Vid2WAM uses that separation to manufacture two training signals: generated future images for a world-model objective, and inverse-dynamics labels for an action objective. The deployed student then acts from its current observation without querying either generator. Read the architecture and ablation together to understand why both channels matter. Then separate the evaluation populations: RoboTwin reports a substantial held-out-task gain, whereas LIBERO's high novel-regime average mostly describes a benchmark that still contains seen tasks. The appendix's held-out-only table provides the necessary boundary. e02e03e05e07e11e18
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Insufficient evidence to decide
WAM, generalization and efficient inference are supported. The architecture explicitly contains a video backbone and separate ActionDiT, so compact student deployment alone does not establish 'One Model'. 'Dual-system' can describe these components, but not an online teacher/IDM cascade. Training jointly predicts futures/actions; deployment uses auxiliary learned world features and direct actions, with IDM confined to offline labeling. 'Other mechanisms/Outside quadrants' is defensible for that deployment path, but ambiguous without the catalog's train-versus-inference criterion. e03e04e05e12e17
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Translate the teacher's imagination into the student's language
Start from an observation and a task instruction, not a target-task action trajectory. The adapted video teacher synthesizes future frames. One route decodes those predictions to RGB and re-encodes them with the student VAE, because the two models use different latent channel counts. This produces direct future-video targets in the representation the student actually predicts. A second route applies embodiment-specific IDMs to recover action and state targets from short temporal windows. The resulting offline buffer stores both forms of supervision. The distinction matters because inverse dynamics is imperfect: all of the teacher's motion information need not survive conversion into action labels. The future-video loss lets generated scene progression reach the student without passing exclusively through that action-recovery bottleneck. e02e03e13e14
5.2 Let source-specific residuals correct a shared policy
Real actions and recovered pseudo-actions supervise the same action backbone, but they need not have the same error distribution. Vid2WAM places separate real and pseudo adapters at the input and output of the action network. Each adds a bottleneck residual; zero-initializing its up-projection makes the initial mapping an identity. The shared path therefore remains active while source-specific corrections are learned. Flow matching supervises both actions and future latents for both sources, with the pseudo-domain objective weighted by 0.25 and the real-domain objective by 1. Each update still contains 128 samples from each domain. Equal batch counts and unequal objective weights are different controls. The ablation compares these residual corrections with both unrestricted source mixing and much larger separate action networks. e04e11e16e17
5.3 Follow what survives deployment, then choose the right success denominator
At deployment the learned representation survives, while the teacher, IDM, future-video prediction head and pseudo-source adapters do not. The retained video backbone processes the current frame and supports direct action generation through the real-source path. This explains why qualitative future predictions can improve without requiring those videos inside every control step. It is consistent with the reported 212.5 ms chunk latency, close to Fast-WAM's 209.1 ms. Reader interpretation: this is efficient transfer of training supervision, not evidence of online search over imagined outcomes. Its generalization claim must then be checked on the correct population. RoboTwin separates held-out tasks explicitly; LIBERO's main novel-regime average includes seen tasks. The held-out-only appendix is essential for judging how much new behavior the deployed policy actually acquires. e03e05e07e12e17e18
5.4 Training and inference
During training
Teachers train for approximately 500 epochs; IDMs for 200,000 steps. Each student update processes 128 real and 128 pseudo samples. Training lasts 10 epochs for LIBERO/LIBERO-Plus and real-world, 5 for RoboTwin. Loss weights are β=1, λ_real=1 and λ_pseudo=0.25. e13e14e16
Table S5 specifies 400 expert/400 synthetic trajectories for LIBERO low-data; 1,366/312 for its novel regime; 1,000/1,000 for RoboTwin low-data; and 19,250/8,250 for RoboTwin novel. Unlike the main text's shorthand '10 per task', RoboTwin low-data includes 10 clean plus 10 randomized demonstrations per task. Teacher/IDM adaptation excludes held-out trajectories. e06e15
During inference
A single current-frame pass through the shared video backbone produces features for the action model. Remove the external teacher, IDMs, auxiliary future-video head and pseudo-source adapters; retain the real action adapters. This is direct chunk prediction, with no online teacher rollout, inverse-dynamics recovery or future-based search. The source leaves the executed chunk length and feedback/replanning cadence unspecified. e03e05e17
5.5 Implementation flow
- Build an embodiment-adapted teacher
Fine-tune the LVP-initialized Wan2.1-I2V-14B diffusion transformer on regime-specific robot videos. Freeze its text/image encoders and VAE, then freeze the adapted teacher for offline generation. e03e13e15
- Translate generated futures into two targets
Decode teacher rollouts to RGB and re-encode with the frozen student VAE: teacher and student latents have 16 and 48 channels. Separately trained action and state IDMs label the videos. Each ResNet-50 IDM processes three temporally spaced RGB frames and learns normalized targets with Smooth L1 loss. e03e13e14
- Mix sources through residual action paths
Real and pseudo samples share video and action backbones, but use separate input/output action adapters. Zero-initialized up-projections start these residual paths at identity. The student video backbone starts from Wan2.2-TI2V-5B; the adapter bottleneck is 128 with GELU and residual scale 1. e04e16e17
- Handle incomplete synthetic observations
Real-world pseudo-rollouts supply only an overhead view. In a 448×448 composite, absent wrist slots are black-padded; video loss covers only the top half of the 28×28 latent grid. Padding masks exclude invalid temporal/action positions. Simulation rollouts use the first 8 seconds; real-world rollouts retain 20 seconds. e20
6. Experiments & results
Vid2WAM trains a robot policy from both expert demonstrations and offline video-teacher rollouts, transferring generated futures directly into a student video objective and through inverse dynamics into action labels. Source-specific action adapters accommodate the two supervision sources. Deployment uses current-observation features and direct action prediction, without online video generation. RoboTwin transfer improves, but held-out LIBERO success remains much lower than its full-benchmark averages (e03–e12, e18).
6.1 Read the original evidence
Figure 3. Matched starting observations expose differences in predicted task progression. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task independently from left to right, comparing the Fast-WAM row above with Vid2WAM below. The left pair follows the instruction to place both loaves in a breadbasket; the right pair follows mouse placement on a display stand. The paper states that each method receives the same initial observation and instruction. Attend to whether motion continues toward the requested placement after an intermediate grasp, rather than judging one isolated image. These are the student models' predicted future videos used for qualitative comparison. They illustrate the representation learned through future supervision, even though the deployed action path does not generate these sequences. e03e05e08
What it supports. The authors interpret the Vid2WAM rows as more coherent progress toward the requested outcome. The comparison makes the purpose of direct future-latent supervision concrete: it preserves a signal about how the scene should develop, beyond the individual robot commands recovered by inverse dynamics. This is illustrative evidence for the learned future-prediction behavior.
Where the evidence stops. The examples are selected predictions, not physical executions or a measured video-quality benchmark. They cannot establish that an imagined grasp is executable or explain how often the pattern occurs. Task success must be assessed from the separate rollout evaluations.
Table 1. RoboTwin's held-out subset improves while the real demonstration budget is fixed. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the three column groups before comparing rows. Low-data tests learning with scarce demonstrations. Novel regime averages all 50 tasks, combining 35 seen tasks with 15 held-out tasks. Novel subset isolates those 15 targets and is the appropriate group for the transfer claim. Every entry is a success percentage; clean and randomized evaluations use 100 trials per task. The caption supplies the population distinction, while Table S5 supplies the training budgets. Its RoboTwin low-data entry specifies 10 clean plus 10 randomized demonstrations per task, refining the main text's shorter description. Vid2WAM additionally receives teacher-generated supervision from target instructions and initial observations. e06e07e15e21
What it supports. On held-out targets, Vid2WAM reports 54.7% clean and 55.3% randomized success, versus Fast-WAM's 45.0% and 42.8%. The gains are 9.7 and 12.5 percentage points. The larger 78.3% and 78.5% numbers belong to the full novel-regime benchmark, where seen tasks also contribute.
Where the evidence stops. The comparison holds downstream real action-labeled data fixed, but adds synthetic target-conditioned training to Vid2WAM. It therefore tests the complete adaptation method. It does not isolate adapters from extra data, and aggregate superiority does not imply improvement on every task.
Table S12. The held-out-only table reveals both gains and the remaining generalization gap. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each left-hand ID pair identifies two excluded tasks within every suite, not two tasks total. The corresponding block compares Fast-WAM and Vid2WAM on those withheld tasks alone. Read the Spatial, Object, Goal and Long columns before the average: the strongest 0/1 gains occur in Spatial and Object, while Goal and Long remain zero. The main paper's split is the bottom 8/9 block. For that block, distinguish the appendix's 2.3% held-out average from Table 3's 78.3% average over the complete LIBERO benchmark. Tables S8 and S9 give the task instructions associated with each suite-local ID. e06e18e19
What it supports. Vid2WAM improves all three held-out averages: 38.3% versus 7.5% on 0/1, 7.0% versus 0.0% on 3/6, and 2.3% versus 0.0% on 8/9. The sign of improvement survives these split changes, but its size and absolute usefulness vary substantially. Difficult held-out tasks remain largely unsolved.
Where the evidence stops. Positive improvements across three selected splits do not establish broad task-independent generalization. The main split is particularly weak in absolute terms, and several cells remain zero. The table reports point estimates without training-seed variability or confidence intervals.
Table 4. Physical trials support limited novel-task transfer on a bimanual robot. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the horizontal divider to separate seen tasks trained with real trajectories from the three novel tasks trained through generated supervision. Every cell summarizes 20 real-robot trials for that model and task. Pick Test Tube requires removing a tube already in its rack; Take Tissue requires extracting a deformable tissue; Tomato Basket requires placing a tomato in the basket. The setup uses two AgileX Piper arms and three camera views. For each novel task, two true overhead observations seed 60 synthetic conditioning images, with pseudo-video supervision restricted to that view because generated views are not mutually aligned. Genuine multi-view samples retain their full supervision. e06e10e15e19e20
What it supports. Vid2WAM reaches 30%, 15% and 30% on the novel tasks, exceeding Fast-WAM's 5%, 5% and 25%. This is evidence about executed actions, unlike the earlier predicted-video comparison. The seen-task rows also prevent a blanket dominance claim: Motus achieves 70% on Click Bell compared with Vid2WAM's 65%.
Where the evidence stops. Only 20 trials underlie each percentage, and the table does not report confidence intervals or training-seed variability. The novel-task rates remain low. Synthetic overhead augmentation and the particular tabletop setup also limit conclusions about broader physical deployment.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin held-out-task manipulation Novel regime: 35 seen/15 held-out tasks; 100 trials per task per clean/randomized condition. Target instructions and initial observations seed synthetic training. | 54.7 clean; 55.3 randomized Mean task success (%) | Fast-WAM: 45.0/42.8; Motus: 48.9/51.5; π0.5: 46.5/46.3. Gains over Fast-WAM are 9.7/12.5 percentage points. These are the held-out subset; the corresponding 50-task averages are 78.3/78.5. Equal real demonstration budgets do not mean equal synthetic-data exposure. e06e07e15e21 |
| LIBERO and LIBERO-Plus aggregate manipulation Four suites; LIBERO uses 50 trials/task. LIBERO-Plus uses 500 variants/suite across seven perturbation factors. Main novel split withholds suite-local IDs 8/9. | LIBERO: 89.7 low-data, 78.3 novel regime. LIBERO-Plus: 39.0 low-data, 48.1 novel regime. Reported overall success (%) | Fast-WAM: LIBERO 87.3/76.7; LIBERO-Plus 33.6/45.8. π0.5 reaches 77.9 on LIBERO novel. Novel-regime aggregates include seen tasks; gains are not uniform across suites or perturbations. e06e09e18 |
| Held-out LIBERO and LIBERO-Plus targets Only the eight withheld tasks; main IDs 8/9 in each suite. | LIBERO 2.3; LIBERO-Plus 4.8 Average success (%) | Fast-WAM: 0.0 and 1.0. Other LIBERO splits give Vid2WAM 38.3 for IDs 0/1 and 7.0 for IDs 3/6. Adaptation improves the baseline but remains weak and strongly dependent on task selection; 78.3% aggregate success is not held-out-task success. e18 |
| Real-world novel bimanual tasks Two AgileX Piper arms and three D435 views; 20 trials/model/task. Six seen tasks supply 360 demonstrations; three novel tasks supply initial views and 180 generated rollouts. | Pick Test Tube 30; Take Tissue 15; Tomato Basket 30 Task success (%) | Fast-WAM 5/5/25; Motus 10/0/20; π0.5 0/0/0. These are executed robot outcomes. Small trial counts and absent confidence intervals limit precision; reliable novel-task execution is not established. e06e10e15e19 |
| LIBERO low-data supervision and adapter ablation Same teacher/IDM buffers and training hyperparameters across student variants. | Full model 89.7 Average success (%) | No Adapter 88.5; Dual Action DiT 89.1; Pseudo Action Only 86.4; Future Latent Only 88.2; Fast-WAM 87.3; online Teacher Policy 71.3. Supports complementary visual/action supervision and source adaptation. Dual Action DiT adds around 1B training parameters versus around 1M for adapters, so that comparison also changes capacity. e11e14e17 |
| Chunk-level inference latency Single RTX 4090; cold-start costs excluded. | 212.5 mean; 11.7 standard deviation Latency in milliseconds, mean and standard deviation | Fast-WAM 209.1; Teacher Policy 4894.5; Motus 2414.9; π0.5 76.7 (means). Student distillation preserves approximately Fast-WAM latency. This is chunk computation latency, not demonstrated end-to-end control frequency or training cost. e12 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. Future-latent supervision and action supervision complement one another in the reported ablation. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the average column, then check suite-specific exceptions. No Adapter uses the shared video and action paths for both sources without source-specific modules. Dual Action DiT separates real and pseudo action networks while sharing the video expert. Pseudo Action Only discards the future-latent loss; Future Latent Only removes pseudo-action supervision while keeping real action anchoring. The appendix says these student variants use the same teacher, IDM, buffers and training hyperparameters. The separated Teacher Policy row is different: it executes actions recovered from online teacher rollouts. It tests the online cascade, rather than another small modification of student training. e04e11e14e17
What it supports. The full student reaches 89.7%, compared with 88.5% without adapters and 89.1% with separate action networks. Future Latent Only reaches 88.2%, versus 86.4% for Pseudo Action Only. These results support retaining visual supervision and combining it with actions. The online Teacher Policy's 71.3% shows that successful distillation does not require a stronger online teacher controller.
Where the evidence stops. The table supplies no seed uncertainty. Dual Action DiT changes training capacity by around 1B parameters, versus around 1M for adapters. Also, Pseudo Action Only removes the future-latent loss broadly, so it does not cleanly isolate the synthetic visual term alone.
7. Analysis & limitations
7.1 What the evidence leaves open
Generated motion and IDM labels can be physically inconsistent. IDM validation randomly splits frame-level samples and counts a sample correct only when every target dimension is within 0.05. Correlated windows may make validation optimistic; these scores do not measure pseudo-label accuracy on unseen generated videos. e02e14
The held-out exclusion concerns regime-specific adaptation, not a verified audit of foundation-model pretraining. Vid2WAM also receives target-conditioned synthetic supervision while transfer baselines do not. Its results establish this complete adaptation recipe, not an equal-total-data comparison. e06e13e15
Figure 2 marks a visual-side 'adapter' frozen and omits retained adapters in its inference panel. Eq. (3) and Table S7 specify trainable source-specific action residuals and real adapters at inference; Appendix A specifies RGB decode/re-encode. The frozen visual box is not fully explained and must not be conflated with the action adapters. e04e05e14e17
7.2 Questions for discussion
- Do adapters still help when pseudo-label noise and synthetic-data quantity are controlled independently?
- How much held-out-task improvement survives trajectory-level IDM validation and matched target-state coverage?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs the named teacher/student checkpoints, regime-specific demonstration lists, identical initial-state pools and synthetic buffers, and the camera layouts/masks. Student training uses BF16 AdamW, learning rate 10⁻⁴ in simulation and 5×10⁻⁵ on the robot. Real novel-task augmentation uses 30 images from each of two initial observations. e13e15e16e20
Missing implementation details include numerical IDM frame strides, student action sampling steps, action horizon/execution cadence, training hardware/runtime, software versions and uncertainty across training seeds. Teacher rollout sampling is specified separately as 40 steps per 49-frame chunk; that must not be substituted for student sampling. e03e12e13e14e16
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate source routing from added capacity and pseudo-label noise
Reader-proposed experiment, not performed: reproduce the LIBERO low-data setup with 400 real and 400 synthetic trajectories, one fixed teacher/IDM buffer and identical training schedules. Compare correct real/pseudo adapter routing against randomly permuted source routing with the same adapters and parameter count; include No Adapter as a reference. Repeat with controlled bias added only to pseudo-actions, leaving future latents unchanged, and use paired evaluation seeds. The source-correction hypothesis predicts that correct routing should resist increasing pseudo-action bias better than random routing. Equal performance across routes, or equal degradation as bias grows, would weaken that explanation even if the full method still beats Fast-WAM. e04e11e15e16e17
Check 2: Recompute generalization from disjoint task populations
Reader-proposed check, not performed: rerun the LIBERO 8/9 and 0/1 splits using the exact suite-local task mapping and excluding held-out trajectories from teacher/IDM adaptation. Compare Vid2WAM and Fast-WAM using identical real demonstrations and paired trial seeds; retain the method's explicitly additional synthetic target supervision. Record per-trial outcomes, then report seen-only, withheld-only and full-benchmark averages separately, with uncertainty across training seeds. The reported pattern predicts improvements on withheld targets but a much smaller absolute success rate for 8/9 than the full benchmark suggests. If gains arise only on seen tasks, the target-transfer interpretation fails; if full averages do not reconcile with their component populations, the evaluation bookkeeping needs correction. e06e15e18e19
8.3 Reading coverage
Visual audit: All 19 supplied PDF pages were rendered and visually inspected, including the p. 1 title/version block, every main and appendix figure/table, p. 3 equations, pp. 10–14 implementation and budget details, pp. 15–17 masking/task details, and pp. 18–19 task-level results. The six final original crops were individually viewed with labels and table rules intact. Figure 2's frozen visual adapter and omitted inference adapters were cross-checked against Eqs. (3)–(7), Appendix A and Table S7; their ambiguity is disclosed. No separate supplements, external videos, code or experiments were inspected beyond this PDF.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF pp. 1–2: title, abstract, introduction and related work
- PDF p. 3: problem formulation, teacher-derived targets, source-aware training and Eqs. (1)–(7)
- PDF pp. 4–7: experimental setup, simulation and real-world results, ablations, latency and conclusion
- PDF pp. 8–9: references
- Appendix A, pp. 10–14: teacher, IDM, student, budgets and ablation configurations
- Appendix B, pp. 11–15: novel-task splits, task mappings and withheld-task results
- Appendix C, pp. 14–17: real-world tasks, augmentation, spatial/temporal masking and duration cutoff
- Appendix D, pp. 17–19: complete RoboTwin task-level results
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Version scope: the inspected title page states arXiv:2608.08558v1 [cs.RO], 9 Aug 2026. Title and all ten authors match the supplied catalog; no other revision or edition was supplied for comparison.
- Acquisition caveat preserved: Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This reading resolves that visual gap by inspecting all 19 original PDF pages and six final crops.
- Separate supplemental material availability has not been fully verified.
- Code, linked resources and external pretraining datasets were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block and arXiv margin
Title matches the supplied observed title; ten authors and three affiliations are printed. Margin identifies arXiv:2608.08558v1 [cs.RO], 9 Aug 2026.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Introduction
The motivation contrasts demonstration-derived futures with generated supervision and identifies generation artifacts, inverse-dynamics ambiguity and online video-policy cost.
Go to primary source ↓e03PDF p. 3, Problem Formulation; Teacher-Derived Distillation Targets; Eqs. (1)–(2)
The student conditions on observation, proprioception and language; generated futures and IDM action/state labels form an offline buffer. Teacher, IDM, future head and pseudo adapters are discarded at deployment.
Go to primary source ↓e04PDF p. 3, Source-Aware Student Training; Eqs. (3)–(7)
Real/pseudo action adapters surround shared backbones, use zero-initialized up-projections and source-specific residuals. Four flow-matching terms supervise real/generated actions and video latents.
Go to primary source ↓e05PDF p. 4, Figure 2 and caption
Training diagram distinguishes frozen and trainable modules and both supervision channels. Inference caption specifies a single current-frame backbone pass. The visual-side adapter is drawn frozen; inference adapter details are omitted.
Go to primary source ↓e06PDF pp. 4–5, Experimental Setup; Evaluation Regimes; Baselines
Defines simulation trial counts, perturbation evaluation and equal downstream real demonstration budgets. Real setup has two Piper arms, three D435 cameras and six seen/three novel tasks. Novel target instructions and initial frames are permitted.
Go to primary source ↓e07PDF p. 5, Table 1, all columns and caption
RoboTwin success is reported separately for low-data, all 50 novel-regime tasks and the 15 held-out tasks. Vid2WAM held-out success is 54.7/55.3 versus Fast-WAM 45.0/42.8, clean/randomized.
Go to primary source ↓e08PDF p. 5, Figures 3–4 and captions; p. 6, RoboTwin results discussion
Figure 3 pairs Fast-WAM and Vid2WAM future predictions for bread placement and mouse placement. Figure 4 reports selected novel-task execution results. Selected generated trajectories are qualitative diagnostics.
Go to primary source ↓e09PDF p. 6, Tables 2–3, overall/average columns and suite/perturbation rows
LIBERO averages are 89.7/78.3 for Vid2WAM and 87.3/76.7 for Fast-WAM. LIBERO-Plus overall values are 39.0/48.1 and 33.6/45.8. Tables include per-suite and perturbation exceptions.
Go to primary source ↓e10PDF p. 7, Table 4 and Real-World Experiments; p. 6, Figure 5
Twenty trials per model per task; novel rows report Vid2WAM 30%, 15%, 30%. Seen Click Bell favors Motus 70% over Vid2WAM 65%. Figure 5 presents representative physical executions.
Go to primary source ↓e11PDF p. 7, Table 5 and Ablation Studies
LIBERO low-data averages: full 89.7, dual action 89.1, no adapter 88.5, action-only 86.4, future-only 88.2, Fast-WAM 87.3, online teacher 71.3. No seed uncertainty is displayed.
Go to primary source ↓e12PDF p. 7, Table 6 and Inference Latency Analysis
RTX 4090 chunk latency excludes cold starts. Vid2WAM mean/std is 212.5/11.7 ms; Fast-WAM mean 209.1, Teacher Policy 4894.5, Motus 2414.9 and π0.5 76.7.
Go to primary source ↓e13PDF p. 10, Appendix A, Teacher and Inverse Dynamics Model Training; Tables S1–S2; Eqs. (S1)–(S5)
Specifies LVP/Wan teacher initialization, frozen encoders/VAE, approximately 500 epochs, 49 frames; ResNet-50 three-frame IDM architecture and normalized Smooth L1 training for 200,000 steps. Domain-dependent frame stride is symbolic.
Go to primary source ↓e14PDF p. 11, Tables S3–S4; Eqs. (S6)–(S7); Pseudo-data construction; Training Details
Teacher uses 40 steps/49-frame chunk; chunks are concatenated. RGB conversion connects 16-channel teacher and 48-channel student latents. IDM validation randomly partitions frames, with 0.05 tolerance per dimension; same buffers/hyperparameters support ablations.
Go to primary source ↓e15PDF p. 13, Table S5 and caption; p. 11, Data Budget and Model Exposure
Lists expert/pseudo budgets: LIBERO 400/400 low-data and 1366/312 novel; RoboTwin 1000/1000 low-data and 19250/8250 novel; real-world 360/180. Exclusion of held-out trajectories differs from using first frames to seed rollouts.
Go to primary source ↓e16PDF p. 13, Table S6; p. 11, Vid2WAM Training Details
Wan2.2-TI2V-5B video initialization; shared action dimensions and 128-dimensional GELU adapters; β=1, real/pseudo weights 1/0.25; 128 samples per domain/update; 10 or 5 epochs; BF16 AdamW and domain-specific learning rates.
Go to primary source ↓e17PDF p. 14, Table S7, all variant rows
Shared video expert and ActionDiT have separate real/pseudo input/output adapters. Real adapters remain at inference. Dual action adds around 1B training parameters; default adapters add around 1M. Action-only discards future-latent loss; future-only discards pseudo-action loss.
Go to primary source ↓e18PDF p. 13, Table S12 and withheld-target discussion; p. 15, Tables S10–S11 and S13; p. 11, Appendix B
Main split uses suite-local IDs 8/9. Held-out LIBERO averages: 2.3 versus 0.0; Plus: 4.8 versus 1.0. Other LIBERO splits yield 38.3/7.5 and 7.0/0.0. Aggregate and withheld-only evaluation are explicitly separated.
Go to primary source ↓e19PDF p. 14, Tables S8–S9; p. 15, Detailed Task Description
Defines suite-local LIBERO task IDs and real tasks, including extracting an inserted test tube, pulling a tissue and placing a tomato in a basket.
Go to primary source ↓e20PDF p. 10, camera layouts; pp. 15–17, Appendix C, Initial-State Augmentation, Spatial/Temporal Masking and Rollout-duration cutoff; Figures S2–S3, p. 16
Novel real tasks use overhead-only augmentation, 30 clean-seeded and 30 distractor-seeded images. Spatial mask retains latent rows [0,14) and columns [0,28); temporal/action padding is excluded. Simulation cutoff is 8 seconds; real-world generation lasts 20 seconds.
Go to primary source ↓e21PDF pp. 18–19, Tables S14–S15, Overall and Novel subset rows
Full task tables give low-data totals 0.5536/0.4536, novel-regime totals 0.7830/0.7848 and held-out totals 0.5467/0.5527 for Vid2WAM, consistent with rounded Table 1.
Go to primary source ↓8.5 Primary sources
Vid2WAM: Distilling Video Diffusion Priors into World Action Models ↗
PDF · 9,009 extracted words
Source fingerprint
d5583e7f0c6eadc048496ce0ffc3c2272539149f2c7e232ff0db2a6cfea0e274