Fast-WAM: Do World Action Models Need Test-time Future Imagination?
1. Paper overview
In one sentence: Future-video supervision can improve an action policy even when deployment uses only current-observation features, avoiding video denoising while retaining iterative action sampling. e-architecturee-variantse-robotwine-liberoe-real-resultse-task-diagnostic
| At a glance | What to know |
|---|---|
| Research problem | Source description Video-based robot policies can entangle two benefits: learning from future-video supervision and conditioning actions on generated futures. Fast-WAM asks whether the latter is necessary enough to justify its latency. The relevant outcome is executed manipulation success and efficiency, rather than generated-video plausibility; the paper studies single action chunks and omits the outer autoregressive rollout from its formulation. e-questione-interface |
| Core mechanism | Source description Repurpose a pretrained video DiT as a single-pass observation encoder at inference while retaining future-video supervision during policy training. e-architecture |
| A key reported result | RoboTwin 2.0 bimanual manipulation: Fast-WAM: 91.88 clean, 91.78 randomized, 91.8 reported average. Success rate (%). Multi-task training on 2,500 clean and 25,000 randomized demonstrations; 30k steps; 100 evaluation trials per task in each setting. Joint 90.6; IDM 91.3; no video co-training 83.8; pretrained LingBot-VA 92.2; pretrained Motus 87.8 (reported averages). Removing the video objective costs 8.0 percentage points in reported averages. External baselines differ in pretraining. LingBot-VA from Wan2.2 reports 80.60 clean, no randomized value, and 80.6 'Average'; that is not a matched two-setting average. e-sim-protocole-robotwin |
| Reading caution | Reader analysis The controls align backbone, tokenization and training recipe 'as much as possible'; Joint and IDM still change attention and inference structure, and IDM has noise augmentation. This is not a complete factorial isolation of every source of benefit. e-variantse-config |
Core contributions
- Source description
Repurpose a pretrained video DiT as a single-pass observation encoder at inference while retaining future-video supervision during policy training. e-architecture
- Source description
Compare this design with joint video/action denoising, video-then-action inference, and an otherwise unchanged policy trained without the video objective. e-variants
- Author claim
The authors interpret the larger no-video-training degradation as evidence that representation learning explains much of WAMs' benefit. This is an empirical interpretation, not a direct measurement of physical understanding. e-robotwine-liberoe-real-results
Figure 1. Follow which visual tokens survive the transition from training to inference. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the top row as the training arrangement and the bottom row as the computation performed for an action chunk. Blue tokens represent frames and yellow tokens actions; the legends distinguish clean targets, noisy tokens and denoised outputs. Column A keeps video and action denoising coupled. Column B first generates future video, then passes a KV cache toward the action transformer. Column C keeps video prediction during training but retains only the first observation token group at inference. Its cache arrow runs from the video DiT toward the action DiT. The numbered boxes distinguish one video encoding pass from subsequent action denoising. e-questione-interfacee-architecturee-confige-latency
What it supports. Fast-WAM removes the expensive need to instantiate and iteratively denoise future video tokens at deployment. It still uses a video-derived representation and an action sampler. The diagram explains where the saved computation comes from; the measured magnitude of the latency change appears separately in Figure 4.
Where the evidence stops. The caption's phrase 'single forward pass' is broader than the implemented sampler. Section 4.1 specifies 10 denoising steps, and the diagram itself labels a second action-denoising stage. Treat the single-pass claim as applying to video encoding.
2. Motivation
2.1 The problem and the proposed response
Video-based robot policies can entangle two benefits: learning from future-video supervision and conditioning actions on generated futures. Fast-WAM asks whether the latter is necessary enough to justify its latency. The relevant outcome is executed manipulation success and efficiency, rather than generated-video plausibility; the paper studies single action chunks and omits the outer autoregressive rollout from its formulation. e-questione-interface
2.2 What this reading follows
Fast-WAM separates what a robot learns to predict from what it must generate during deployment. Its video backbone learns alongside an action expert, but a carefully restricted attention mask prevents the action branch from depending on future frames. At test time, the video backbone becomes an observation encoder. The decisive comparisons therefore involve both inference structure and the presence of video supervision during training. Read the aggregate tables alongside the real-world tradeoff and the task-level appendix: the training objective helps substantially on average, while explicit future conditioning still improves some tasks and towel-folding success. Lower latency does not by itself establish equal policy quality. e-architecturee-variantse-robotwine-liberoe-real-resultse-task-diagnostic
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The catalog is broadly supported: distinct video and action experts support Dual-system at the branch level, although they share attention and train jointly rather than operating independently. At inference, future prediction is neither joint with actions nor an inverse-dynamics conditioning input, supporting Other mechanisms / Outside quadrants. Efficient inference is directly measured. Latent prediction & JEPA fits only the latent-prediction portion: the objective is flow matching, with no JEPA-specific mechanism established. e-architecturee-objectivee-variantse-latency
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why learning a future need not require generating it
The key distinction is between information used as supervision and information supplied to an action predictor. During training, Fast-WAM sees demonstrated future frames because it must learn their flow-matching velocity field. Yet its action tokens cannot inspect those future tokens. The clean observation tokens also cannot inspect them, closing an indirect information route. The paper's interpretation is that video prediction improves the video backbone through learning, leaving a useful representation of the present observation after future tokens are removed. Reader analysis: this explains why a training objective can matter despite disappearing from deployment computation. It does not show that the latent features explicitly simulate a future or encode a verified physical state. Figure 2 and the two-part loss support the supervision mechanism; the ablation tests its behavioral value. e-architecturee-objectivee-robotwine-libero
Figure 2(a–b). The attention mask lets future frames supervise learning without becoming action inputs. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the current frame and future frames entering the VAE, and the instruction entering the text encoder. The noisy future tokens and noisy action tokens have separate transformer branches. Next read each attention-matrix row as the querying token group and each column as a possible information source, as verified against Section 3.2. The clean first-frame row has only its own visible block. Future-video rows see the first frame and other future tokens. Action rows see the first frame and actions, with blank future-video columns. In the inference matrix, those future rows and columns disappear entirely; the remaining action access pattern stays intact. e-architecturee-objectivee-reproduction-gaps
What it supports. The mask blocks both direct access to future tokens and an indirect route through the first-frame representation. This makes removing future tokens at inference compatible with the action branch's training-time context. Future prediction remains a learning objective for the shared video backbone, rather than a required intermediate prediction for the action branch.
Where the evidence stops. The mask and accompanying prose agree on these information paths. They specify visibility, not the exact gradient-freezing policy. The paper does not report which pretrained modules are frozen, and the figure should not be read as supplying that missing implementation detail.
5.2 Turn a video generator into a context provider
At deployment, the language instruction is encoded and the current camera observation enters the pretrained VAE. Fast-WAM retains only the clean first-frame latent tokens, processes them once with the video transformer, and supplies the resulting context to the action expert. Figure 1 depicts this connection as a KV cache. The action expert still performs denoising: the implementation uses 10 steps and classifier-free guidance scale 1.0 for a horizon of 32 actions. This is why the computational claim is specifically about eliminating future-video generation. The PDF's formulation focuses on one action chunk and omits the outer autoregressive loop. It does not establish how many actions are executed before a fresh observation is encoded, so an exact closed-loop execution schedule cannot be recovered from these equations. e-questione-interfacee-architecturee-confige-reproduction-gaps
5.3 Read the controls as a bounded causal argument
The no-video variant is the closest control for the training objective because the paper says it leaves architecture and inference unchanged. Joint and IDM instead test alternative routes through future generation, while changing attention and, for IDM, adding noise augmentation during training. On RoboTwin and LIBERO, deleting video supervision hurts more than switching among the three co-trained designs changes the aggregate result. Reader analysis: this favors retaining the video objective when choosing an efficient policy, but it is not a complete factorial decomposition of every design choice. The physical task makes the boundary especially clear: Fast-WAM has lower latency and completion time than IDM, yet IDM succeeds more often. The appendix likewise shows task-level reversals. A useful reproduction should measure these tradeoffs rather than test only the headline average. e-variantse-confige-robotwine-liberoe-real-resultse-latencye-task-diagnostic
5.4 Training and inference
During training
Optimize action and future-latent flow-matching losses together. The recipe uses logit-normal noise times, AdamW at learning rate 10⁻⁴, weight decay 0.01, cosine annealing, mixed precision and gradient clipping at 1.0. LIBERO training lasts 20k steps; RoboTwin and towel folding use 30k. No embodied pretraining is reported, but general video-model pretraining is explicitly used. e-objectivee-confige-sim-protocole-real-protocol
During inference
Retain only clean observation-frame latent tokens, run the video backbone once and pass its cached context to the action expert. Denoise actions for 10 steps with classifier-free guidance scale 1.0. 'Single forward pass' describes video encoding, not the complete action sampler. The paper does not specify how many predicted actions execute before observations are refreshed. e-questione-architecturee-confige-reproduction-gaps
5.5 Implementation flow
- Encode visual and language context
Wan2.2-5B supplies the video DiT, pretrained VAE and text encoder. Language reaches both branches through cross-attention. A 1B action expert with hidden dimension 1024 brings the reported total to 6B parameters. Temporal downsampling by four gives nine video frames per training chunk. e-architecturee-config
- Separate supervision from accessible information
The shared-attention Mixture-of-Transformer contains clean first-frame, noisy future-video and action token groups. Future tokens see the video context; action tokens see other actions and the first frame. Actions cannot attend to future tokens, and first-frame tokens cannot attend to future or action tokens. Figure 2's row-wise visibility pattern agrees with this description. e-architecture
- Construct controlled alternatives
Joint permits video/action attention and denoises both together. IDM generates future video first, then conditions actions on it; its training augments ground-truth video tokens with noise with probability 0.5. The no-video variant removes the video loss while retaining Fast-WAM's architecture and inference procedure. e-variantse-config
6. Experiments & results
Fast-WAM trains a video transformer and an action expert together, then uses the video transformer only to encode the current observation during deployment. Action generation still uses iterative denoising. Controlled comparisons associate larger performance gains with the video training objective than with explicit test-time future generation, within the evaluated manipulation tasks (e-architecture, e-variants, e-robotwin, e-libero, e-real-results).
6.1 Read the original evidence
Table 1. Compare controlled variants before interpreting the broader baseline ranking. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the Clean and Rand. columns, then read their reported Average. The upper block mixes methods with and without embodied pretraining, so its ranking combines architectural and data differences. The lower block is the paper's more direct test: Fast-WAM, Joint and IDM all retain video co-training, while the last row removes it. Section 4.2 specifies 2,500 clean and 25,000 randomized training demonstrations, 30k training steps and 100 evaluation trials per task in each setting. Keep the dash in the non-pretrained LingBot-VA row visible: its randomized result is absent, despite the displayed Average entry. e-variantse-sim-protocole-robotwin
What it supports. Fast-WAM reports 91.8% average success, versus 90.6% for Joint and 91.3% for IDM. Removing video co-training yields 83.8%, an 8.0-point reduction from the rounded averages. Fast-WAM's aggregate result is also near pretrained LingBot-VA's 92.2%, although that comparison does not isolate the training objective.
Where the evidence stops. The 80.6 entry for LingBot-VA from Wan2.2 is supported only by its clean result; it is not a matched clean/randomized average. No uncertainty estimates accompany these rows, and external methods have different pretraining histories.
Figure 4. Separate task-level effectiveness from model-call latency. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. In the left panel, higher success and shorter completion time make the upper-left region preferable. Read each point label rather than assuming a single ranking: pretrained π0.5 is best on both axes, while Fast-WAM has shorter completion time but lower success than IDM. The right panel answers a different question with explicit latency labels. All reported latency measurements use one RTX 5090D V2 32GB GPU. The task setup on page 7 uses 60 hours of Galaxea R1 Lite demonstrations and 30k training steps. Scatter coordinates are approximate readings from axes; the bar values are explicitly printed. e-real-protocole-real-resultse-latencye-config
What it supports. Fast-WAM takes 190 ms versus Joint's 580 ms and IDM's 810 ms, giving calculated speedups of about 3.05× and 4.26×. Its plotted towel-folding success is approximately 75%, below IDM's roughly 90%. Removing video co-training preserves the 190 ms latency but drops success to the text-confirmed 10%, separating cheap inference from effective behavior.
Where the evidence stops. The paper omits real-world trial counts, confidence intervals, exact success/timeout criteria and how failed episodes enter average completion time. Latency is not the same as the robot's control period. The 'over 4×' claim is comparison-specific and does not hold against Joint.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin 2.0 bimanual manipulation Multi-task training on 2,500 clean and 25,000 randomized demonstrations; 30k steps; 100 evaluation trials per task in each setting. | Fast-WAM: 91.88 clean, 91.78 randomized, 91.8 reported average. Success rate (%) | Joint 90.6; IDM 91.3; no video co-training 83.8; pretrained LingBot-VA 92.2; pretrained Motus 87.8 (reported averages). Removing the video objective costs 8.0 percentage points in reported averages. External baselines differ in pretraining. LingBot-VA from Wan2.2 reports 80.60 clean, no randomized value, and 80.6 'Average'; that is not a matched two-setting average. e-sim-protocole-robotwin |
| LIBERO four-suite manipulation 500 demonstrations and 10 tasks per suite; 20k training steps; 2,000 evaluation trials across 40 tasks with different random seeds. | Fast-WAM: Spatial 98.2, Object 100.0, Goal 97.0, Long 95.2; average 97.6. Success rate (%) | Joint 98.5; IDM 98.0; no video co-training 93.5; pretrained LingBot-VA 98.5; Motus 97.7; π0.5 96.9 (averages). The no-video gap is 4.1 percentage points, versus 0.9 and 0.4 deficits to Joint and IDM. No uncertainty is reported; the small gaps do not establish statistical equivalence. e-sim-protocole-libero |
| Real-world towel-folding effectiveness Galaxea R1 Lite; 60 hours of teleoperated demonstrations; 30k training steps. Evaluation trial count and precise timeout/success criteria are unstated. | Figure 4 places Fast-WAM at approximately 0.75 success and 150 seconds; IDM near 0.90 and 177 seconds. These are plot readings, not tabulated exact values. The text explicitly reports 10% success without video co-training. Success rate and average completion time | Pretrained π0.5 is best on both plotted metrics; Fast-WAM completes faster than IDM but succeeds less often. The real-world evidence concerns one task. It supports a speed–success tradeoff, not equal effectiveness across variants; treatment of failed episodes in completion-time averaging is unspecified. e-real-protocole-real-results |
| Towel-folding policy inference latency Reported inference measurements on one NVIDIA RTX 5090D V2 32GB GPU. | Fast-WAM 190; no video co-training 190; Joint 580; IDM 810; π0.5 180. Latency (ms; lower is better) | Calculated from the plotted values: IDM/Fast-WAM ≈4.26×; Joint/Fast-WAM ≈3.05×. The headline 'over 4×' holds against IDM, not Joint. Model-call latency differs from task completion time and is not a demonstrated end-to-end control frequency. e-confige-latency |
| RoboTwin task-level diagnostic Table 3, clean/randomized evaluation, under the RoboTwin protocol. | Move Stapler Pad: Fast-WAM 77/64 versus IDM 89/85. Press Stapler: Fast-WAM 90/97 versus IDM 50/57. Success rate (%) | The direction of the Fast-WAM–IDM difference reverses across these tasks. Aggregate competitiveness hides substantial task dependence; these rows do not identify the causal source of each failure. e-sim-protocole-task-diagnostic |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. The strongest ablation change is unevenly distributed across LIBERO suites. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read horizontally to see how a model's behavior changes across the four suites, and vertically within the lower block to compare the controlled designs. Each suite supplies 500 demonstrations for 10 tasks; the reported evaluation totals 2,000 trials across 40 tasks after 20k training steps. Fast-WAM reaches 100.0 on Object, but its Long score is lower. The no-video row makes the distribution of the supervision benefit clearer: Spatial falls from 98.2 to 89.2 and Long from 95.2 to 90.0, whereas Object changes only from 100.0 to 99.2. Average alone hides this concentration of the gap. e-sim-protocole-liberoe-variants
What it supports. Fast-WAM averages 97.6%, compared with 98.5% for Joint, 98.0% for IDM and 93.5% without video co-training. The calculated 4.1-point no-video gap exceeds either future-generation advantage. This supports the training-objective interpretation across a second benchmark, while also showing that the future-generating variants have higher aggregate LIBERO scores.
Where the evidence stops. The small differences among the three co-trained models have no reported uncertainty or repeated-training analysis. Near-ceiling Object results also limit how much that suite can discriminate among methods; the table does not establish statistical equivalence.
Appendix Table 3. Task-level reversals qualify the aggregate claim that future generation is unnecessary. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the paired Clean and Rand. columns to keep evaluation conditions aligned, then follow one task across the method groups. Move Stapler Pad provides a counterweight to the overall Fast-WAM advantage: IDM scores 89/85, compared with Fast-WAM's 77/64. Press Stapler reverses the comparison, with Fast-WAM at 90/97 and IDM at 50/57. The final row returns to the setting-level means used in the main results. This appendix is therefore a diagnostic of where aggregate rankings hide variation, not a separate experiment isolating a new component. It also gives the explicit task list needed for a reproduction. e-task-diagnostice-sim-protocole-variants
What it supports. The table supports a narrower conclusion than universal dispensability of future conditioning. Different tasks favor different inference designs even when the average scores are close. These reversals motivate testing which task properties predict a benefit from explicit futures, rather than assuming a single best design for all manipulation problems.
Where the evidence stops. Individual rows have no uncertainty estimates and do not explain their own failure mechanisms. Table 3 contains 50 task rows, whereas Section 4.2 describes the benchmark/setup as spanning more than 50 tasks; the exact evaluated list needs clarification.
7. Analysis & limitations
7.1 What the evidence leaves open
The controls align backbone, tokenization and training recipe 'as much as possible'; Joint and IDM still change attention and inference structure, and IDM has noise augmentation. This is not a complete factorial isolation of every source of benefit. e-variantse-config
Tables and plots omit confidence intervals. Real-world evidence is limited to towel folding, and the conclusion leaves larger-scale pretraining and model scaling for future work. General necessity or uselessness of future imagination remains untested. e-robotwine-liberoe-real-resultse-conclusion
Section 4.2 describes RoboTwin as spanning more than 50 tasks, while Table 3 lists 50 task rows. The exact evaluated task list should be resolved rather than silently equating these descriptions. e-sim-protocole-task-diagnostic
7.2 Questions for discussion
- Would the video-loss advantage persist across training seeds and data scales?
- Which tasks benefit from future conditioning once deployment delay is controlled?
- Does the learned context encode dynamics beyond what an equally expensive auxiliary objective would learn?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs the Wan2.2-5B components, action expert, exact attention mask, demonstrated actions and synchronized future frames, and the stated benchmark data and robot platform. The PDF does not specify λ, batch size, image resolution, exact action parameterization/normalization, frozen modules, training hardware/time, solver details, software versions or real-world train/test partitioning. e-architecturee-objectivee-confige-sim-protocole-real-protocole-reproduction-gaps
Proposed checks: repeat the video-loss ablation with matched initialization and multiple training seeds, and compare action-only versus future-conditioned policies under matched deployment timing. These would test reproducibility of the representation benefit and separate inference delay from policy quality; neither check was run here. e-variantse-confige-robotwine-real-resultse-latency
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the video-loss benefit survive matched repeated training?
Reader-proposed check, not performed: train Fast-WAM and its no-video-loss control from matched pretrained weights and action-expert initialization, using identical demonstration order, masks, optimizer, update count and action sampler. First resolve and record the unspecified video-loss weight, batch size and module-freezing policy. Repeat paired runs across multiple training seeds, then evaluate with matched environment seeds under the stated clean/randomized RoboTwin protocol. Report task-level paired differences and uncertainty as well as the aggregate. A consistently positive co-training advantage would support the paper's interpretation; disappearance or reversal across seeds would weaken it. Record training compute because an extra objective may change cost even at fixed update count. e-variantse-objectivee-confige-sim-protocole-robotwine-reproduction-gaps
Check 2: Separate future conditioning from deployment delay
Reader-proposed check, not performed: compare Fast-WAM, Joint and IDM on towel folding using matched initial towel configurations, success criteria, episode limits and action-execution schedules. Measure native end-to-end latency, then repeat with a common decision interval at least as long as the slowest measured policy call, inserting explicit waits for faster policies. Report success, time to completion, failure handling and correction counts separately, with repeated trials. If Fast-WAM's completion-time advantage shrinks under matched timing while IDM's success advantage persists, latency and policy quality explain different parts of the original plot. If the ranking changes, the native timing difference was a material confound. Document the previously unspecified execution/reobservation schedule. e-architecturee-confige-real-protocole-real-resultse-latencye-reproduction-gaps
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/affiliations and v2 stamp on page 1; Figure 1 on page 2; formulation on pages 3–4; architecture, masks and all objective equations on pages 4–5; variants, configuration, hardware and simulation protocol on page 6; Figure 3, Table 1 and real-world setup on page 7; Table 2 and comparison text on page 8; Figure 4 and conclusion on page 9; and all of Appendix Table 3 on page 13. All six final original crops were separately viewed. Figure 2 visibility was checked against Section 3.2; Figure 1's single-pass caption was checked against its action-denoising box and page 6's 10-step specification. References and the appendix heading on pages 10–12 were read in the text chunks but are outside the visual pass. Separate supplements were not supplied.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 13. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3 Method: 3.1 Problem Formulation, 3.2 Model Architecture, 3.3 Controlled Variants
- 4 Experiment: 4.1 Implementation Details, 4.2 Experiment Setup, 4.3.1–4.3.3 Main Results
- 5 Conclusion
- References
- A Appendix: A.1 RoboTwin Detailed Results, Table 3
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied title page identifies arXiv:2603.16666v2 [cs.CV], 23 March 2026. Title and all four authors match the catalog. The catalog submission date is 17 March 2026; v1 was not supplied, so changes between revisions cannot be established.
- All four text chunks covering the complete 13-page PDF were read. PDF pages 1–9 and 13 were visually inspected; reference pages 10–12 were read as text.
- The acquisition's text-extraction omission concerns figure images and equation/table layout; this reading addresses it through inspection of the retained PDF and all six final crops.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Linked code and project resources were not inspected. No paper scripts or experiments were run.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title/author/affiliation block and arXiv margin stamp
Fast-WAM: Do World Action Models Need Test-time Future Imagination?; Tianyuan Yuan, Zibin Dong, Yicheng Liu, Hang Zhao; IIIS, Tsinghua University and Galaxea AI. Stamp: arXiv:2603.16666v2 [cs.CV], 23 Mar 2026.
Go to primary source ↓e-questionPDF pp. 1–2, Abstract, Introduction and Figure 1(A–C) with caption
The paper separates video co-training from test-time future generation. Figure 1(C) shows a single video pass followed by action denoising through a KV-cache connection; its caption abbreviates this as single-pass prediction.
Go to primary source ↓e-interfacePDF pp. 3–4, Section 3.1, Eqs. (1)–(4) and surrounding paragraphs
Defines observation, instruction, action horizon and future observations; contrasts future-conditioned factorization with conditioning on z(o,l). The outer autoregressive loop is omitted.
Go to primary source ↓e-architecturePDF pp. 4–5, Section 3.2 Architecture, Figure 2(a–b), and inference paragraph
Wan2.2 video DiT, VAE and T5 serve the MoT policy with an action expert. Language cross-attention reaches all tokens. Mask rows allow action-to-first-frame and action-to-action attention but block future-to-action leakage; first-frame tokens only see themselves within the visual/action mask. Inference removes future tokens.
Go to primary source ↓e-objectivePDF p. 5, Section 3.2 Training objective, Eqs. (5)–(9)
Interpolation y_t=(1−t)y+tε yields velocity target ε−y. The joint objective is action flow matching plus λ times future-VAE-latent flow matching.
Go to primary source ↓e-variantsPDF pp. 5–6, Section 3.3 Controlled Variants
Joint co-denoises video/actions; IDM generates video before action conditioning; no-video removes the video objective while leaving architecture and inference unchanged. The shared implementation aligns training choices as much as possible.
Go to primary source ↓e-configPDF p. 6, Section 4.1 Implementation Details
Reports 1B action expert, 1024 hidden dimension, 6B total, 32-action horizon, fourfold video downsampling to nine frames, camera concatenation, logit-normal noise, 10 denoising steps, CFG 1.0, optimizer settings and one RTX 5090D V2 32GB for latency. IDM noise augmentation probability is 0.5.
Go to primary source ↓e-sim-protocolPDF p. 6, Section 4.2 LIBERO and RoboTwin 2.0 paragraphs
LIBERO has 500 demonstrations per 10-task suite, 20k training steps and 2,000 trials across 40 tasks. RoboTwin uses 2,500 clean plus 25,000 randomized demonstrations, 30k steps and 100 trials per task/setting; prose says more than 50 tasks.
Go to primary source ↓e-robotwinPDF p. 7, Table 1, Clean/Rand./Average columns, all baseline and variant rows
Fast-WAM 91.88/91.78/91.8; Joint 90.84/90.32/90.6; IDM 91.16/91.34/91.3; no-video 82.76/84.80/83.8. Pretrained LingBot-VA averages 92.2 and Motus 87.8. Non-embodied-pretrained LingBot-VA omits randomized results.
Go to primary source ↓e-liberoPDF p. 8, Table 2, Spatial/Object/Goal/Long/Average columns and Section 4.3.2
Fast-WAM 98.2/100.0/97.0/95.2/97.6; Joint average 98.5; IDM 98.0; no-video 89.2/99.2/95.4/90.0/93.5. External baseline averages include LingBot-VA 98.5, Motus 97.7 and π0.5 96.9.
Go to primary source ↓e-real-protocolPDF p. 7, Figure 3 and Real-World Evaluation paragraph in Section 4.2
Towel folding uses Galaxea R1 Lite, 60 hours of teleoperated demonstrations and 30k training steps. Success measures eventual completion; average completion time evaluates efficiency. Trial count and handling of failures are not specified.
Go to primary source ↓e-real-resultsPDF pp. 8–9, Section 4.3.3 and Figure 4 left panel/caption
Pretrained π0.5 has highest success and shortest completion time. Fast-WAM lies near 0.75/150 s and IDM near 0.90/177 s on plotted axes; these coordinates are approximate. Text reports 10% success without video co-training and the longest completion time. No error bars are shown.
Go to primary source ↓e-latencyPDF p. 9, Figure 4 right panel and Section 4.3.3 runtime paragraph; p. 6, Section 4.1 hardware statement
Bars explicitly label π0.5 180 ms, Fast-WAM 190, no-video 190, Joint 580 and IDM 810; hardware is one NVIDIA RTX 5090D V2 32GB.
Go to primary source ↓e-task-diagnosticPDF p. 13, Appendix Table 3, task rows and Average row
Table lists 50 tasks. Move Stapler Pad is 77/64 for Fast-WAM and 89/85 for IDM; Press Stapler is 90/97 versus 50/57. All methods have paired clean/randomized columns; the bottom row reproduces aggregate setting means.
Go to primary source ↓e-conclusionPDF p. 9, Section 5 Conclusion
The authors propose future study of larger-scale pretraining data and model scaling; current conclusions concern the reported benchmark and real-world comparisons.
Go to primary source ↓e-reproduction-gapsPDF pp. 4–7, Sections 3.2–4.2, model/objective/implementation/evaluation specifications
The supplied method and implementation sections do not give a numerical λ, batch size, image resolution, exact action representation, module-freezing policy, training hardware or duration, detailed solver/software versions, execution/reobservation schedule or real-world split. Section 4.1's named GPU is for latency measurements.
Go to primary source ↓8.5 Primary sources
Fast-WAM: Do World Action Models Need Test-time Future Imagination? ↗
PDF · 7,009 extracted words
Source fingerprint
4eea24883dcc4d8a5c0f760870f501baabb07db4eb65e5fd6c3b4b500601be8d