Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models
1. Paper overview
In one sentence: A video expert can supply useful future-aware context from one noisy pass, while sparse access and depth-wise fusion reduce its repeated cost during action generation. interfacecachesparse-fusionood-resultslatency
| At a glance | What to know |
|---|---|
| Research problem | Source description Can a policy retain the robustness associated with future modeling without repeatedly denoising video? The paper contrasts Joint-WAM’s costly evolving future context with Fast-WAM’s current-only inference interface. Its controlled experiments ask whether cached future slots and selective access offer a better tradeoff. problemcontrolled |
| Core mechanism | Source description One-pass future conditioning preserves video-derived temporal features throughout action generation; SparseMoT reduces repeated cross-branch attention and Interval KV-Fusion preserves intermediate-depth information at fixed context length. cachesparse-fusion |
| A key reported result | LIBERO-Plus distribution-shift robustness: 73.57% Success rate (%). LIBERO-trained policies; seven unseen perturbation categories; no additional training. Fast-WAM 49.14%; Joint-WAM 66.27%. A calculated gain of 24.43 percentage points over Fast-WAM. Table 3 rounds to one decimal; Table 5 supplies these precise values. Faster-WAM beats Joint-WAM in six categories, but is slightly lower on Layout. evaluationood-results |
| Reading caution | Reader analysis C-O Faster-WAM is trained separately. Its large gap supports the future-slot design but does not cleanly identify the effect of removing future context only at test time from an otherwise identical trained policy. future-ablation |
Core contributions
- Source description
One-pass future conditioning preserves video-derived temporal features throughout action generation; SparseMoT reduces repeated cross-branch attention and Interval KV-Fusion preserves intermediate-depth information at fixed context length. cachesparse-fusion
- Author claim
The authors argue that inference-time future context is critical for robustness. Their ablations support this design in the evaluated settings, with the causal qualifications recorded below. future-ablationvideo-ablation
Figure 2. Cache future context once, then let selected action layers consult it. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read panel (a) from the current observation at the bottom toward the video features above it, then follow the green Interval KV-Fusion blocks into the yellow action branch. Each block summarizes multiple video depths for a selected cross layer; intervening action self layers still operate. Panel (b) separates the inference schedules: Joint-WAM repeats video and action computation N times, whereas Faster-WAM creates a K/V cache once and repeats only action evaluation. The one-way arrows agree with the text: action queries may read video features, while video queries cannot read action tokens. That independence makes cache reuse possible. interfacecachesparse-fusionimplementation
What it supports. The acceleration changes communication and reuse rather than simply shortening the action Transformer. All action stages remain active, but only selected stages read the fused video context. The fixed cache can condition every action integration step because the video representation does not depend on the evolving action tokens.
Where the evidence stops. Panel (a) includes a VAE decoder as part of the overall framework. The deployment description on p. 4 explicitly skips future-video updating and decoding; this drawing should not be read as an inference-time decoded rollout.
2. Motivation
2.1 The problem and the proposed response
Can a policy retain the robustness associated with future modeling without repeatedly denoising video? The paper contrasts Joint-WAM’s costly evolving future context with Fast-WAM’s current-only inference interface. Its controlled experiments ask whether cached future slots and selective access offer a better tradeoff. problemcontrolled
2.2 What this reading follows
A robot policy may benefit from anticipating scene changes even when it never decodes a future video at deployment. Faster-WAM explores that possibility by separating the cost of constructing video features from the cost of repeatedly consulting them. Its video expert runs once on the current observation and noisy future slots; an action expert then reuses selected summaries while generating a movement chunk. The following figures trace why this cache is valid, how the two interaction mechanisms work, and what the manipulation and timing experiments establish. The key boundary is that better action success demonstrates useful conditioning, not necessarily accurate or interpretable imagined trajectories. interfacecachesparse-fusionood-resultslatency
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Insufficient evidence to decide
The separate video and action experts support Dual-system, and inference caching supports the efficiency category. A strict IDM/Q4 assignment remains unclear: action flow is conditioned on cached noisy-future features, without an explicit inverse-dynamics decoder over a reconstructed state transition. Joint training alone does not establish One Model, either. The recorded catalog classification is preserved. interfacecachesparse-fusionobjective
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make the future context independent before trying to cache it
Begin with the clean current-frame latent and Gaussian future slots. The word future describes their temporal role in a model trained on future-video flow prediction; it does not imply that those slots already contain a visible prediction. At the noisy endpoint, one video evaluation produces attention keys and values at multiple depths. The attention restrictions are what make this practical: future slots may read the current anchor, but the anchor cannot read future slots, and video tokens never read action tokens. Consequently, changing the noisy action state during integration does not invalidate the video features. The action expert can reuse the same context throughout its ten evaluations. Training still supervises both video and action flows with independently sampled noise levels, so this deployment schedule should be distinguished from removing future modeling during learning. cacheobjectiveimplementationtraining-details
5.2 Separate how often actions consult video from how much depth they can access
SparseMoT selects the stages at which action queries can read video keys and values. It does not discard the other action layers: their self-attention and feed-forward updates continue carrying and refining information already injected. Selecting fewer interaction stages creates a second problem, because useful features from intervening video depths could be missed. Interval KV-Fusion addresses that problem by summing corresponding key/value pairs over the depth interval assigned to each selected stage. Softmax-normalized weights preserve key–value correspondence, and the common token layout keeps the resulting context as short as one video layer’s context. At an interaction stage, action queries attend to this fused video pair together with action keys and values, as Eq. (6) specifies. The default implementation selects eight interaction stages within a 30-layer action network. sparse-fusionimplementation
5.3 Use several controls to separate robustness, representation and runtime claims
Reader analysis: no single result establishes the whole design argument. Standard LIBERO scores are already high for all three controlled WAM implementations, so LIBERO-Plus offers a more discriminating comparison. The separately trained current-only control then asks whether sparse interaction alone explains the gain; its lower success makes that explanation incomplete. The same-checkpoint video-step sweep asks a different question: whether spending more computation on future reconstruction improves the cached conditioning. It does not improve the aggregate here. Finally, the latency breakdown connects sparse access to reduced action-side cost, while the fusion ablation suggests a performance benefit with little timing overhead. Together these observations support the proposed tradeoff. They still leave open the effect of test-time removal alone, interactions among components, and uncertainty across independently trained models. id-resultsood-resultsfuture-ablationvideo-ablationcomponent-ablationlatencytraining-details
5.4 Training and inference
During training
Train video/action experts, fusion and the proprioceptive encoder jointly, using precomputed language embeddings. Independent video/action noise levels supervise flow prediction with masked, timestep-weighted MSE and equal branch weights. Training samples pair 32 actions with views at indices 0,4,…,32. objectivetraining-details
The shared recipe uses AdamW, learning rate 1e-4, weight decay 0.01, BF16 and ZeRO-1; 5% warmup precedes cosine decay to 1e-6. Seed is 42. The appendix supplies optimizer betas, clipping, schedulers and dataset-specific batches. training-details
During inference
At each planning step, cache the video hierarchy at video flow time 1, fuse it, and run ten action integration steps with guidance scale 1.0. Future latents are neither iteratively updated nor decoded. The paper specifies chunk prediction but does not establish the executed prefix length or feedback cadence. cacheimplementationinterface
5.5 Implementation flow
- Encode the current scene
Wan2.2-5B initializes the video DiT, text encoder and VAE. Spatially assembled camera views form a clean current-frame anchor. A separate 30-layer, width-1024 action Transformer receives language and proprioceptive conditioning. interfaceimplementationtraining-details
- Build an action-independent future context
The video expert reads the clean anchor and Gaussian future slots once. Future slots can read the anchor and each other; the anchor cannot read future slots. Video queries cannot read action tokens, making cached K/V independent of subsequent action updates. cache
- Fuse across depth, then read sparsely
Softmax-weighted sums combine corresponding video K/V pairs over each preceding interaction interval. Selected action stages attend to this summary together with their own action K/V. All action layers remain active: the default stride-four design selects eight of 30 stages for video access. sparse-fusionimplementation
6. Experiments & results
Faster-WAM generates a robot action chunk using future-aware features cached from one video-expert pass. SparseMoT limits where action layers read those features; Interval KV-Fusion combines information across video depths. The strongest evidence is improved manipulation under distribution shifts with lower measured inference latency, rather than improved decoded-video quality. The methods and comparisons below trace these claims to the inspected v1 paper.
6.1 Read the original evidence
Table 3. The distribution-shift benchmark separates policies that look similar on standard LIBERO. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the last three rows, which compare the authors’ implementations of Fast-WAM, Joint-WAM and Faster-WAM. Their backbone, training data and action-sampling recipe are controlled, while their future context and interaction designs differ. P.T. means embodied pretraining; its cross does not mean that the video backbone was initialized from scratch. Follow a single shift column downward before comparing the final average column. Camera and noise expose particularly large differences. All policies in this evaluation were trained on LIBERO and transferred without additional LIBERO-Plus training. This table rounds percentages to one decimal; Table 5 provides the more precise controlled-model values. controlledevaluationid-resultsood-resultstraining-details
What it supports. Faster-WAM reaches 73.6% overall in this rounded table, compared with 49.1% for Fast-WAM and 66.3% for Joint-WAM. Its advantage over Fast-WAM spans all seven shift categories. Against Joint-WAM it improves six categories, with Layout slightly lower rather than universally superior.
Where the evidence stops. The controlled comparison concerns the three WAM implementations, not identical training for every listed external baseline. Success-rate uncertainty is absent, and the category columns should not be interpreted as a guarantee across arbitrary distribution shifts.
Figure 4. Physical robustness is tested by changing the scene around the same strawberry-picking task. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read V1 as the standard scene, V2 as novel backgrounds, V3 as altered lighting and V4 as unseen distractor objects. Orange, blue and green bars respectively denote Joint-WAM, Fast-WAM and Faster-WAM; the same ordering is repeated below every image. Compare methods within each panel using its own axis. V1 starts near 90%, while the OOD panels use much lower and differing ranges, so apparent bar lengths cannot be compared directly across panels. The photographs show the concrete visual interventions. The associated text reports aggregate OOD success separately from the four-task standard evaluation shown in Figure 3. real-resultsreal-data
What it supports. The green bar is highest in all three OOD panels, supporting the reported robustness advantage on this physical task. The accompanying text gives OOD averages of 71.1% for Faster-WAM, 55.6% for Joint-WAM and 45.6% for Fast-WAM. These are manipulation outcomes, not judgments of generated images.
Where the evidence stops. OOD testing here covers Pick Strawberries only. The paper specifies 30 standard trials per task but does not separately state the count per OOD condition or provide success intervals. Unlabelled bars should not be assigned extra numerical precision.
Table A4. Most of the measured efficiency gain comes from reducing repeated action-side interaction. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each entry as mean plus or minus standard deviation in milliseconds across ten synchronized runs after five warmups. Measurements use BF16 on one NVIDIA L20 with 224×448 inputs, horizon 32 and ten action steps. Faster-WAM spends more time constructing visual context than Fast-WAM, but considerably less in action denoising. The final two rows correspond to cumulative removals, matching the component study. Dashes in Joint-WAM mean visual and action costs cannot be separated under joint denoising. Overall includes shared preparation overhead, so adding the displayed component means does not reproduce the overall value exactly. latencycomponent-ablation
What it supports. Faster-WAM’s 252.95 ms overall latency is 2.21× faster than Joint-WAM’s 559.84 ms and below Fast-WAM’s 320.97 ms. Removing fusion barely changes latency, to 252.00 ms; additionally removing SparseMoT raises it to 339.00 ms. This links most of the measured component-level saving to sparse interaction.
Where the evidence stops. These are inference-entrypoint timings, excluding image preprocessing, language encoding, action denormalization and gripper post-processing. They do not establish complete sensor-to-actuator latency or sustained robot control frequency. Reported variation concerns repeated timing runs, not independently trained policies.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Plus distribution-shift robustness LIBERO-trained policies; seven unseen perturbation categories; no additional training. | 73.57% Success rate (%) | Fast-WAM 49.14%; Joint-WAM 66.27%. A calculated gain of 24.43 percentage points over Fast-WAM. Table 3 rounds to one decimal; Table 5 supplies these precise values. Faster-WAM beats Joint-WAM in six categories, but is slightly lower on Layout. evaluationood-results |
| LIBERO standard manipulation Four suites; 50 rollouts per task. | 99.0% Average success rate (%) | Joint-WAM 98.5%; Fast-WAM 97.6%. High in-distribution performance leaves little headroom; these three controlled implementations use no embodied pretraining. evaluationid-resultscontrolled |
| RoboTwin 2.0 bimanual manipulation Training: 2,500 clean and 25,000 randomized demonstrations; 100 trials per task in each evaluation setting. | 92.8 / 92.3 / 92.6 (source-reported in Table 2) Clean / randomized / average success rate (%) | Fast-WAM average 91.9; Joint-WAM 90.6; pretrained LingBot-VA 92.2. Best reported combined mean among listed methods, but LingBot-VA has the higher clean mean, 92.9%. Table A5 reports Faster-WAM clean/randomized means of 92.78/92.26%; their calculated equal-weight mean is 92.52%, which rounds to 92.5%, not Table 2's 92.6%. The PDF does not explain this discrepancy. Both source summaries are preserved; individual task reversals also remain. evaluationid-resultstask-detail |
| Physical dual-arm manipulation Four tasks on two Piper arms; 30 standard trials per task. OOD tests concern only Pick Strawberries across three unseen conditions. | 95.8 / 71.1 Standard overall / OOD average success rate (%) | Joint-WAM 90.8 / 55.6; Fast-WAM 88.3 / 45.6. These are executed robot outcomes. The OOD aggregate covers one task, whereas the standard aggregate covers four. real-results |
| Inference latency One NVIDIA L20, BF16, 224×448 inputs, horizon 32, ten action steps; ten synchronized measurements after five warmups. | 252.95 ± 2.42 ms Overall milliseconds, mean ± standard deviation | Joint-WAM 559.84 ± 7.61; Fast-WAM 320.97 ± 2.86 ms. The reported 2.21× speedup concerns this timed inference entrypoint. Preprocessing, language encoding, action denormalization and gripper post-processing are excluded. latency |
| Future-context and interaction ablations LIBERO-Plus; separately trained current-only control and cumulative component removals. | Full 73.57; current-only 51.00; without fusion 69.99; then without SparseMoT 69.78. Average success rate (%) | Stride 2/4/7/14: 71.65/73.57/71.05/70.05. Moderate sparsity performs best in this sweep. Cumulative removals do not independently isolate every component. future-ablationcomponent-ablation |
| Additional video denoising Same checkpoint; fixed ten action steps; cache only the final video step. | One pass 73.57; two steps 73.24; ten steps 68.33. LIBERO-Plus average success rate (%) | Nine steps reach 67.86, so the decline is not strictly monotonic. More video computation does not improve the aggregate here; this does not test retraining for each video-step budget. video-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5(a)–(c). Future slots matter in the trained comparison, while moderate interaction sparsity performs best. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the three blocks as different comparisons. In (a), C-O Faster-WAM keeps the interaction architecture but forms current-only context and is trained separately; compare it with the full model to assess the future-slot design. In (b), removal is cumulative: the w/o SparseMoT row also follows removal of KV-Fusion, and the last row additionally returns to iterative joint denoising. These are not independent single-component knockouts. In (c), track the Avg. column from dense interaction through stride 14, then inspect individual shifts for exceptions to the aggregate trend. All entries report LIBERO-Plus success percentages, with larger values better. future-ablationcomponent-ablationtraining-details
What it supports. The current-only control reaches 51.00%, versus 73.57% for the full model, a calculated 22.57-point difference. Removing fusion yields 69.99%. The stride sweep peaks at four, reaching 73.57%, above both dense interaction at 69.78% and stride 14 at 70.05%. More frequent video access is therefore not automatically better in this experiment.
Where the evidence stops. The current-only control is retrained, so this is not a pure test-time intervention on one checkpoint. Cumulative removals leave interactions between components unresolved, and the small gaps lack uncertainty across training seeds.
Table A3. Extra video denoising does not improve the overall result for the same trained checkpoint. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The first column changes video computation only. Every row uses the same Faster-WAM checkpoint, ten action integration steps and a cache formed from the final video step. Thus the default one-pass noisy-endpoint representation is compared with contexts obtained after additional video processing. Follow Avg. downward to see the broad decline, then use the shift columns to check that individual categories need not follow it. The original caption defines Avg. over all tasks rather than as the arithmetic mean of the seven displayed category scores. The default row is a single expert evaluation at the noisy endpoint, not a fully reconstructed video. video-ablationcache
What it supports. One pass gives 73.57%; two steps give 73.24%; ten give 68.33%. The trend is broadly downward but not monotonic: ten steps recover somewhat from 67.86% at nine. These results support using an inexpensive early representation for action conditioning in this checkpoint and evaluation.
Where the evidence stops. This inference-only sweep does not show that an independently retrained multi-step model must underperform. It also reports action success, so the trend cannot establish that the corresponding generated videos become visually worse.
7. Analysis & limitations
7.1 What the evidence leaves open
C-O Faster-WAM is trained separately. Its large gap supports the future-slot design but does not cleanly identify the effect of removing future context only at test time from an otherwise identical trained policy. future-ablation
Success tables do not report uncertainty across training seeds; the appendix specifies seed 42. Timing standard deviations are available and should not be transferred to success rates. Real-world OOD evidence covers one task, and the trial count per OOD condition is not separately specified. training-detailslatencyreal-results
Future-aware hidden features are not direct evidence of accurate decoded dynamics. Fixed stride selection is heuristic; learned interaction placement and systems acceleration remain future work. cachevideo-ablationfuture-work
7.2 Questions for discussion
- Would matched-checkpoint removal of future slots retain the large OOD gap seen with the separately trained current-only control?
- Does the preferred interaction stride persist under different camera resolutions and action horizons?
8. Reproducibility audit
8.1 Requirements and known gaps
Match temporal sampling and spatial camera assembly. LIBERO uses 224×448 images, eight A800 GPUs, ten epochs and batch 128; RoboTwin/real use 384×320, 32 A800 GPUs, five epochs and batches 1024/512. The real dataset has 1,600 demonstrations and 14-dimensional joint/gripper actions and proprioception. training-detailsreal-data
The source leaves exact selected-layer indices, VAE freezing status, action normalization, executed chunk prefix and software versions unspecified. These require resolution before a faithful implementation; the code/model link on the title page alone establishes none of them. identityimplementationinterfacetraining-detailslatency
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate test-time use of future slots from training with them
Reader-proposed check, not performed: reproduce the matched full and current-only training protocols over several seeds, then evaluate paired LIBERO and LIBERO-Plus initial states. For each full checkpoint, compare intact context with future-token K/V masked at inference, retaining the current-anchor context and identical action noise. Keep the separately trained current-only model as a control for distribution changes caused by masking. Report overall and per-shift success with paired uncertainty. A repeatable OOD decline under inference masking would strengthen the claim that the trained policy uses future-slot context; a gap only between separately trained models would weaken a purely inference-time explanation. Masking itself may create unfamiliar inputs, so neither comparison should stand alone. cachecontrolledevaluationfuture-ablationtraining-details
Check 2: Test fusion and interaction sparsity as separate factors
Reader-proposed check, not performed: train stride-two and stride-four models, each with learned interval fusion versus K/V from the selected video layer only; include dense interaction without fusion as a reference. Retain one-pass context, action depth, data, optimizer and ten action steps, with matched training seeds and paired evaluation starts. Measure LIBERO-Plus success and repeat the specified L20 timing protocol, reporting visual, action and overall latency with the source’s inclusion boundaries. Sparse access should reduce action latency, while fusion should help retain depth information without increasing context length. No reproducible fusion gain at either sparse stride would weaken its proposed benefit; no action-latency reduction would challenge the efficiency mechanism. Dense intervals contain one layer, so fusion there is an identity operation rather than a separate experimental factor. sparse-fusionimplementationcomponent-ablationtraining-detailslatency
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page and Figure 1; the complete method and equations on pages 2–4, including Figure 2; Tables 1–3 and evaluation protocols on page 5; Figures 3–4, Table 4 and real-world/latency/ablation descriptions on page 6; Table 5 and conclusion on page 7; all appendix configuration, hardware, timing, dataset and ablation pages 10–11; and the full per-task Table A5 on page 12. Pages 5 and 12 support the retained RoboTwin result and the explicit difference between Table 2's reported combined average and the mean calculated from Table A5. All six final original crops were inspected at their helper-produced dimensions. Figure 2 arrows were cross-checked against the attention restrictions and Eqs. (6)–(8); its decoder is a framework element, while the deployment text explicitly omits decoding. All supplied text, including references on pages 8–9, was read. Separate supplements and linked code were not inspected. No experiment was reproduced.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 10, 11, 12. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF pp. 1–2: Abstract, Introduction and Related Work
- PDF pp. 2–4: Problem Formulation and complete Faster-WAM method, Eqs. (1)–(11)
- PDF pp. 4–7: Implementation Details, Experiment Setup, Main Results, Ablation Study and Conclusion
- PDF pp. 8–9: References
- PDF pp. 10–12: complete Appendix, training, video-step ablation, latency, real-world dataset and per-task RoboTwin results
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction limitation was addressed by visually inspecting PDF pages 1–7 and 10–12 and all six final crops; references on pages 8–9 were read as text.
- No supplied text was skipped. Separate supplements, code, models and linked sources were not inspected; no experiments were reproduced.
- Identity notes: title and all eight authors match the catalog after name-order normalization. The inspected artifact is arXiv:2608.04404v1, dated 5 August 2026 on PDF p. 1; the catalog submittedDate is 4 August 2026. No other revision or edition was supplied or compared.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author block, affiliations and arXiv margin
The observed title matches the catalog. Authors are Weiheng Zhao, Haoyi Jiang, Xin Shi, Liu Liu, Fan Huang, Zhizhong Su, Wei Sui and Xinggang Wang. Affiliations are Huazhong University of Science and Technology, D-Robotics, Horizon Robotics and Xiamen University. The margin reads arXiv:2608.04404v1 [cs.CV], 5 Aug 2026.
Go to primary source ↓problemPDF pp. 1–2, Abstract, Introduction and Figure 1
The paper contrasts expensive inference-time joint video/action denoising with removing future slots at inference, and proposes retaining a reusable future-aware context.
Go to primary source ↓interfacePDF p. 2, Problem Formulation, Eq. (1); p. 3, Eqs. (2)–(3), Overview and Figure 2
The policy takes observation, language and proprioception and predicts an action chunk. Separate video and action experts interact through MoT. Figure 2(a) shows video and action paths; Figure 2(b) contrasts repeated joint computation with a one-pass cache.
Go to primary source ↓cachePDF p. 3, One-pass Future Conditioning, Eq. (4); p. 4, Efficient Inference
Future slots attend to the current-frame anchor and each other; the anchor cannot attend to future slots. Action queries read video features, but video queries cannot read action tokens. One video pass at video flow time 1 produces reusable K/V features; deployment does not update or decode future latents.
Go to primary source ↓sparse-fusionPDF p. 3, SparseMoT, Eq. (5); p. 4, Eqs. (6)–(8) and Interval KV-Fusion
Selected action stages attend to concatenated fused video and action K/V. Other action stages remain active without reading video K/V. Softmax-normalized weights sum aligned K/V pairs over preceding depth intervals, retaining one stage’s token length.
Go to primary source ↓objectivePDF p. 4, Joint Training, Eqs. (9)–(11)
Video and action noise levels are sampled independently. Linear noise interpolation gives flow targets noise minus clean data. The loss combines separately timestep-weighted video and action flow MSE terms.
Go to primary source ↓implementationPDF p. 4, Implementation Details
Wan2.2-5B initializes the video DiT, text encoder and VAE. The action Transformer has 30 layers, width 1024, horizon 32; nine visual frames use stride four. Stride-four interaction yields eight selected stages. Inference uses 10 integration steps and guidance scale 1.0.
Go to primary source ↓controlledPDF pp. 4–5, Implementation Details, controlled comparison paragraph
Joint-WAM, Fast-WAM and Faster-WAM share pretrained video backbone, tokenization, training data, flow-matching objective, optimization and action sampling settings. Their video context and interaction mechanisms differ.
Go to primary source ↓evaluationPDF p. 5, Experiment Setup: LIBERO, RoboTwin 2.0 and LIBERO-Plus
LIBERO has four suites, each with 10 tasks and 500 demonstrations; evaluation uses 50 rollouts per task. RoboTwin training uses 2,500 clean and 25,000 randomized demonstrations, with 100 trials per task per setting. LIBERO-trained policies are tested on seven LIBERO-Plus shifts without additional training.
Go to primary source ↓id-resultsPDF p. 5, Tables 1–2, Faster-WAM, Fast-WAM, Joint-WAM and LingBot-VA rows; captions
LIBERO averages are 99.0%, 97.6% and 98.5% for Faster, Fast and Joint. RoboTwin clean/randomized/average scores are 92.8/92.3/92.6 for Faster, 91.9/91.8/91.9 for Fast, 90.8/90.3/90.6 for Joint and 92.9/91.5/92.2 for LingBot. P.T. denotes embodied pretraining.
Go to primary source ↓ood-resultsPDF p. 5, Table 3, all columns; p. 7, Table 5(a)–(b), Faster-WAM, Fast-WAM and Joint-WAM rows
Table 3 reports rounded LIBERO-Plus averages 73.6%, 49.1% and 66.3%; Table 5 gives 73.57%, 49.14% and 66.27%. Faster beats Joint in six shift categories, but Layout is 79.09% versus 79.55%.
Go to primary source ↓real-resultsPDF p. 5, Real-World Evaluation; p. 6, Figures 3–4, captions and Real-World Evaluation paragraph
Two Piper 6-DoF arms perform Pick Strawberries, Build Tower, Store Boxes and Stack Plates. Standard evaluation uses 30 trials per task. Overall success is Faster 95.8%, Joint 90.8%, Fast 88.3%. Pick Strawberries OOD averages across novel backgrounds, altered lighting and distractors are 71.1%, 55.6% and 45.6%. Figure 4 labels V1 standard, V2 backgrounds, V3 lighting and V4 distractors.
Go to primary source ↓future-ablationPDF p. 6, Future Conditioning; p. 7, Table 5(a) and accompanying paragraph
C-O Faster-WAM retains SparseMoT and Interval KV-Fusion but removes future temporal slots from its visual context and is trained from scratch under the same protocol. Its average is 51.00%, compared with 73.57% for Faster-WAM; camera scores are 16.25/53.75 and noise scores 32.95/63.57.
Go to primary source ↓component-ablationPDF p. 7, Table 5(b)–(c), captions and Component Ablation/Interaction Sparsity paragraphs
Cumulative removal gives 73.57%, 69.99% without fusion, 69.78% after further removing sparse interaction, and 66.27% for Joint-WAM. Stride 1/2/4/7/14 yields 69.78/71.65/73.57/71.05/70.05%.
Go to primary source ↓training-detailsPDF p. 10, Training Details of Faster-WAM and Tables A1–A2
Video/action experts, fusion and proprioceptive encoder are optimized together with precomputed language embeddings. Training uses BF16, ZeRO-1, AdamW betas (0.9,0.95), learning rate 1e-4, weight decay 1e-2, 5% warmup, cosine decay to 1e-6, gradient clipping 1.0, independent 1,000-timestep schedulers with shift 5.0, equal loss weights, padding masks and seed 42. LIBERO uses 8 A800 GPUs, 10 epochs, batch 128; RoboTwin/real use 32 A800 GPUs, 5 epochs, batches 1024/512. Views at indices 0,4,...,32 give one current and eight future frames.
Go to primary source ↓video-ablationPDF p. 10, Effect of Video Denoising Steps; p. 11, Table A3 and caption
One checkpoint is evaluated with 1–10 video steps, caching the final step’s K/V and retaining 10 action steps. Averages are 73.57, 73.24, 72.38, 72.05, 70.72, 70.19, 70.39, 69.06, 67.86 and 68.33%. The caption defines Avg. over all tasks, not over categories.
Go to primary source ↓latencyPDF p. 6, Table 4 and Inference Latency; pp. 10–11, Details of the Latency Comparison and Table A4
BF16 timing on one NVIDIA L20 uses 224×448 inputs, horizon 32, 10 action steps, five warmups and ten synchronized measurements. Overall mean±SD ms: Joint 559.84±7.61, Fast 320.97±2.86, Faster 252.95±2.42, without fusion 252.00±2.38, then without SparseMoT 339.00±3.66. Faster visual/action times are 43.04±0.04/192.11±2.36 versus Fast 27.67±0.44/276.56±2.77. Timing excludes image preprocessing, language encoding, action denormalization and gripper post-processing; total includes shared preparation overhead and action transfer to CPU. Model/input seeds are 42/0.
Go to primary source ↓real-dataPDF p. 11, Details of the Real-World Dataset; p. 10, Table A2
The real dataset has 1,600 demonstrations, 400 per task, 488,393 synchronized timesteps at 30 FPS, approximately 4.52 hours. Three 480×640 RGB streams come from head and wrist cameras. Proprioception/actions have 14 channels: six joint and one gripper channel per arm. Training assembles views into 384×320 images.
Go to primary source ↓task-detailPDF p. 12, Table A5, Average, Hanging Mug and Blocks Ranking Size rows; p. 5, Table 2, Faster-WAM row
RoboTwin Faster-WAM clean/randomized averages in Table A5 are 92.78/92.26 versus LingBot-VA 92.90/91.50; Table 2 instead prints 92.8/92.3/92.6 for Faster-WAM clean/randomized/combined. Faster-WAM gets 61/44 on Hanging Mug versus Fast-WAM 58/62, and 74/93 on Blocks Ranking Size versus Fast-WAM 94/98. Aggregate superiority does not hold on every task.
Go to primary source ↓future-workPDF p. 7, Conclusion
The authors propose learned selection of interaction stages, graph compilation and custom CUDA kernels as future work rather than measured improvements.
Go to primary source ↓8.5 Primary sources
Faster-WAM: Efficient Inference-Time Future Conditioning for Robust World Action Models ↗
PDF · 7,079 extracted words
Source fingerprint
a494edc5b609ec0841066083851daa699f8c2cc3dd1af6863f39a13f1d3ed00d