Robust-WAM: Bridging Generative Pretraining and Semantic Foresight in World-Action Models
1. Paper overview
In one sentence: Robust-WAM adds future semantic supervision to an existing WAM's action stream, improving tested OOD success while retaining the pretrained video formulation and extra query tokens at inference. e02e03e06e08e10e13e14e15e16
| At a glance | What to know |
|---|---|
| Research problem | Author claim VAE reconstruction preserves texture and illumination, which the authors argue can make video-derived action features sensitive to irrelevant appearance changes. Rebuilding prediction in semantic space forfeits direct reuse of VAE-space video pretraining. Robust-WAM seeks semantic robustness while preserving that pretrained formulation. e02 |
| Core mechanism | |
| A key reported result | LIBERO-Plus robustness after clean LIBERO training: GE-Act + Robust-WAM: 80.9 OOD, 97.3 clean; FastWAM + Robust-WAM: 58.9 OOD, 97.9 clean. Average success rate (%) over evaluation trials. Train on four standard LIBERO suites; evaluate all perturbed LIBERO-Plus tasks and clean suites. Compare matched variants within each backbone. GE-Act: 78.0/96.5; FastWAM: 49.7/97.6 (OOD/clean). OOD gains: 2.9 and 9.2 percentage points. Average robustness improves without a clean-score decrease. Cross-family rankings do not isolate the effect of video pretraining. e08e09e10 |
| Reading caution | Reader analysis The paper's assertion that semantic supervision should enhance rather than replace video pretraining is stronger than the controlled evidence: different model families use different recipes. Within-backbone gains support the added method, but do not isolate pretraining as the cause of cross-model ranking. e08e10e13 |
Core contributions
Figure 2. Future semantics supervise the action stream while the video branch retains its generative objective. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the diagram from the bottom upward. On the left, current-frame context and noisy future video tokens enter the Video DiT, whose outputs retain a video flow-matching objective. On the right, learnable queries precede noisy action tokens in the Action DiT. Query outputs are compared with the frozen teacher's future-frame CLS vectors, while action outputs use the original decoder and action loss. The dotted colored connections pair query and action positional encodings. The double-headed loss arrows denote comparisons, not teacher features supplied as online inputs. Sections 3.1–3.3 supply the video-attention coupling and architecture-specific masks that this overview does not draw explicitly. e03e04e05e06e07
What it supports. The new semantic path changes what internal action features must predict during training. It leaves the VAE-space video objective intact and supplies future semantic supervision through queries. At inference, the teacher and alignment head disappear while those queries remain, so no ground-truth future frame is required to condition the deployed policy.
Where the evidence stops. Notation differs: the graphic labels learnable inputs tilde-q and semantic outputs q, whereas Section 3.2 calls the learned inputs q and output states bar-h. Follow the verified input/output roles. The overview also does not specify the horizon clamp or unified attention mask.
2. Motivation
2.1 The problem and the proposed response
VAE reconstruction preserves texture and illumination, which the authors argue can make video-derived action features sensitive to irrelevant appearance changes. Rebuilding prediction in semantic space forfeits direct reuse of VAE-space video pretraining. Robust-WAM seeks semantic robustness while preserving that pretrained formulation. e02
2.2 What this reading follows
A robot can see the same task under a different light and produce a different action. Robust-WAM addresses this problem inside a pretrained video-action model: it keeps the VAE-based generative path and teaches added action-stream queries to predict future scene semantics. A frozen DINOv3 teacher supplies training targets, then is removed for deployment. The useful evidence is the combination of architecture, controlled backbone comparisons, temporal-query ablations and actual robot rollouts. Read the results as evidence for an added training objective under specific recipes. They do not establish universal appearance invariance or isolate video pretraining from every other difference between model families. e02e03e06e08e10e13e14e15e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Not applicable |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
Outside quadrants / Other mechanisms fits Robust-WAM as an architecture-agnostic auxiliary alignment method. Its hosts include both action-expert and unified WAMs, so the method itself has no single One Model architecture. Future semantics regularize action representations during training; this is neither a separate inverse-dynamics controller nor sufficient evidence to relabel every host's prediction paradigm. The latent/alignment tags fit, while JEPA denotes a broad catalog grouping rather than a claimed JEPA implementation. e05e06e07
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Keep reconstruction and semantic supervision in different roles
Begin with the existing WAM rather than with the teacher. Its frozen VAE converts future training frames into reconstruction latents, and the video transformer learns the inherited flow-matching objective. Action prediction reads video hidden states, so changing the latent space would change the representation on which that pretrained path depends. Robust-WAM instead inserts learned queries beside action tokens and supervises their output states with future DINOv3 CLS embeddings. The linear alignment head maps query states into the teacher's feature space; it does not replace the original action decoder. Ground-truth teacher vectors supply a loss, never query inputs. At deployment the teacher and projection are removed. The retained queries must infer useful foresight from the same observation-conditioned model context available during control. e03e04e06
5.2 Match a future frame to the action that reaches it
Temporal indexing is more precise than giving each query an arbitrary sequence position. The paper's j-th future frame is observed at t+jΔ, after action t+jΔ−1, so it corresponds to action token jΔ. Equation (4) clamps that index to L when the video horizon extends beyond the action chunk. Equation (5) applies the host positional map to each query at that corresponding index and preserves the original action indices. Multiple camera views of a future step therefore share its temporal location. In LingBot-VA, matching sequence, frame and noise IDs also preserves the existing autoregressive mask. The ablation compares this design with missing temporal PE and current-frame supervision. Its higher full-model score supports temporal grounding, while leaving the effect of horizon clamping untested. e05e07e14
5.3 Separate portability, robustness and causal explanation
Reader analysis: the strongest causal comparisons are each backbone against its own Robust-WAM variant, because their data and schedules are matched. FastWAM and GE-Act supply the action-expert evidence. The uncropped Table 3 adds LingBot-VA: after 2,500 clean RoboTwin demonstrations and 15K steps, randomized success rises from 29.8% to 34.4%, while clean success remains 81.2%. That supports portability to a unified model, although randomization still causes a large drop. The real-robot protocol then narrows the intervention to lighting and measures executed actions. Together these experiments triangulate the method's utility. They do not prove that semantic invariance alone explains every gain, or that a semantic-latent architecture must underperform: cross-family recipes differ, and the paper reports no matched pretraining ablation. e08e10e12e13e15e16e17
5.4 Training and inference
During training
The original video and action flow losses remain active. The frozen VAE encoder and frozen DINOv3 teacher have distinct roles; the added queries and projection learn semantic alignment. This is post-training of an existing WAM, without a newly specified semantic-space video pretraining stage. e03e04e06
Use eight future frames and offline DINOv3 ViT-B/16 CLS targets of dimension 768, with alignment weight 0.1. Two cameras imply 16 queries on LIBERO and the real robot; three imply 24 on RoboTwin. Baseline/variant data, schedule and optimization match within each backbone; different backbones retain different released recipes. e08
During inference
Remove the DINOv3 teacher and alignment head but retain the learned query inputs and inherited attention path. The stated flow formulation generates actions from Gaussian noise by integrating from s=1 to s=0. The semantic target is auxiliary supervision, not an inference-time oracle, planner or separate inverse-dynamics model. e04e06
Exact backbone-specific video inference, integration step counts and observation-refresh/action-execution schedules are not restated. Retained queries add sequence elements; teacher removal alone does not establish zero latency overhead. e08e17
5.5 Implementation flow
- Retain coupled video and action prediction
The frozen VAE encodes future frames spaced Δ control steps apart. An H-step action chunk becomes L=H tokens. Action blocks attend to video hidden states, transferring generative features into action prediction; preserving the video path does not mean freezing the entire video transformer. e03e04
- Give each query a temporal destination
Prepend K=T_f C learned queries to the action sequence, with one query for each future frame and camera. Apply the backbone's positional map Φ at the matching action index; action tokens retain indices 1 through L. Frames beyond the action horizon share its final positional index. e05
- Supervise hidden foresight
Frozen DINOv3 CLS vectors summarize ground-truth future frames. A linear head g projects query output states into this target space. Queries and action tokens interact through existing attention, while the original action head receives only the L action outputs. Future teacher vectors never enter query inputs. e06
- Respect the host architecture
Action-expert models use expert self-attention and existing video cross-attention. LingBot-VA queries inherit matching sequence, frame and noise IDs: same-step action/query interaction is bidirectional, but cross-step autoregressive ordering remains. e07
6. Experiments & results
Robust-WAM regularizes an existing video-generation WAM's action stream with future semantic targets while retaining its VAE-space video path. Temporally indexed learnable queries receive frozen DINOv3 supervision during training and remain as internal tokens at deployment. Controlled backbone comparisons show higher average OOD manipulation success; the evidence concerns executed policies, not merely plausible generated videos.
6.1 Read the original evidence
Table 1. The matched WAM pairs improve average perturbed-task success while retaining clean performance. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the two shaded Robust-WAM rows near the bottom, and compare each with the backbone immediately above it. The left numerical column is clean LIBERO; the right is LIBERO-Plus after training on the four standard suites. Values are average success percentages over evaluation trials. Parentheses show percentage-point changes from that particular backbone, not relative percentage improvements. FastWAM has the larger OOD gain but a lower final OOD score than GE-Act. Read the upper VLA rows and LDA-1B as contextual comparisons: Section 4.1 explicitly says that training regimes differ across WAMs, even though each baseline/variant pair shares its recipe. e08e09e10e11e17
What it supports. GE-Act rises from 78.0% to 80.9% on LIBERO-Plus, with clean success moving from 96.5% to 97.3%. FastWAM rises from 49.7% to 58.9% OOD and from 97.6% to 97.9% clean. These paired results support the added post-training method across both tested action-expert backbones.
Where the evidence stops. The table gives no uncertainty or seed variation. Its cross-model ranking cannot establish that video pretraining alone causes the advantage over semantic-latent models; architecture and training recipe also differ. The aggregate score can conceal individual perturbation regressions.
Table 2. The gains vary by perturbation, with two small decreases preserved in the evidence. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read horizontally within one perturbation row before comparing different axes. The first pair of columns belongs to FastWAM and the second to GE-Act; the parenthesized deltas are percentage points. Init denotes robot initial state, BG background texture, and Noise sensor noise. Camera changes viewpoint, while Layout changes spatial arrangement, so the evaluation extends beyond photometric changes. The Overall row repeats the all-trial aggregate from Table 1; it should not be silently recomputed as an unweighted average of the seven rows. Use both the absolute success rate and its change: a large gain can still leave a difficult condition largely unsolved. e06e09e10e11e17
What it supports. FastWAM's largest gain is on sensor noise, from 37.9% to 56.8%; camera success improves from 16.3% to 28.7%. However, its initial-state score falls by 1.0 point, and GE-Act's light score falls by 0.2. The evidence supports broad average improvement, with remaining weaknesses and exceptions.
Where the evidence stops. Neither the small decreases nor differences between axis gains have reported uncertainty estimates. This table measures policy success, not direct invariance of the learned features; interpreting feature invariance as the mechanism remains a hypothesis motivated by the training objective.
Figure 3. The physical experiment holds the task scene fixed and changes the illumination. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The left panel identifies the Franka Research 3 arm and wrist camera, then defines standard light as in-distribution and purple/cyan light as OOD. Each horizontal task strip shows the same instruction under those three lighting conditions: placing carrot on bread, placing kiwi in a basket, and stacking the purple cup on the green cup. Read the snapshots as examples of the evaluated setting, not as the trial record. The accompanying setup states that training uses standard lighting only and that scene, objects and cameras remain fixed at evaluation. A fixed third-person camera supplements the wrist camera. e15e16
What it supports. This experiment targets an appearance shift while holding scene geometry and task instructions fixed. It therefore offers a more focused physical check of illumination sensitivity than a benchmark that changes poses, textures and lighting together. Table 5 supplies the executed-policy success rates; the image itself establishes the setting.
Where the evidence stops. Selected snapshots and success marks do not establish aggregate success or the distribution of failures. The source tests three tasks and two unseen light colors; it does not establish physical generalization to new objects or camera placements.
Table 5. Robust-WAM raises GE-Act's pooled OOD success from 57.3% to 80.0%. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each task has an In-D success column, an OOD success column and a downward-arrow gap column. The gap is the difference in percentage points between standard and unseen-light performance; lower is better. The final group averages across the three tasks. The caption's 25 rollouts per condition means 25 standard, 25 purple and 25 cyan trials per task and policy, as clarified in the setup prose. Thus each task has 50 OOD trials across the two colors. Compare the bottom two rows to assess the Robust-WAM change on the same GE-Act backbone, rather than attributing the FastWAM-to-GE-Act difference to alignment. e15e16e17
What it supports. The GE-Act variant improves OOD success from 64% to 92% on Carrot→Bread, 48% to 68% on Kiwi→Basket, and 60% to 80% on Stack Cups. Average OOD success reaches 80.0%, and the InD–OOD gap shrinks from 22.7 to 2.7 points. Average clean success also rises, from 80.0% to 82.7%.
Where the evidence stops. The table pools the two OOD colors and reports no confidence intervals, seed variation or raw rollout outcomes. It cannot show whether both colors improve equally. The fixed-scene illumination protocol supports a narrower conclusion than unrestricted real-world robustness.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Plus robustness after clean LIBERO training Train on four standard LIBERO suites; evaluate all perturbed LIBERO-Plus tasks and clean suites. Compare matched variants within each backbone. | GE-Act + Robust-WAM: 80.9 OOD, 97.3 clean; FastWAM + Robust-WAM: 58.9 OOD, 97.9 clean. Average success rate (%) over evaluation trials | GE-Act: 78.0/96.5; FastWAM: 49.7/97.6 (OOD/clean). OOD gains: 2.9 and 9.2 percentage points. Average robustness improves without a clean-score decrease. Cross-family rankings do not isolate the effect of video pretraining. e08e09e10 |
| LIBERO-Plus perturbation-axis diagnosis Same clean-trained FastWAM/GE-Act variants; success pooled within each perturbation axis. | FastWAM noise: 37.9→56.8 (+18.9); camera: 16.3→28.7 (+12.4). Axis success rate (%) and percentage-point change | FastWAM initial state: 44.4→43.4 (−1.0); GE-Act lighting: 95.4→95.2 (−0.2). Gains are heterogeneous and not universal across axes; camera robustness remains weak for FastWAM. e09e11 |
| RoboTwin clean-to-random transfer LingBot-VA: 2,500 clean bimanual demonstrations, 15K training steps, 100 episodes per task on clean and randomized splits. | Robust-WAM: 34.4 random, 81.2 clean. Mean task success rate (%) | LingBot-VA: 29.8 random, 81.2 clean; random improvement 4.6 percentage points. Supports applicability to a unified architecture; randomized performance still falls far below clean performance. e12e13 |
| Semantic-target and temporal-query ablation FastWAM on full LIBERO-Plus; matched data and schedule; 16 queries except the 32-query combined-target variant. | Future DINOv3 CLS with temporal queries: 58.9. Average success rate (%) | Baseline 49.7; no temporal PE 55.0; current-frame target 52.7; DINOv3 Patch 57.0; DA3 56.8; CLS+DA3 56.5. Future supervision and temporal indexing each help under this recipe. Combined targets also alter query count, so their failure is not an isolated test of semantic/geometric complementarity. e14 |
| Real-robot robustness to unseen illumination Three Franka Research 3 tasks; standard-light training; 25 rollouts per task/policy under standard, purple and cyan light, giving 50 OOD rollouts. | Robust-WAM on GE-Act: 82.7 InD, 80.0 OOD, gap 2.7. Mean task success (%) and InD–OOD gap (percentage points) | GE-Act: 80.0 InD, 57.3 OOD, gap 22.7. FastWAM: 64.0 InD, 26.7 OOD. Physical policy execution supports an illumination-robustness benefit with fixed scenes. OOD success increases 22.7 points over GE-Act; separate color scores and uncertainty are absent. e15e16e17 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Future targets and temporal indexing outperform the tested alternatives under the same FastWAM recipe. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The baseline at the top is 49.7% success. Group (i) changes the teacher target: DINOv3 Patch uses compressed patch features, DA3 uses DepthAnything3 features, and the combined variant uses both target types. Group (ii) changes query design by removing temporal positional encoding or replacing future supervision with the current observation. The two shaded 58.9 rows represent the same full design in these two comparisons, not independent replications. Read the setup on this page alongside the crop: ordinary variants use 16 queries, but CLS plus DA3 uses two groups totaling 32. That extra change matters when interpreting its lower score. e05e06e14e17
What it supports. The full future-CLS design reaches 58.9%, exceeding no temporal PE at 55.0% by 3.9 points and current-frame supervision at 52.7% by 6.2 points. CLS also exceeds the tested patch and geometric targets. The results favor both future information and explicit temporal correspondence for this backbone.
Where the evidence stops. The combined-target comparison changes query count as well as supervision, and patch supervision changes the alignment head. It cannot isolate inherent incompatibility between semantic and geometric information. No repeated-seed uncertainty accompanies these ablations.
7. Analysis & limitations
7.1 What the evidence leaves open
The paper's assertion that semantic supervision should enhance rather than replace video pretraining is stronger than the controlled evidence: different model families use different recipes. Within-backbone gains support the added method, but do not isolate pretraining as the cause of cross-model ranking. e08e10e13
No seed variation, confidence intervals or latency measurements are reported. Three fixed-scene real tasks under two unseen light colors do not establish robustness to changed objects, geometry or long-horizon deployment. e15e16e17
7.2 Questions for discussion
- Does temporal correspondence still help when several future frames clamp to the same final action index?
- Would a query-count-matched combined-target ablation change the conclusion about semantic and geometric supervision?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the selected backbone checkpoint and released recipe, camera-aligned future frames/actions, cached frozen CLS targets, and matching query/action position IDs. The PDF leaves full optimizer settings, exact horizons/strides, hardware, software versions, real-demo count and deployment scheduling unspecified or delegated. e05e07e08e15e17
Reader-proposed checks: compare correct versus shuffled temporal query/action correspondence with fixed targets and query count; separately repeat real-lighting evaluation with per-color trial counts and uncertainty. These test temporal grounding and whether the pooled illumination gain masks a weak condition. e05e14e15e16
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Break temporal correspondence while holding the query budget fixed
Reader-proposed experiment, not performed: reproduce the FastWAM full design and no-temporal-PE variant on LIBERO-Plus, then add a control that consistently permutes query-to-action positional indices while retaining each query's original future-frame target. Keep 16 queries, frozen CLS targets, alignment weight, training data, optimizer schedule and evaluation episodes matched; repeat across seeds. Report overall and per-axis success plus alignment loss. The temporal-grounding explanation predicts correctly indexed queries should outperform the permuted control. Equal policy success would weaken the claim that the specific correspondence matters, even if both outperform the unregularized baseline. Record how many frame indices are clamped by the inherited horizon. e05e08e09e11e14
Check 2: Resolve the pooled physical illumination result
Reader-proposed experiment, not performed: evaluate GE-Act and its Robust-WAM variant on the same three Franka tasks with matched resets and the source's standard-only training data. Preserve 25 trials per task, policy and light condition, randomize evaluation order, and retain outcomes separately for standard, purple and cyan light. Report success counts and uncertainty for each task/color, then reconstruct the pooled OOD mean and InD–OOD gap. A stable robustness benefit should appear under both unseen colors rather than depend on one favorable condition. A concentrated gain or intervals consistent with no improvement would narrow the source's pooled conclusion. Record any departures from the fixed-scene protocol. e15e16e17
8.3 Reading coverage
Visual audit: All 13 supplied PDF pages were rendered and visually inspected, including the title and affiliations, Figures 1–3, Tables 1–5, method equations, implementation/evaluation prose, conclusion and references. Six original crops were inspected at final resolution; narrow simulation tables use a higher-DPI PDF render. Figure 2's query notation differs from Section 3.2 and is explicitly explained. Table 3 is visually read on page 8 but not cropped. No appendix occurs in the supplied PDF. Separate supplements, code and earlier revisions were not supplied for inspection; no reproduction was run.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title/author block (p. 1)
- 1 Introduction (pp. 1–2)
- 2 Related Work (pp. 3–4)
- 3.1 Preliminaries: World-Action Models (p. 4)
- 3.2 Semantic Foresight Alignment (pp. 4–5)
- 3.3 Integration with Different Architectures (pp. 5–6)
- 4.1 Experimental Setup (p. 6)
- 4.2 OOD Robustness on LIBERO-Plus (pp. 6–7)
- 4.3 Generalization Across Different WAM Architectures (pp. 7–8)
- 4.4 Ablation Study (p. 8)
- 4.5 Real-World Deployment (pp. 8–9)
- 5 Conclusion (pp. 9–10)
- References (pp. 10–13)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All supplied text and all 13 PDF pages were read; original figure/table images were inspected separately, resolving the extraction-only visual omission.
- Identity/version: the exact title and all 15 catalog authors agree with the title page, including the displayed capitalization Yingjie CAI. The reviewed artifact is arXiv:2608.05903v2, whose margin reads 7 Aug 2026. No earlier revision or revision history was supplied, so changes from v1 cannot be established.
- The catalog affiliation field includes trailing introduction prose; verified metadata uses only the three institutions printed on the title page.
- No code, linked project materials or separate supplements were inspected, and no experiments were reproduced. No appendix is present in this PDF.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin
The exact title and all 15 authors match the catalog identity. The PDF is labeled arXiv:2608.05903v2 [cs.CV], 7 Aug 2026, with a separate August 2026 date. Three affiliations are printed.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section 1; p. 2, Figure 1 and caption
The authors motivate preserving pretrained VAE-space video dynamics while adding appearance-insensitive semantic supervision to the action stream; Figure 1 is a conceptual comparison.
Go to primary source ↓e03PDF p. 3, Figure 2, legend and caption; p. 5, Section 3.2
The graphic separates video flow matching, action flow matching and semantic alignment, with shared positional encodings. Its legend denotes learnable query inputs by tilde-q and predicted semantic outputs by q; Section 3.2 instead denotes learnable inputs by q and output hidden states by bar-h. The frozen teacher and alignment head are training-only.
Go to primary source ↓e04PDF p. 4, Section 3.1, Eqs. (1)–(3) and final paragraph
A frozen VAE encodes future observations; video and action streams use flow matching, and actions attend to video hidden states. Flow time zero is clean and one is noise. Action generation integrates the learned velocity from one to zero.
Go to primary source ↓e05PDF p. 5, Section 3.2, Per-Frame Semantic Query Tokens, Eqs. (4)–(5)
There is one query per future frame and camera. Queries reuse the action positional map at min(jΔ,L); action indices remain 1 through L. Future frames beyond the action horizon share the final action step.
Go to primary source ↓e06PDF p. 5, Section 3.2, Semantic Foresight Targets and Alignment Objective, Eqs. (6)–(7)
Frozen DINOv3 CLS targets supervise projected query outputs through mean cosine distance. Teacher features are never query inputs. Query and action tokens interact through the existing attention; the teacher and linear projection are removed at inference.
Go to primary source ↓e07PDF pp. 5–6, Section 3.3
Action-expert WAMs prepend queries to expert tokens and use existing video cross-attention. LingBot-VA inserts queries into the action segment with matching sequence, frame and noise IDs; its mask permits same-step bidirectional interaction while retaining autoregressive ordering across steps.
Go to primary source ↓e08PDF p. 6, Section 4.1, Implementation Details
Eight future frames give 16 queries for two-camera LIBERO and real-robot setups, or 24 for three-camera RoboTwin. Targets are offline frozen DINOv3 ViT-B/16 CLS vectors of width 768; alignment weight is 0.1. Horizons, strides and training recipes are inherited, and regimes differ across WAMs.
Go to primary source ↓e09PDF p. 6, Section 4.2, Setup
FastWAM and GE-Act are trained on the four standard LIBERO suites, then evaluated on clean suites and all LIBERO-Plus perturbed tasks across camera, lighting, background, layout, robot initial state, language and sensor noise.
Go to primary source ↓e10PDF p. 7, Table 1, FastWAM/GE-Act and + Robust-WAM rows, both columns; Section 4.2 Results
Success percentages are FastWAM 97.6/49.7 versus 97.9/58.9 with Robust-WAM, and GE-Act 96.5/78.0 versus 97.3/80.9, for clean LIBERO/LIBERO-Plus. The table reports averages over evaluation trials without uncertainty estimates.
Go to primary source ↓e11PDF p. 7, Table 2, all perturbation rows and Overall row
FastWAM improvements are +7.3 layout, +12.4 camera, −1.0 initial state, +5.5 language, +11.3 light, +10.3 background and +18.9 noise percentage points. GE-Act improves on six axes, with light decreasing from 95.4 to 95.2. FastWAM camera success rises from 16.3 to 28.7.
Go to primary source ↓e12PDF p. 7, Section 4.3, Setup
LingBot-VA and its variant use 2,500 official clean-split RoboTwin bimanual demonstrations, train for 15K steps, and evaluate with 100 episodes per task on clean and randomized conditions with unseen poses, textures and lighting.
Go to primary source ↓e13PDF p. 8, Table 3, LingBot-VA and + Robust-WAM rows; Section 4.3 Results
Mean task success is 81.2% clean for both models and 29.8% versus 34.4% randomized. The table also compares independently configured VLA and semantic-latent WAM baselines.
Go to primary source ↓e14PDF p. 8, Section 4.4, Setup and Table 4, all rows
FastWAM LIBERO-Plus success is 49.7 baseline; 57.0 DINOv3 Patch; 56.8 DA3; 56.5 CLS+DA3; 58.9 CLS; 55.0 without temporal PE; and 52.7 current-frame targets. Single-target/query-design variants have 16 queries; CLS+DA3 has 32. Patch targets use an MLP compression head; variants share data and schedule.
Go to primary source ↓e15PDF pp. 8–9, Section 4.5 Setup; p. 9, Figure 3 and caption
Three Franka Research 3 tasks use third-person and wrist cameras. Training uses standard illumination; evaluation changes only lighting to purple and cyan with scene, objects and cameras fixed. Each policy/task has 25 standard and 25 rollouts under each unseen color, totaling 50 OOD rollouts.
Go to primary source ↓e16PDF p. 9, Table 5, all task and Avg. columns; Section 4.5 Results
GE-Act has 80.0% InD and 57.3% OOD average success, versus 82.7% and 80.0% with Robust-WAM; gaps are 22.7 and 2.7 points. OOD task rates change 64→92, 48→68 and 60→80 for Carrot→Bread, Kiwi→Basket and Stack Cups. FastWAM averages are 64.0%/26.7%.
Go to primary source ↓e17PDF pp. 4–9, Sections 3–4.5; pp. 9–10, Section 5; pp. 10–13, References
The supplied PDF defers backbone optimization and horizon settings to original recipes, does not specify hardware, software versions, full optimizer settings, latency measurements, training seed variation or confidence intervals, and ends with references rather than an appendix or dedicated limitations section.
Go to primary source ↓8.5 Primary sources
Robust-WAM: Bridging Generative Pretraining and Semantic Foresight in World-Action Models ↗
PDF · 7,286 extracted words
Source fingerprint
3fb998b1986c4e2fa8fc175de63a33f982d51ca85df8b71692219284ca114651