GlanceWAM: Sparse Test-Time Imagination for World-Action Models
1. Paper overview
In one sentence: GlanceWAM retains generated future guidance through a shared latent backbone and a fast action path, but its sparse-refresh deployment claim is limited by inconsistent scheduling descriptions. e02e03e07e09e11e12e13e14e24
| At a glance | What to know |
|---|---|
| Research problem | |
| Core mechanism | |
| A key reported result | RoboCasa kitchen manipulation: 72.2 Average success rate (%). 24 simulated tasks; 50 replay-filtered demonstrations/task; 1,200 evaluation episodes across five held-out layouts and unseen objects. Cosmos Policy 67.1 with 50 demonstrations/task; other Table 1 baselines use 300. A 5.1-percentage-point reported advantage. External baselines come from publications; headline confidence intervals are absent. e08e09 |
| Reading caution |
Core contributions
Figure 2. Separate the future being predicted from the future that guides actions. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper panel from left to right. Blue observations enter both VAE passes; the red future target enters the generative pass, while the green clean lookahead enters the policy pass. These are distinct training roles, even though both future frames come from demonstrations. In the central query/key mask, future tokens cannot read the clean lookahead and observations cannot read either future stream. Purple taps at layers 5, 12, 19 and 26 feed observation/lookahead features to the action head, together with instruction c and offset Delta. The lower panel shows the intended deployment interface: generate a latent destination, retain it and decode several action chunks against it. e03e04e05e06e07e13e14
What it supports. The design gives one backbone both a future-prediction objective and a future-conditioned action interface. Isolation prevents the clean guidance frame from providing a direct attention shortcut to the video target. At inference, the policy consumes the generated latent directly, so the diagram’s imagined future need not be decoded into pixels for control.
Where the evidence stops. The schematic pairs 0.8 s chunks with 48 ms decoding, but Appendix B measures 8-action LIBERO chunks. Its roughly three-second refresh timeline also differs from Appendix D’s tested every-chunk request pipeline; the drawing alone does not verify either combined performance claim.
2. Motivation
2.1 The problem and the proposed response
Generating a visual future every action cycle is expensive, while future prediction used only during training removes explicit destination guidance. The authors separate foresight and action horizons to retain guidance without making control wait for video sampling. e02e03
2.2 What this reading follows
A useful imagined future need not arrive on every control cycle. GlanceWAM trains a video transformer to predict a later scene and lets an action head use that scene as a latent destination. Its reported kitchen success and channel-removal ablation show that the learned policy uses this guidance. The engineering claim needs careful scoping: headline latency times action decoding with a lookahead already cached, and the appendix tests a different refresh cadence from the main diagram. The six visuals explain information flow, task success, channel dependence, sampling budget and spatial sensitivity. The latency result remains in the evidence report, grounded in Appendix B. e02e03e07e09e11e12e13e14e24
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | IDM |
| Quadrant | Q2 · One Model × IDM |
3.1 Evidence-based assessment
Supports the recorded classification
One Model × IDM is supported by the shared video DiT and explicitly inverse-dynamics action objective conditioned on current and future representations. This is architectural evidence beyond co-training. Lookaheads precede action selection and remain in inference, distinguishing the method from auxiliary future prediction or synchronous joint denoising. Cached multi-second guidance offers only limited evidence for general long-term memory. e03e04e05e07
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why training needs two different future frames
The future used to train prediction and the future used to guide action have different jobs. GlanceWAM asks its video objective to predict a frame at the full foresight horizon, while the policy receives a clean demonstrated lookahead at a randomly chosen intermediate offset. The latter prepares it to act as a cached destination becomes temporally closer. Putting both frames through an unrestricted encoder would let the target leak into guidance; unrestricted transformer attention would also let the video stream read the clean answer. The two VAE passes and three-class mask block these routes. Video and action losses still update the shared DiT, and the action head reads observation/lookahead features together with instruction and offset. At deployment, a generated latent replaces the demonstrated guidance frame. This transition is why the inference-time channel matters. e03e04e05e07
5.2 Separate an aging destination from an old source observation
Reader analysis: two clocks must be audited independently. Hold aging means the robot approaches a destination while retaining the same guide; randomized training offsets explicitly cover that situation. Source staleness means the guide was generated from a state that was already old when adopted. Appendix D says this second situation is not produced in training. Its successful pipeline requests guidance every 0.8-second chunk and adopts it one chunk later, reaching 70.8% against a paired 71.3% synchronous baseline. Separately, naive 3.2-second lag reaches 49.9%, below the paired zeroed floor. Those observations support prompt replacement of stale guidance. They do not demonstrate that the main diagram’s approximately three-second refresh cadence has the same envelope. Reproducing the system therefore requires timestamps for capture, generation and adoption, not merely an action-decoding timer. e05e07e14
5.3 What the ablation establishes about imagination
Reader analysis: the zeroing result establishes dependence more directly than it establishes a theory of imagination. A single-layer policy falls from 71.5% to 61.6% when its learned guidance channel disappears, but that intervention changes the input distribution. The sampler sweep adds another constraint: moving from one to ten steps sharpens decoded images without a corresponding success improvement. Appendix C further distinguishes ordinary variation among generated lookaheads from damaging off-manifold perturbations. Together these observations are consistent with a policy using coarse, plausible spatial guidance. They also fit the authors’ alternative explanation that teacher-forced lookaheads scaffold training and generated lookaheads preserve the resulting interface. The source explicitly leaves these explanations unresolved. A discriminating follow-up must manipulate the semantic correctness of a plausible destination while controlling sampling variability, rather than treating sharper images as inherently better plans. e11e12e15e16
5.4 Training and inference
During training
Train the DiT and action head end-to-end with equally summed video/action flow losses. Sample u uniformly over (0,H_f], embed it sinusoidally and drop lookahead tokens with probability 0.1. Clean training guidance is a demonstrated future. e03e05
Use 50 demonstrations/task, four H200 GPUs, bf16 and batch 64; reported lengths are 10k RoboCasa and 15k LIBERO steps. AdamW learning rates are 1e-5 for the backbone and 1e-4 for the head, with 5,000 warmups, cosine decay and EMA 0.999. e06e08e17
During inference
Generate an action-independent lookahead, cache its normalized latent and condition subsequent actions on it without VAE decoding. New observations feed each action pass. A clean backbone pass still feeds a four-step action flow head in the timed LIBERO configuration. e07e13
The main schedule refreshes near every 3 s and decays the offset across held chunks. Appendix D instead tests requests every 0.8 s with one-chunk adoption delay; its small asynchronous loss cannot establish the sparse-refresh schedule. e07e14
5.5 Implementation flow
- Separate prediction from guidance
SkyReels-V2-DF-1.3B receives clean observations, a noised future at H_f and a clean teacher-forced lookahead at u. Temporal slots distinguish their roles. The main formulation uses H_f near 3 s and H_a of 16 actions at 20 Hz, or 0.8 s. e03
- Prevent compression and attention shortcuts
Encode observation/future and observation/lookahead separately so the causal VAE cannot leak the full-horizon target into guidance. Observation queries read only observations; future queries cannot read lookaheads; lookahead queries read observations and themselves. e04
- Decode actions from shared features
The flow head cross-attends to observation/lookahead representations from layers 5, 12, 19 and 26, plus instruction and remaining offset. It learns inverse dynamics from current visual context toward a future visual condition. e05e06
6. Experiments & results
GlanceWAM uses one video transformer to imagine a future latent scene and condition faster action decoding on it. It reports 72.2% RoboCasa success and 99.0% LIBERO success; 48 ms measures an optimized LIBERO hold phase. Controlled ablations support the lookahead channel, while inconsistent refresh descriptions limit the deployment claim (e02, e09–e14).
Figure 7’s original Motus marker obscures the beginning of LingBot-VA’s parameter-count label, so that plot is not presented as a fully legible crop and the hidden text is not inferred. Figure 5 provides the sixth original visual instead. The reported LIBERO success and hold-phase latency remain in the base report, supported by Table 2 and Appendix B. Page 9 remains in the inspected set because its tables, horizon plot and surrounding discussion support retained claims. e10e13e23e24
6.1 Read the original evidence
Table 1. The headline kitchen result comes from a low-demonstration setting. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the rightmost column before comparing success rates. GlanceWAM and Cosmos Policy both use 50 demonstrations per task, whereas the remaining listed methods use 300. The success-rate column averages across 24 RoboCasa kitchen tasks. Section 4.1 specifies 50 evaluation episodes per task, giving 1,200 episodes across five held-out layouts with unseen object instances. The tasks include object transfers, doors, drawers, knobs and buttons, observed through two external cameras and a wrist camera. The upper block contains imitation or VLA policies, the middle block world-action models, and the final row the proposed model. Keeping the demonstration counts visible makes the comparison interpretable. e08e09e11
What it supports. GlanceWAM reports 72.2% success versus 67.1% for Cosmos Policy: a 5.1-percentage-point difference at the same reported demonstrations per task. It also exceeds the table’s 300-demonstration rows. This establishes the paper’s strongest aggregate kitchen result, with the internal component study providing a more controlled explanation of where improvement arises.
Where the evidence stops. Baseline numbers are imported from their publications, so equal demonstration counts do not establish equal pretraining, compute or implementations. The table gives no headline confidence intervals or task-level breakdown, and these are simulated manipulation results.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboCasa kitchen manipulation 24 simulated tasks; 50 replay-filtered demonstrations/task; 1,200 evaluation episodes across five held-out layouts and unseen objects. | 72.2 Average success rate (%) | Cosmos Policy 67.1 with 50 demonstrations/task; other Table 1 baselines use 300. A 5.1-percentage-point reported advantage. External baselines come from publications; headline confidence intervals are absent. e08e09 |
| LIBERO manipulation Four ten-task suites; 50 demonstrations and 50 evaluation episodes/task, 2,000 episodes total. | 99.0 average; Spatial/Object/Goal/Long: 99.4/100.0/99.0/97.8. Success rate (%) | Cosmos Policy and LingBot-VA 98.5 average; Fast-WAM 97.6. Near-saturation parity is more defensible than broad generalization. e08e10 |
| Lookahead conditioning ablation RoboCasa internal variants with shared backbone, data and evaluation; zeroing uses the single-layer checkpoint. | 64.4 plain; 71.5 single-layer; 61.6 same checkpoint zeroed; 72.2 multi-layer. Success rate (%) | Single-layer guidance adds 7.1 points; zeroing removes 9.9; multi-layer adds 0.7. The trained policy depends on the channel. Zeroing changes its input distribution and does not isolate why guidance helps. e11e15 |
| Lookahead sampler budget Single-layer checkpoint; paired RoboCasa reevaluation, 1,200 episodes/point. | 1/2/5/10/30 steps: 71.2/71.7/71.4/71.5/69.8. Success rate (%) | One step is 0.3 points below ten; the full range spans 1.9 points. More sampling is not monotonically better. This is not the multi-layer 72.2% result or a formal equivalence test. e12 |
| LIBERO hold-phase latency A100 SXM4 40 GB, bf16, batch one; 100 synchronized calls after 15 warmups; cached lookahead; 8-action chunks. | 47.95 mean, 0.39 standard deviation; quoted as 48. Milliseconds per chunk | Unoptimized hold 108.3 ms; unoptimized ten-step refresh-plus-action 478.6 ms; compiled Fast-WAM 91.5 ms for 32-action chunks. Fast action availability, not equal-horizon throughput or a measured complete concurrent-service budget. e13 |
| Source-staleness sensitivity Single-layer paired RoboCasa sweep, 1,200 episodes/arm; per-chunk pipelining and separate naive-lag arms. | Synchronous 71.3; pipelined 70.8 (p=0.72); naive 0.8/1.6/3.2 s lag: 68.8/60.5/49.9. Success rate (%) | The same sweep has a 62.7% zeroed floor. Frequent pipelining tolerates bounded delay; expired guidance can be worse than no guidance. These baselines differ from Table 3. e14 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Removing the learned lookahead channel hurts the same checkpoint. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the internal rows as two different comparisons. Moving from plain co-training to single-layer lookahead compares trained models and raises success from 64.4% to 71.5%. The indented zeroed row instead intervenes on that trained single-layer checkpoint at evaluation: its lookahead tokens are set to zero and success becomes 61.6%. Finally, the multi-layer row changes feature extraction and reaches 72.2%. Cosmos Policy is explicitly labeled an external anchor, so it is not another controlled internal variant. The accompanying text says internal methods share the backbone, data and evaluation pipeline; its footnote also warns that training-step comparisons across systems are not compute-matched. e11e14e15e19
What it supports. Single-layer conditioning adds 7.1 percentage points over plain co-training, while zeroing the learned channel removes 9.9 points from the same checkpoint. Multi-layer extraction adds another 0.7 points. These results show that guidance is used functionally; the zeroed policy’s score below the trained no-lookahead baseline also shows that removal is not a harmless fallback.
Where the evidence stops. Zeroing creates an input shift and cannot separate useful forecast computation from dependence on a training scaffold. Use the table’s 0.7-point multi-layer difference: the methods text’s 1.3-point statement is inconsistent. Appendix D uses different paired baseline and zeroed scores.
Table 4. Extra lookahead sampling gives no monotonic success improvement. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read columns as reevaluations of the same single-layer checkpoint, not separately trained models. The top row changes only the lookahead-generation Euler budget; each point uses 1,200 paired RoboCasa evaluation episodes. The bottom row measures manipulation success after that lookahead conditions the policy. One and ten steps give 71.2% and 71.5%, respectively, while two steps score 71.7% and thirty score 69.8%. Thus the lowest and highest sampling budgets are not ordered by task performance. Figure 8 supplies the qualitative counterpart: decoded lookaheads visibly sharpen with additional sampling. Keeping those two measures separate is central to interpreting the result. e12e15e16
What it supports. The observed spread is 1.9 percentage points across a thirty-fold range in sampler steps. The one-step setting lies only 0.3 points below the ten-step setting, making a cheap proposer plausible for this checkpoint. The table supports weak sensitivity to sampling budget over these settings, rather than better control from increasingly sharp generated images.
Where the evidence stops. The table supplies neither confidence intervals nor an equivalence-test margin. Its checkpoint is single-layer, so do not transfer its numbers to the final 72.2% multi-layer model. Similar scores also leave the underlying role of generated guidance unresolved.
Figure 5. Attention and intervention reveal different ways the policy uses lookahead features. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two halves as different measurements over the same three camera views. The left maps show cross-attention from action queries to generated lookahead tokens; the right maps show changes in predicted action chunks after localized 2-by-2 lookahead-token patches are replaced by observation tokens at corresponding coordinates. Each half is normalized separately across the two third-person views and the wrist view. The printed attention shares are 0.38, 0.38 and 0.24; the sensitivity shares are 0.27, 0.29 and 0.43. These are camera distributions, not success rates. The caption averages over twelve demonstration contexts, and Appendix C identifies the single-layer checkpoint used for the probes. e15e20e24
What it supports. The wrist view has a larger share of perturbation sensitivity than of attention, so attention weight alone does not rank how strongly each view affects action output. Appendix C separately reports a 0.399 value-weighted lookahead contribution and 0.340 raw attention mass. Together with the intervention maps, these measurements support active use of the guidance channel.
Where the evidence stops. The source supplies no calibrated heatmap color scale; use the printed shares for numerical comparisons. Twelve contexts do not establish generalization or a unique causal mechanism. The graphic prints 0.27 for the first sensitivity share, while the page-9 prose says 0.28; that discrepancy remains unresolved.
Figure 8. Sharper imagined scenes need not yield more successful actions. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow a task row across the five columns. The observation is the starting visual context; K=1, K=2 and K=10 are generated lookaheads with matched seeds within that row; the final column is the real demonstration frame three seconds later. The rows depict closing cabinet doors, moving squash from pan to plate, and moving a shaker from counter to cabinet. Inspect the wrist insets as well as the external view: low-budget images smear the arm or object, while ten-step images show sharper local structure. These images are decoded for analysis. The deployed action head consumes latent representations, so this grid visualizes the conditioning signal rather than an RGB control interface. e07e12e15e16
What it supports. The visual examples show fidelity improving with sampling, while Table 4 reports similar manipulation success across budgets. Together they support the authors’ interpretation that coarse destination information may suffice within the tested setting. Appendix C’s contrast with damaging off-manifold edits further suggests that arbitrary corruption is not interchangeable with reducing sampler steps.
Where the evidence stops. These are three selected demonstration contexts, not a distribution-wide fidelity metric or proof that every generated future is reachable. The final column is a demonstration target, not a rollout caused by the adjacent image. A training-scaffold explanation remains open.
7. Analysis & limitations
7.1 What the evidence leaves open
Evaluation is simulated; physical deployment is untested here. External baselines differ in data or supervision, and training-step comparisons are not compute-matched. e08e11e21
Appendix C leaves additional test-time computation versus a training scaffold unresolved. Attention and perturbation probes establish use, not a unique mechanism; horizon sweeps favor about 3 s only in the tested regime. e15e18
Preserve discrepancies: stated +8.4% foresight and +1.3% multi-layer gains disagree with Table 3 differences of 7.8 and 0.7 points. Figure 5 prints 0.27 where prose says 0.28. Appendix B phases total 48.9 ms, and its 1–2 ms serving overhead is unreconciled with the 47.95-to-48 ms quote. e19e20
Hold-aging training does not cover source staleness. Main-text three-second refresh, Appendix B’s 3–4 s account and Appendix D’s every-chunk pipeline are unreconciled; concurrent proposer resource contention is not quantified. e07e13e14
7.2 Questions for discussion
- Can three-second refresh maintain success when source age and concurrent GPU contention are measured? (e07, e13, e14)
- Does correct guidance outperform task-mismatched but plausible guidance after controlling sampling noise? (e15)
- Will benefits survive physical disturbances beyond the trained offset band? (e08, e14)
8. Reproducibility audit
8.1 Requirements and known gaps
Required: pretrained diffusion-forcing backbone, compatible causal VAE, two-pass encoding, exact mask and replay-filtered splits. Appendix E specifies optimizer and latent geometry but omits exact environment versions, evaluation seeds, action normalization and VAE/text-encoder freezing status. e03e04e08e17
Reproduce GPU preprocessing, cached text/mask/lookahead, CUDA-graph compilation and truncation to blocks 1–27 for timing. Measure concurrent refresh separately. A proposed mechanism check should compare generated guidance with resampling and task-mismatched guidance in paired episodes, not only zero tokens. e13e15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Measure sparse refresh and bounded-delay pipelining under the same workload
Reader-proposed check; not executed. Use one single-layer RoboCasa checkpoint, identical paired episodes and the same sampler budget. Compare synchronous current-observation guidance, main-text approximately three-second refresh, and Appendix D’s every-0.8-second request with one-chunk adoption. Log observation capture, proposal completion, adoption, target timestamp, action availability and GPU allocation. Include a zeroed-guidance control and measure both success and tail action latency with the proposer actually running. The decisive observation is whether sparse refresh preserves success and source-age bounds as well as frequent pipelining; failure would confine the reported staleness result to the latter. Report shared-GPU and separately served configurations distinctly. e07e13e14
Check 2: Test semantic destination guidance with plausible replacements
Reader-proposed check; not executed. In paired RoboCasa contexts, compare the normal generated lookahead, an independent same-budget resample, a lookahead generated for a task-mismatched destination with the same observation, and zeroed guidance. Keep the checkpoint, temporal offset, camera layout and sampler budget fixed. Inspect and match replacement fidelity and latent-distance ranges so gross corruption is not the only intervention. Record action displacement and per-task success. A selective loss for plausible but incorrect destinations beyond ordinary resampling variability would support destination-specific guidance. If effects track only latent mismatch, the experiment would leave the training-scaffold explanation unresolved. e11e12e15
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version block; Figures 1–8 and Tables 1–5; method equations and masks; evaluation and training details; Appendix B timing; Appendix C probes; Appendix D staleness and fidelity; and Appendix E configuration. Figure 7’s overlapping model-size label is explicitly unresolved and is not used for a numerical claim. The six final crops of Figure 2, Table 1, Table 3, Table 4, Figure 5 and Figure 8 were separately viewed with headers, labels and relevant visual context intact. Pages 11–12 were read as reference text but not rendered. No separate supplement, linked code or external baseline source was inspected. Crops omit long captions; the guides retain relevant protocols and boundaries. Page 9 supports the retained tables and discrepancies even though Figure 7 is not a selected crop; pages 13–14 supply the latency protocol.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3 Method (3.1–3.4)
- 4 Experiments (4.1–4.5, including 4.3)
- 5 Conclusion
- References
- Appendix A Concurrent Work
- Appendix B Latency Measurement
- Appendix C Mechanism Analysis
- Appendix D Additional Analyses and Figures (D.1–D.2)
- Appendix E Reproducibility and Training Details
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity: title and all ten authors match the catalog. The inspected artifact is arXiv:2608.23927v1, dated 25 August 2026; the catalog submittedDate is 24 August 2026. No other revision or edition was supplied or compared (e01).
- All six supplied text chunks were read, covering all 17 pages. PDF pages 1–10 and 13–17 were also visually inspected; reference pages 11–12 were read as text.
- The acquisition note that text extraction does not reconstruct figure images was addressed by inspecting the retained PDF and all six final crops.
- Separate supplemental material availability has not been fully verified.
- Code and external baseline publications were not inspected; no experiments were reproduced. The preliminary base report was not used as scientific evidence.
- Figure 7 has an original overlap: the Motus marker obscures the beginning of LingBot-VA’s model-size label. That label cannot be completely read and is not transcribed or reconstructed. Figure 5 supplies a legible diagnostic crop instead; Appendix B supports the retained latency result (e13, e23).
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author block and arXiv margin
The title is GlanceWAM: Sparse Test-Time Imagination for World-Action Models. Ten authors run from Linhan Wang to Chang-Tien Lu; affiliations are Virginia Tech, Drexel University, Northeastern University and Purdue University. The margin identifies arXiv:2608.23927v1 [cs.CV], 25 Aug 2026; the paper is marked Preprint.
Go to primary source ↓e02PDF pp. 1–2, Abstract, Introduction and Figure 1
The proposed remedy for synchronous video sampling latency is asynchronous latent lookahead generation with faster action decoding inside one video DiT.
Go to primary source ↓e03PDF pp. 3–4, Sections 3.1–3.2 and Figure 2
Language-conditioned observation history produces action chunks. The main setup uses 16 actions at 20 Hz, a 0.8 s action horizon and approximately 3 s foresight. Training separates clean observations, a noised full-horizon future and a clean randomly offset lookahead.
Go to primary source ↓e04PDF p. 5, Section 3.2, Two-pass causal visual encoding and Figure 3
Separate VAE passes encode observation/future and observation/lookahead pairs. The mask allows observation queries to read observations, future queries to read observations and future tokens, and lookahead queries to read observations and lookahead tokens.
Go to primary source ↓e05PDF p. 5, Section 3.3, Eqs. (1)–(2) and Staleness-robust horizon randomization
Video and action velocity losses are summed; the shared DiT and action head train end-to-end. The head receives observation/lookahead features, instruction and offset. Training samples u uniformly in (0,H_f], embeds the exact offset and drops lookahead tokens with probability 0.1.
Go to primary source ↓e06PDF pp. 6–7, Sections 3.3 and 4.1, Multi-layer visual extraction and Training details
Action conditioning uses layers {5,12,19,26}. Models initialize from SkyReels-V2-DF-1.3B and train on demonstrations, without online rollouts or auxiliary labels beyond RGB/actions, on four H200 GPUs; reported lengths are 10k RoboCasa and 15k LIBERO steps.
Go to primary source ↓e07PDF p. 6, Section 3.4
A 1–10-step ODE sampler produces a lookahead latent, retained without RGB decoding as slot-2 conditioning. The main account refreshes about every 3 s, reuses guidance across approximately four 0.8 s chunks and supplies a decaying offset.
Go to primary source ↓e08PDF p. 6, Section 4.1, RoboCasa kitchen and LIBERO; p. 7, Baselines
RoboCasa uses 24 tasks, 50 replay-filtered demonstrations/task, three views and 1,200 evaluation episodes in five held-out layouts with unseen objects. LIBERO uses four ten-task suites, 50 demonstrations/task, two views and 2,000 evaluation episodes. External success rates are taken from prior publications.
Go to primary source ↓e09PDF p. 8, Table 1, GlanceWAM, Cosmos Policy and other baseline rows
RoboCasa success is 72.2% for GlanceWAM and 67.1% for Cosmos Policy, both with 50 demonstrations/task. Other listed rows use 300 demonstrations/task; the highest of these are 66.4%.
Go to primary source ↓e10PDF p. 8, Table 2, GlanceWAM and comparison rows
GlanceWAM reports Spatial 99.4, Object 100.0, Goal 99.0, Long 97.8 and average 99.0 percent. Cosmos Policy and LingBot-VA average 98.5%; Fast-WAM averages 97.6%.
Go to primary source ↓e11PDF p. 9, Table 3; p. 8, Section 4.3 and footnote 1
Internal RoboCasa rows are 64.4% plain co-training, 71.5% single-layer lookahead, 61.6% with that checkpoint zeroed, and 72.2% multi-layer. Additional runs score 71.4% and 71.2%; the footnote says training steps across systems are not compute-matched.
Go to primary source ↓e12PDF p. 9, Table 4; p. 10, Section 4.5
Paired single-layer reevaluation uses 1,200 episodes per setting: 1/2/5/10/30 Euler steps give 71.2/71.7/71.4/71.5/69.8 percent success.
Go to primary source ↓e13PDF pp. 13–14, Appendix B, Protocol, Deployment inference path, Lookahead refresh and Same-hardware baseline comparison
Timing uses a LIBERO multi-layer checkpoint, A100 SXM4 40 GB, bf16, batch one, 15 warmups and 100 synchronized calls. An 8-action, four-step head yields 47.95 ms mean, 0.39 ms standard deviation, quoted as 48 ms. A refresh including action takes 478.6 ms on the unoptimized 10-step path. Fast-WAM uses 32-action chunks and measures 91.5 ms compiled.
Go to primary source ↓e14PDF pp. 15–16, Appendix D.1
Hold aging is trained; source staleness is not. Single-layer paired tests use 1,200 episodes/arm: synchronous 71.3%, zeroed 62.7%, one-chunk pipelining 70.8%, p=0.72 versus synchronous. The pipeline requests each chunk and adopts one 0.8 s chunk later. Naive 0.8/1.6/3.2 s lag yields 68.8/60.5/49.9%.
Go to primary source ↓e15PDF p. 15, Appendix C, layout, lookahead-read and candidate-mechanism paragraphs
Single-layer probes report 0.399 value-weighted lookahead share versus 0.340 raw attention mass. A two-step lookahead differs by 13.4% from ten steps; a same-magnitude off-manifold edit costs 28.2 points. Authors leave amortized test-time computation versus a training scaffold unresolved.
Go to primary source ↓e16PDF p. 16, Figure 8 and Appendix D.2
Three demonstration contexts compare observation, 1/2/10-step generated lookaheads and the real demonstration future at t+3 s. Seeds are matched within each row; more sampling visibly sharpens images. Insets show the wrist camera.
Go to primary source ↓e17PDF p. 17, Appendix E
The VAE downsamples time by 4 and space by 8, with 16 latent channels and 2-by-2 patches. Training uses bf16, batch 64, AdamW (0.9,0.95), epsilon and weight decay 1e-8, clipping 1.0, DiT/head learning rates 1e-5/1e-4, 5,000 warmups, cosine minimum 5e-7, attention dropout 0.2 and EMA 0.999.
Go to primary source ↓e18PDF p. 9, Figure 6 and Section 4.3, horizon sweep
The lookahead sweep rises from 66.3% at 1.4 s to 71.6% at 3.0 s and 71.1% at 3.8 s. Plain co-training benefits from extending its video horizon beyond 0.8 s; the text reports 63.1% to 65.7%.
Go to primary source ↓e19PDF p. 3, Test-time imagination and efficiency; p. 6, Multi-layer visual extraction; pp. 8–9, Section 4.3 and Table 3; p. 14, Appendix B
The text claims +8.4% foresight and +1.3% multi-layer gains, whereas Table 3 yields 7.8 and 0.7 percentage points for final-minus-plain and multi-minus-single. Appendix B lists phase times 8.1, 28.1, 0.6 and 12.1 ms alongside a 47.95 ms total and says a 48 ms quote includes approximately 1–2 ms serving overhead.
Go to primary source ↓e20PDF p. 7, Figure 4; p. 8, Figure 5; p. 9, Section 4.3
Figure 4 shows eight tasks with observation, generated lookahead and reached future; RGB decoding is for visualization. Figure 5 shows attention and patch-replacement sensitivity across cameras. Its first sensitivity share is 0.27, while the p. 9 discussion says 0.28.
Go to primary source ↓e21PDF pp. 13–14, Appendix A and Table 5
Concurrent-work comparisons use each paper’s own evaluation protocol. DeVA reports auxiliary affordance/depth supervision and Flex-pi uses additional future modalities; these are not controlled within-GlanceWAM ablations.
Go to primary source ↓e22PDF p. 15, Appendix C, Which video pretraining matters
SkyReels-V2-DF, Wan2.1 and Self-Forcing-DMD score 71.5, 70.3 and 68.9 under the authors’ recipe; pairwise margins are within one evaluation sigma. Cosmos-Predict2 scores 8.3 in this regime and lacks the required per-frame-timestep pretraining interface.
Go to primary source ↓e23PDF p. 9, Figure 7, overlapping Motus marker and LingBot-VA parameter label
The original red Motus marker covers the beginning of the adjacent purple LingBot-VA model-size label. The overlap is present on the PDF page and prevents complete visual transcription of that label.
Go to primary source ↓e24PDF p. 8, Figure 5 panels and caption; p. 15, Appendix C, How the lookahead read was measured
Figure 5 averages probes over 12 demonstration contexts and normalizes shares within each half. Attention shares for third-person L/R and wrist are 0.38/0.38/0.24; sensitivity shares printed in the graphic are 0.27/0.29/0.43. The sensitivity probe replaces localized 2-by-2 lookahead-token patches with observation tokens at corresponding coordinates and measures predicted-action displacement. Appendix C distinguishes 0.399 value-weighted lookahead share from 0.340 raw attention mass and states that these analyses use the single-layer checkpoint.
Go to primary source ↓8.5 Primary sources
GlanceWAM: Sparse Test-Time Imagination for World-Action Models ↗
PDF · 8,937 extracted words
Source fingerprint
6a14f9d1a257593bad15aacb9bc3ce73f14f9adc675e4424327cce135fc54850