WAM4D: Fast 4D World Action Model via Spatial Register Tokens
1. Paper overview
In one sentence: WAM4D uses future-depth supervision to train causal video-action features, then removes the geometry branch for efficient control, with gains that depend on the evaluation setting. e02e04e05e08e11e13e16
| At a glance | What to know |
|---|---|
| Research problem | Source description Manipulation needs object extent, free space and contact geometry that plausible RGB predictions may miss. The paper asks whether future-depth supervision can strengthen the causal features used for actions without paying for dense geometry decoding during deployment. e02 |
| Core mechanism | |
| A key reported result | RoboTwin 2.0 full-suite control: Table 1: 93.8 clean, 89.9 randomized, 91.8 average. Success rate (%). Unified policy; 50 tasks; clean and randomized settings. LingBot-VA: 92.9/91.6/92.3; Fast-WAM: 91.9/91.8/91.8. WAM4D leads clean success but not randomized or average success. Table 3's baseline aggregates differ slightly from Table 1; its baseline rows are imported from earlier reports, limiting claims of completely matched retraining. e10e11 |
| Reading caution |
Core contributions
Figure 2. Spatial registers read causal video features while remaining outside the action predictor's inputs. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the left tower: history RGB and actions feed the video-action backbone, which produces video and action predictions. The blue depth tower reads intermediate features and sends four feature sets into a geometric head. Its arrows carry features toward the auxiliary readout, not depth predictions back into the policy. Then read the matrices by query rows and key columns: gray cells mark allowed visibility. The action row can see historical video and actions but not future video; the depth rows can see historical video and registers, but neither action columns nor future-video columns. The actual default taps are layers 12/14/16/18, despite the schematic block labels. e03e04e05e06e07e16
What it supports. The geometry branch supervises information already available to the causal policy. Eq. (7), Section 3.3 and the page-7 visibility table corroborate the illustrated separation. As a reader inference, the mask explains why deleting the auxiliary readout can preserve the action computation after the shared features have been trained.
Where the evidence stops. This is an attention/data-flow diagram, not evidence that generated depth controls the robot. The final geometric head is trainable according to the objective and head ablation; the absence of a flame icon on that box does not imply it is frozen.
2. Motivation
2.1 The problem and the proposed response
Manipulation needs object extent, free space and contact geometry that plausible RGB predictions may miss. The paper asks whether future-depth supervision can strengthen the causal features used for actions without paying for dense geometry decoding during deployment. e02
2.2 What this reading follows
A robot can generate a plausible video while still misunderstanding the space around its gripper. WAM4D addresses that gap by making history-video features support future-depth prediction during training. Learnable spatial registers provide the readout, and a pretrained geometric head supplies an adaptable prior. The critical detail is visibility: the action predictor cannot read those registers or future-video tokens. This permits deployment with real observations and action denoising alone. The paper's evidence supports a useful geometry-training mechanism, but its comparisons are mixed: clean simulation success is strong, randomized success is lower than leading baselines, and the physical study reports sub-actions rather than complete-task success. e02e04e05e08e11e13e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Conflicts with the recorded classification
The WAM and geometry/efficiency tags fit. However, Eqs. (4)–(5) describe one integrated video-action MoT sequence with prediction heads, while the separate geometry branch is removed for deployment. This does not support a planner-plus-policy Dual-system reading. The mechanism is joint future-video/action training with masked action-only inference, not inverse dynamics. MoT experts alone do not establish two deployed systems; the recorded composite classification therefore needs editorial review. e03e04e05e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make history features answer a question about future geometry
Begin with the information available at a decision: language, historical camera images and executed actions. During training, the backbone also receives noisy future video and action states so it can learn their flow targets. WAM4D's addition is a set of learnable queries arranged on the camera mosaic. The same spatial grid is copied across future depth timesteps; time and image coordinates are encoded through RoPE. Each register can read itself, other registers and historical video features. Four intermediate register readouts are adapted to the pretrained depth head's input space. The depth loss therefore asks the history representation to support future-depth reconstruction. Backpropagation updates those shared features even though the policy is prohibited from reading the depth tokens. This is the paper's concrete route for transferring geometric priors. e03e04e05e06e07
Figure 3. The training objectives are larger than the computation retained for deployment. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two halves as different operating modes. On the left, striped future RGB/action tokens are prediction states for the flow objectives, while the depth blocks and geometric head contribute the future-depth loss. The two loss groups are optimized together, with both weighting coefficients set to one. On the right, the geometric head and registers have disappeared. Real observations and executed-action history fill the backbone's cache, which supports action prediction. Algorithm 1 specifies the loop behind the arrows: encode the observation queue, replace the video cache, update the action cache, denoise a chunk, execute it and collect observations every four actions. e03e06e07e08e12e17
What it supports. The default controller predicts 32 actions per chunk and uses fresh observations from execution. A separate analysis mode can keep the depth branch and generate RGB-D rollouts, but the paper does not use those imagined point clouds as the default controller's feedback. This distinction is essential when interpreting its 4D visualizations.
Where the evidence stops. Removing geometry decoding does not make the whole control loop instantaneous. Table 9 still reports 525.43 ms mean action-generation latency on one A800; queue limits, capture delays and complete sensor-to-actuator timing are not specified by this diagram or algorithm.
5.2 Use the visibility rules to understand what can be deleted
A reader can trace the deployment simplification directly from the mask. Future action queries need history-video keys, history-action keys and their own noisy action chunk. They do not need future-video keys or register keys. The auxiliary readout can consequently train the shared representation without becoming an action-time dependency. This is a structural deduction from the specified attention pattern, not merely an inference from joint training. At deployment, Algorithm 1 encodes real observations into the cache, supplies executed actions, generates a new chunk and refreshes observations during execution. By contrast, the RGB-D examples retain the depth branch and roll forward generated futures. Their reconstruction quality and occlusion failures concern that analysis path; neither constitutes evidence that a geometric planner selects the robot's executed actions. e05e08e17e18
5.3 Ask whether geometry training improves the control decision
The decisive comparison is not whether the depth head can draw a recognizable scene. It is whether changing that training signal improves executed control under a stated protocol. Table 8 supplies a useful sequence: random depth-head training performs worse than no depth on clean success, frozen pretraining helps, and trainable pretraining helps more. That supports the specific prior-and-adaptation design within the ten-task experiment. Table 7 adds a warning: shallow registers can improve RGB metrics while trailing middle registers on control, and bidirectional visibility changes the deployment dependency. Finally, the full-suite and real-robot tables bound the conclusion. Clean simulation improves, randomized simulation does not lead the baselines, and the physical average combines intermediate goals with an omitted bottle-placement column. These boundaries matter when choosing a reproduction target. e11e13e14e15e16
5.4 Training and inference
During training
Sample at most 17 frames: history lengths 1/5/9 when available, plus eight future targets every four collected steps. Mask padded latent slots for video loss, while every valid depth frame contributes. Head images are 256 × 320 and wrist images 128 × 160. Position normalization uses dataset q01/q99; quaternion and gripper bounds are −1 and 1. e07
Video/action conditional flow matching and depth SmoothL1 are optimized together. The final pretrained geometric head is trainable; depth gradients also update the backbone, registers, blocks and adapters. AdamW uses learning rate 2 × 10^-5√N for N machines, ten warmup steps, clipping 2.0, bf16, 50k main steps and 10k ablation steps. e06e09e16
RoboTwin training spans 50 tasks with 50 clean and 500 randomized trajectories per task. Real-world training uses 400 AstriBot demonstrations across plate lifting, bottle placement, pen-cap removal and LEGO sorting, with offline DA3 pseudo-depth. e10
During inference
Delete registers, depth blocks and the geometric head. VAE-encode the observation queue, replace the video KV cache, and encode executed-action history into the action cache. Denoise and execute a 32-step chunk; collect an observation every four actions. This loop uses real feedback rather than generated depth for control. e07e08
Retaining the depth branch enables a separate autoregressive RGB-D/point-cloud visualization mode. These imagined futures are not the default policy's sensory feedback or a demonstrated geometric planner. e08e17
5.5 Implementation flow
- Encode causal context and prediction states
LingBot-VA supplies the video-action backbone and Wan2.2 supplies the video VAE. History latents, noised future-video latents, history-action embeddings and noised future actions form one MoT sequence. Future clean frames and actions construct training targets; they are not additional causal observations. e03e07
- Read future depth from history features
Copy a learnable spatial grid across eight future depth frames. The three-view mosaic gives a 12 × 10 grid per frame, hence 960 registers. At layers 12/14/16/18, register queries read registers and history video; temporal/spatial RoPE supplies positions. Four linear adapters feed a DA3-GIANT-1.1 DualDPT head. e04e07
- Keep the action path causal
Future actions attend to history video, history actions and their own noisy chunk. Registers cannot read action or future-video tokens, and the policy cannot read registers. Future video reads history video and future-video noise. Figure 2 agrees with the explicit mask table. e05
6. Experiments & results
WAM4D trains a causal video-action model to expose future geometry through auxiliary spatial registers, then deletes the depth branch for control. Its strongest mechanism evidence is the benefit of a trainable pretrained depth head; full-suite results show competitive success and reduced memory, with mixed accuracy and latency advantages.
6.1 Read the original evidence
Table 1. Clean success, randomized success and inference cost identify different winners. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read Clean and Rand. separately before consulting Avg.; these are success percentages across the 50-task suite. WAM4D has the strongest clean entry, while Fast-WAM leads the randomized column and LingBot-VA leads the average. Move right to compare latency and memory, whose lower values are preferable. The timing setup on page 15 uses one A800 80GB GPU and ten action-denoising steps. LingBot-VA and Motus also perform five video-denoising steps, whereas WAM4D and Fast-WAM use action-only inference. The VRAM figure is peak allocated video-action backbone memory, rather than a full accounting of training or robot-system memory. e10e11e12
What it supports. WAM4D reports 93.8% clean, 89.9% randomized and 91.8% average success. Its 9.71 GiB memory is below LingBot-VA's 12.97 and Fast-WAM's 11.55 GiB. Its 525.43 ms mean latency falls between those methods: faster than LingBot-VA at 843.57 ms and slower than Fast-WAM at 425.53 ms.
Where the evidence stops. Table 3 imports per-task baseline results and has small aggregate discrepancies: LingBot-VA randomized is 91.50% there versus 91.6% here, and Motus clean is 88.52% versus 88.7%. Preserve the table-specific values; do not claim every baseline was retrained under identical conditions.
Table 2. The physical result is an average over displayed intermediate goals. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each entry is a success fraction from ten physical rollouts per method per task. The seven displayed columns are plate S1, bottle S1, LEGO/Blocks S1–S3 and pen S1–S2. Table 4 defines plate S1 as lifting the plate, bottle S1 as grasping a bottle, each LEGO step as placing another brick in its matching box, and the pen steps as grasping then removing the cap. Read a task's columns in sequence: if one step fails, later steps are not attempted and receive zero. The Avg. column averages the displayed sub-action outcomes and should not be read as the fraction of four complete tasks solved. e10e13
What it supports. WAM4D's reported average is 0.90, versus LingBot-VA's 0.84 and Fast-WAM's 0.80. Its third LEGO step reaches 0.8 versus LingBot-VA's 0.4, a concrete long-horizon difference in this small study. WAM4D is not better on every column: its plate and bottle S1 values are both 0.9, below LingBot-VA's 1.0.
Where the evidence stops. Table 4 and Figure 4 include Bottle S2, placing the bottle into a tray, but this quantitative table omits it. Its value and complete bottle-task success cannot be reconstructed. Ten rollouts also give coarse 0.1 increments without reported confidence intervals.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| RoboTwin 2.0 full-suite control Unified policy; 50 tasks; clean and randomized settings. | Table 1: 93.8 clean, 89.9 randomized, 91.8 average. Success rate (%) | LingBot-VA: 92.9/91.6/92.3; Fast-WAM: 91.9/91.8/91.8. WAM4D leads clean success but not randomized or average success. Table 3's baseline aggregates differ slightly from Table 1; its baseline rows are imported from earlier reports, limiting claims of completely matched retraining. e10e11 |
| Action-generation latency and memory One A800 80GB GPU; ten action-denoising steps; geometry removed. | 525.43 ± 5.64 ms; 9.71 GiB. Mean ± SD milliseconds per chunk; peak allocated backbone GiB | Fast-WAM: 425.53 ± 6.01 ms, 11.55 GiB; LingBot-VA: 843.57 ± 11.55 ms, 12.97 GiB, including five video steps. Lower memory than both comparators, faster than LingBot-VA and slower than Fast-WAM. These are model-generation measurements, not complete sensor-to-actuator loop latency. e12 |
| AstriBot S1 reported sub-action success Four tasks; ten physical rollouts per method per task; seven displayed sub-action columns. | 0.90; LEGO/Blocks S3: 0.8. Mean reported sub-action success fraction | LingBot-VA average 0.84 and Blocks S3 0.4; Fast-WAM average 0.80; π0.5 average 0.74. This is not 90% complete-task success. Later steps are scored zero after failure. Bottle placement S2 is defined but absent from Table 2, so full bottle-task success cannot be recovered. e10e13 |
| Pretrained geometric-head ablation Fixed ten-task RoboTwin split; 10k training steps; 100 evaluation rollouts per task and condition. | Trainable pretrained: 80.1%; 0.049; 164.5. Clean success (%); AbsRel; FVD | Fixed pretrained: 75.2%, 0.053, 179.8; trainable random: 70.0%, 0.059, 189.8; no depth: 71.7%, depth unavailable, 181.2. Pretraining plus adaptation improves control and reconstruction in this subset. The clean gain over no depth is 8.4 percentage points, not a full-suite gain. No seed uncertainty is supplied. e09e14e16 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 7. Register placement trades image quality against control and geometric readout quality. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. These rows belong to the ten-task ablation setting with the depth head fixed, not the full-suite headline result. First compare shallow, middle and deep registers to isolate placement among unidirectional designs. Read the metric arrows: smaller FVD, LPIPS, AbsRel and CD1 are preferred; larger PSNR, threshold accuracy and overlap scores are preferred. Middle registers at layers 12/14/16/18 achieve stronger control and geometry than shallow registers, even though shallow has better selected RGB scores. To examine visibility, compare Uniform against Bidirectional, which both use 6/12/18/24. Comparing bidirectional directly against Middle also changes placement, so it cannot isolate visibility by itself. e05e09e14e15e19
What it supports. Middle registers yield 75.2% clean success and 0.053 AbsRel, versus shallow's 72.5% and 0.058. Bidirectional reaches 76.6% success but 0.074 AbsRel. The authors choose middle unidirectional registers as the practical compromise, keeping register features out of the main policy path; these results do not make RGB quality a reliable proxy for control.
Where the evidence stops. The VAE-head row lists layers 27–30, while the adjacent implementation prose says its readout is from layer 26; this remains unresolved. Bidirectional and no-depth comparisons also change parameter counts in Table 6. No repeated-seed uncertainty is reported.
Table 8. An adaptable pretrained head gives the clearest evidence for the proposed geometry-training mechanism. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Hold the default middle-register interface in mind, then compare the two trainable rows: they differ in whether the geometric head starts from random or pretrained weights. Next compare trainable pretrained with fixed pretrained to examine adaptation. No depth supplies the original policy baseline but lacks a depth branch, so its geometry entries are intentionally absent. Clean SR is control success; AbsRel and CD1 evaluate reconstruction error, while FVD concerns generated videos. Table 6 on page 11 supplies the complementary randomized results and per-task counts for the same ten-task runs. These ablations use the shorter 10k-step budget described on page 7. e09e14e16e19
What it supports. Trainable pretrained reaches 80.1% clean success, compared with 75.2% for fixed pretrained, 70.0% for trainable random and 71.7% without depth. Its AbsRel is 0.049 and FVD 164.5. The 8.4-percentage-point gain over no depth supports the complete training intervention on this subset; the random-head result shows depth supervision alone is insufficient here.
Where the evidence stops. This is a ten-task, shorter-budget experiment, not the 50-task result. No-depth also removes parameters, whereas the two trainable-head rows provide a closer initialization comparison. Agreement among geometry and success metrics does not independently prove a causal contact-reasoning mechanism.
7. Analysis & limitations
7.1 What the evidence leaves open
Autoregressive rollouts can replace occluded objects with different identities because explicit long-term memory is absent. The authors say fresh observations protect their evaluated control path; robustness under sustained real occlusion remains unestablished. e18e08
The real-robot study is small and omits Bottle S2 results. Table 1 and Table 3 disagree on some baseline aggregates. Table 7 lists VAE-head layers 27–30 while the prose says layer 26; neither configuration should be silently substituted. e10e11e13e15
Attention maps show selected correlations, not causal explanations. Section 4.4.1's reference to action-history cues should not imply direct register access to action tokens, which the stated mask forbids. Fixed-intrinsic point-cloud overlap also does not by itself establish calibrated metric geometry. e05e15e17e19
7.2 Questions for discussion
- Does the pretrained-head advantage persist across seeds and the complete randomized suite?
- Can explicit object memory improve sustained-occlusion control while preserving action-only deployment?
8. Reproducibility audit
8.1 Requirements and known gaps
Required ingredients include the named backbone/VAE/DA3 checkpoints, re-collected depth-annotated RoboTwin data, AstriBot demonstrations, action quantiles and exact temporal/mosaic alignment. The paper leaves batch size, actual training machine count, VAE freezing, detailed depth validity/pseudo-depth processing, queue limits, software versions and evaluation seeds unspecified. The A800 specification concerns inference measurements. e07e08e09e10e12
Resolve metric thresholds, depth scaling, the numerical intrinsic matrix and VAE-head layer ambiguity before comparing geometry scores. A useful proposed check repeats no-depth, frozen-pretrained and trainable-pretrained heads with matched data and seeds, reporting per-task success alongside depth error; another checks that removing masked auxiliary tokens leaves action predictions unchanged. e05e14e15e16e19
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether the auxiliary branches are truly outside action inference
Reader-proposed, not performed: load one trained checkpoint and fix language, RGB/action histories, future-action noise, denoising steps and numerical precision. Compare action outputs with the masked future-video/register branches present versus physically removed. Also perturb future-video noise and register values while holding the action inputs fixed. The stated mask predicts unchanged actions within numerical tolerance, with lower auxiliary computation after removal. A material output change would falsify the assumed separation or expose a mask/cache implementation error. Time both versions with the paper's ten action steps on the same hardware, reporting the measured scope separately from capture and execution delays. e03e05e08e12
Check 2: Separate geometric pretraining from extra training capacity
Reader-proposed, not performed: repeat Table 8 on the exact Table 5 ten-task split with matched demonstrations, backbone initialization, action horizons, optimizer, 10k-step budget and paired evaluation scenes. Use several declared seeds for no depth, trainable random, fixed pretrained and trainable pretrained heads; add a same-architecture control with depth gradients blocked from the backbone. Report per-task clean/randomized success and AbsRel, retaining the published 100-rollout-per-task protocol. Pretrained-head gains should persist beyond seed variability, and blocking depth gradients should reduce the policy benefit if representation transfer is responsible. Comparable success after blocking those gradients would weaken that explanation even if depth reconstruction remained good. e04e06e07e09e14e16
8.3 Reading coverage
Visual audit: The title/author block, all seven figures, all nine numbered tables, attention mask table, equations and deployment algorithm were visually inspected on pages 1–15. Figure 2's query/key shading and arrow directions were checked against Eq. (7), Section 3.3 and the page-7 mask; Figure 3 was checked against Eqs. (9)–(10) and Algorithm 1. Six final original crops were separately viewed, including higher-DPI renders for narrow result tables. Pages supporting training, data, metric definitions, task definitions, ablation settings, failure analysis and A800 inference measurements are included even when not cropped. Table 2's missing Bottle S2, the VAE-head layer discrepancy and baseline aggregate differences remain explicit. References on pages 16–19 were read as text only. No appendix or separate supplement was supplied; code and experiments were not inspected or executed.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, abstract and Section 1: Introduction (pp. 1–3)
- Section 2: Related Work (p. 3)
- Sections 3.1–3.5: backbone, registers, masks, objectives, implementation and Algorithm 1 (pp. 3–7)
- Sections 4.1–4.3: setup, simulation and real-world results (pp. 7–10)
- Sections 4.4–4.7: all ablations, qualitative rollouts, failure analysis and compute (pp. 10–15)
- Section 5: Conclusion and Limitation (p. 15)
- References (pp. 16–19)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity/version note: the inspected artifact is arXiv:2606.14048v3, stamped 7 July 2026, with a July 8, 2026 title-block date. The supplied catalog's submittedDate is July 7, 2026. Title and all 13 authors agree; earlier revisions were not supplied and no cross-version scientific equivalence is asserted.
- The supplied acquisition notes state: Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This limitation was addressed by inspecting PDF pages 1–15 and all six final crops.
- Separate supplemental material availability has not been fully verified. No separate supplement was supplied.
- No appendix occurs in this 19-page PDF. References were read as text; reference-only pages 16–19 were not visually inspected.
- The title-page code link was recorded as provenance only. Code was not inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block and arXiv margin stamp
The title and all 13 authors match the supplied catalog. The PDF identifies arXiv:2606.14048v3, stamped 7 July 2026, while its title block is dated July 8, 2026. The three affiliations and a code link are printed here.
Go to primary source ↓e02PDF pp. 1–2, Abstract and Section 1
The paper motivates future-depth supervision as a way to improve geometric representations without retaining dense geometry decoding in the deployed policy.
Go to primary source ↓e03PDF pp. 3–4, Section 3.1, Eqs. (1)–(5)
Language, RGB history and historical actions form the causal context. History and noised future video/action tokens occupy one sequence processed by video-action MoT blocks; separate heads predict the video and action flow targets.
Go to primary source ↓e04PDF p. 5, Section 3.2, Eqs. (6)–(8)
A shared register grid is repeated across future timesteps. Registers query themselves and history video features using temporal and mosaic-coordinate RoPE. Four intermediate readouts are projected into a pretrained geometric head to predict future depth; gradients reach history video features.
Go to primary source ↓e05PDF p. 4, Figure 2; p. 5, Section 3.3; p. 7, Attention mask table
The figure and explicit visibility table agree: future actions read history video, history actions and future-action noise; registers read history video and registers; future video reads history video and future-video noise. History video and history action each read only their own group. Text cross-attention conditions video and action tokens.
Go to primary source ↓e06PDF p. 6, Section 3.4, Eqs. (9)–(10)
The objective sums video and action conditional-flow losses and valid-pixel SmoothL1 future-depth loss. Both weights are one. Depth gradients update the backbone, registers, depth blocks, projection and geometric head.
Go to primary source ↓e07PDF pp. 6–7, Section 3.5, frame sampling, actions, view layout and geometric readout
Initialization uses LingBot-VA and the Wan2.2 VAE. Training samples 1/5/9 history frames when available and eight future frames at stride four, with padding masks. Three views yield 960 registers over eight depth frames; layers 12/14/16/18 feed four adapters into DA3-GIANT-1.1 DualDPT. Actions are 32-step chunks of 16-dimensional absolute bimanual end-effector/gripper commands with specified normalization.
Go to primary source ↓e08PDF p. 6, Figure 3; p. 7, Algorithm 1
Deployment removes registers, depth blocks and the geometric head. Observations are VAE-encoded into the video KV cache, executed actions fill the action cache, and future action tokens are denoised and executed. A new observation is captured every four actions.
Go to primary source ↓e09PDF p. 7, Training Hyperparameters
AdamW uses learning rate 2 × 10^-5 times the square root of the machine count, ten warmup steps, gradient clipping 2.0 and bf16 parameters. Main experiments use 50,000 training steps and ablations 10,000.
Go to primary source ↓e10PDF p. 7, Section 4.1, Datasets and Tasks; p. 8, Section 4.2
RoboTwin uses a unified policy across 50 tasks with 50 clean and 500 randomized trajectories per task. AstriBot S1 uses four tasks with 100 demonstrations each and ten physical rollouts per method per task. Real-world depth supervision uses offline Depth Anything 3 pseudo-depth.
Go to primary source ↓e11PDF p. 8, Table 1; p. 9, Table 3 and its caption
Table 1 gives WAM4D clean/randomized/average success of 93.8/89.9/91.8%, versus LingBot-VA 92.9/91.6/92.3% and Fast-WAM 91.9/91.8/91.8%. Table 3 gives WAM4D 93.82/89.86%; it lists LingBot-VA randomized 91.50% and Motus clean 88.52%, differing from Table 1's 91.6% and 88.7%. Per-task baseline results are attributed to earlier reports.
Go to primary source ↓e12PDF pp. 8–9, latency and memory protocol; p. 15, Section 4.7 and Table 9
On one A800 80GB GPU, WAM4D action-only inference takes 525.43 ± 5.64 ms per chunk and 9.71 GiB peak allocated backbone memory. Fast-WAM takes 425.53 ± 6.01 ms and 11.55 GiB; LingBot-VA takes 843.57 ± 11.55 ms and 12.97 GiB. All use ten action steps; LingBot-VA additionally uses five video steps. Latency uncertainty is mean ± standard deviation.
Go to primary source ↓e13PDF p. 8, Table 2 and Figure 4; p. 10, Section 4.3 and Table 4
Table 2 reports seven sub-action columns, with WAM4D average 0.90, LingBot-VA 0.84, Fast-WAM 0.80 and π0.5 0.74. WAM4D Blocks S3 is 0.8 versus LingBot-VA 0.4. Failed sequential steps make subsequent steps unattempted and scored zero. Table 4 defines Bottle S2 placement, but Table 2 omits that column.
Go to primary source ↓e14PDF p. 10, Section 4.4; p. 11, Tables 5–6
Ablations use a fixed ten-task RoboTwin split. Table 6 supplies 100-rollout counts per task and condition, full video/geometry metrics and parameter counts. Trainable pretrained DA3 has 80.1% clean and 75.4% randomized success versus no-depth 71.7% and 69.1%. The no-depth model has 5.089B transformer parameters versus 5.690B for the middle-register variants.
Go to primary source ↓e15PDF p. 12, Table 7 and Sections 4.4.1–4.4.2
With the depth head fixed, middle registers achieve 75.2% clean success and 0.053 AbsRel, shallow 72.5% and 0.058, and bidirectional 76.6% and 0.074. Uniform and bidirectional variants use layers 6/12/18/24. The VAE-head row lists layers 27/28/29/30, whereas the prose describes a layer-26 readout. Attention interpretation mentions action-history cues despite the register mask excluding action tokens.
Go to primary source ↓e16PDF p. 13, Table 8 and Section 4.4.3; p. 11, Table 6
At the default register interface, trainable pretrained, fixed pretrained and trainable random heads achieve clean success 80.1%, 75.2% and 70.0%. Their AbsRel values are 0.049, 0.053 and 0.059. Trainable pretrained gives FVD 164.5, δ1 0.948, CD1 0.0099, F-score 0.710 and F-score-T 0.848. It is selected as the final setting.
Go to primary source ↓e17PDF p. 10, Figure 5; p. 12, Section 4.4.1; pp. 13–14, Section 4.5 and Figure 6
Register attention maps are averaged over heads at selected layers. The separate qualitative analysis retains the depth branch, generates RGB and depth autoregressively from an initial frame, and back-projects RGB-D into point clouds.
Go to primary source ↓e18PDF p. 13, Section 4.6; p. 15, Figure 7 and Section 5
The authors show identity-inconsistent object completions after occlusion and attribute them to missing explicit long-term memory. They argue fresh real observations protect their evaluated policy path. The conclusion also acknowledges slower inference than VLAs.
Go to primary source ↓e19PDF p. 8, Metrics
Video metrics are FVD, PSNR, SSIM and LPIPS. Depth is evaluated using AbsRel and threshold accuracies; predicted and reference depth are back-projected using a fixed dataset-level intrinsic matrix. CD1/CD2 measure Chamfer distance, and F-score/F-score-T describe spatial/temporal overlap. Numerical thresholds and the intrinsic matrix are not given here.
Go to primary source ↓8.5 Primary sources
WAM4D: Fast 4D World Action Model via Spatial Register Tokens ↗
PDF · 9,724 extracted words
Source fingerprint
35ab949057809b5e150f6b2c6dff3d2f559bfc61b29dfa371c07fa1894d02b6f