4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields
1. Paper overview
In one sentence: Two training-only alignment losses transfer local motion and destination correspondence from a frozen trajectory teacher into existing WAMs, improving robustness while leaving deployment architecture unchanged. e02e05e06e09e10e11e13e14e15e17
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that predicting 2D pixels or independent per-frame geometry does not adequately teach the evolving 3D structure needed for manipulation. Their diagnostic finds poorly matched absolute WAM/teacher features but more compatible temporal differences, motivating relational supervision instead of direct feature reconstruction. e02e03 |
| Core mechanism | |
| A key reported result | LIBERO-Plus robustness: 71.01 average; camera 45.15; noise 69.08. Success rate (%). FastWAM-Joint backbone; trained on LIBERO only, evaluated under seven unseen perturbation categories. FastWAM-Joint: 62.21 average, camera 27.89, noise 51.09. All seven columns improve. The average gain is 8.80 percentage points; Figure 1's +9.8 annotation conflicts with Table 2 and Section 4.2. e10e18 |
| Reading caution |
Core contributions
Figure 3. Two relational losses connect a trainable WAM to frozen trajectory features. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the video frames on the left and follow the upper WAM and lower Trace Anything branches. Fire symbols mark the trainable WAM and student projector; snowflakes mark the frozen teacher and its projector. In branch (1), projected tokens are normalized before adjacent-frame differences are compared by cosine distance, as specified in Equations 3–4. In branch (2), inspect the labels beside the dot products: both distributions use the teacher's final-frame destination tokens. The upper query comes from the WAM's first frame; the lower query comes from the teacher's first frame. Equation 7 matches the teacher distribution to the student distribution with KL divergence. e04e05e06e07e15e19
What it supports. The method transfers relationships between representations: local change directions and source-to-destination similarities. It does not ask the WAM to output the teacher's complete trajectory field. Appendix A states that the teacher and both projectors are removed after training, so these branches describe supervision rather than additional deployed control modules.
Where the evidence stops. The central banner is a schematic video loss: it omits the action objective and balancing coefficients present in Equation 8. The bidirectional-looking loss arrows do not imply teacher updates; the freeze markers and Sections 3.2/Appendix A specify a fixed teacher target.
2. Motivation
2.1 The problem and the proposed response
The authors argue that predicting 2D pixels or independent per-frame geometry does not adequately teach the evolving 3D structure needed for manipulation. Their diagnostic finds poorly matched absolute WAM/teacher features but more compatible temporal differences, motivating relational supervision instead of direct feature reconstruction. e02e03
2.2 What this reading follows
A video model can preserve appearance without learning the spatial changes needed to manipulate objects reliably. 4D-WAM asks whether a trajectory-aware teacher can supply that missing supervision without becoming part of the deployed policy. Its answer combines adjacent-frame feature differences with a distribution over where first-frame regions correspond in the final frame. Read the architecture first, then contrast the OOD results with the small standard-benchmark ablations and low physical success rates. The evidence supports useful representation transfer across two base models, but teacher-derived probes and inconsistencies in several reported summaries leave the extent of independent 4D understanding unresolved. e02e05e06e09e10e11e13e14e15e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Not applicable |
| Prediction paradigm | Joint prediction |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The snapshot fits a model-agnostic WAM training strategy: joint video/action objectives are explicit, while the added teacher and projectors disappear at inference. 'Not applicable' architecture and 'Outside quadrants' reasonably avoid assigning one fixed deployment architecture to the recipe. Joint training alone establishes no One Model conclusion. The spatial and alignment tags fit the trajectory supervision and multiview physical setup. e04e07e15e16
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Why align change before aligning features?
Consider the same visual token at two consecutive frames. The WAM representation serves video denoising and action learning; the teacher representation serves reconstruction of trajectory structure. Their absolute coordinates in feature space need not match. Figure 2 gives the authors' empirical motivation for looking at temporal differences instead. Sections 3.2–3.3 first project and normalize the features, then constrain their change directions. Reader interpretation: this relaxes the requirement that student and teacher encode everything identically, while retaining a shared temporal signal. It also sets an important boundary: a cosine loss on feature changes does not directly measure displacement in physical units. The teacher provides a trajectory-informed training target, and whether the learned features support useful control must still be tested through executed-policy evaluations. e03e04e05e07e10
5.2 Two source queries share the same destination keys
The destination objective complements the adjacent-frame loss by connecting the beginning and end of an episode. For each source token, the teacher compares its own first-frame representation with all final-frame teacher tokens. The student makes the analogous comparison using its first-frame representation but the same final-frame teacher keys. A temperature-scaled softmax converts each similarity vector into a distribution over destination tokens, and KL divergence transfers the teacher distribution to the student. This detail prevents a misleading reading of Figure 3: the upper destination branch does not use a predicted final WAM frame as its key. The final observed frame supplies a completion signal during training. Equation 8 keeps action prediction and latent denoising in place, with the new terms acting as auxiliary constraints. e06e07e19
5.3 Separate representation transfer, prediction quality, and control
The experiments test different links in the argument. Table 8 asks whether teacher-defined correspondences are easier to recover from frozen hidden states. Table 4 asks whether predicted observations better match simulator renderings. Tables 2 and 3 evaluate task success under shifts, and Table 6 tests execution on a physical robot. Reader interpretation: agreement across these measures is suggestive, but none substitutes for another. The probe is tied to the training teacher, image fidelity is not task completion, and real-world success remains low. Deployment also has a precise boundary: Appendix A removes the teacher and projectors and preserves the original WAM architecture. Consequently, any benefit at inference must come through the trained base model rather than an additional trajectory planner running beside it. e10e11e12e14e15e17
5.4 Training and inference
During training
Alignment uses DiT layer 20. FastWAM-Joint training uses batch 1024 for 30K RoboTwin steps and batch 512 for 25K LIBERO steps. Lingbot-VA Clean2Rand training uses batch 64 for 20K steps. The setup specifies eight NVIDIA H100 GPUs. e08
RoboTwin training provides 500 randomized and 50 clean demonstrations per task over 50 tasks; LIBERO provides 100 demonstrations per task over 40 tasks. Clean2Rand uses only 50 clean demonstrations per task. Both alignment weights are recommended below 0.1, but exact values are absent. e07e09e11
Teacher features are normally pre-extracted and cached. On LIBERO with eight H100 80GiB GPUs, baseline/cached/online step times are 7.3/7.5/8.4 seconds. Table 7 lists 51/52/59 under 'Training time (8 GPUs, h)', while surrounding prose calls these GPU hours; retain this unit ambiguity. The added trainable parameter storage is reported as 56 MiB. e15
During inference
After training, remove Trace Anything, the alignment projectors, and their losses. The authors state that the base WAM architecture and parameter count are unchanged, with no additional inference latency or memory. The recipe adds no destination-search or trajectory-control module at execution time. e15
The paper leaves detailed base-model decoding, action horizon, and feedback cadence to the underlying systems. Auxiliary teacher correspondence prediction should therefore not be interpreted as an independently specified inference-time controller. e07e15
5.5 Implementation flow
- Project corresponding tokens
For each frame, extract DiT features H and teacher features Z. Separate two-layer MLPs map their differing dimensions into a shared space; normalize each projected token before comparison. The teacher projector is frozen, as is Trace Anything. The source assumes matching frame/spatial tokens but does not fully specify their implementation-level registration. e04e05e15
- Align local changes
Subtract normalized features at adjacent frames at the same token index. Minimize cosine distance between student and teacher difference vectors. This supervises directions of feature change, rather than requiring identical absolute representations or explicitly regressing metric 3D displacement. e05
- Align destinations
Use first-frame tokens as queries and final-frame teacher tokens as keys. Both student and teacher distributions use teacher destination keys; only their source queries differ. Temperature-scaled softmax creates destination distributions, and teacher-to-student KL divergence transfers the correspondence structure. e06
- Retain the base learning problem
Add both alignment terms to the original video-latent denoising and action-prediction losses. Figure 3 illustrates the video-side terms schematically; Equation 8 additionally includes action loss and balancing coefficients. e07e19
6. Experiments & results
4D-WAM trains existing video–action models with trajectory-informed auxiliary losses. It aligns adjacent-frame feature changes and first-to-final-frame correspondence distributions with a frozen Trace Anything teacher, then removes the teacher and projectors for deployment. The strongest evidence is improved simulated robustness under distribution shifts; standard benchmark gains are small, and real-world completion remains low. Several reporting inconsistencies limit precise interpretation.
6.1 Read the original evidence
Table 2. Robustness gains span all seven reported perturbation categories. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read vertically within each perturbation column before examining the average at the right. Both rows were trained on LIBERO only; the camera, robot, language, lighting, background, noise, and layout variations belong to LIBERO-Plus evaluation. The highlighted row is the alignment-trained FastWAM-Joint model, not the Lingbot-VA integration used for Clean2Rand. Camera success moves from 27.89 to 45.15, while noise success moves from 51.09 to 69.08. These are percentages of successful manipulation, so subtracting them yields percentage-point gains. Section 4.2 supplies the training/evaluation distinction that the compact table itself does not show. e09e10e18
What it supports. The reported average increases from 62.21% to 71.01%, an 8.80-point gain, and every listed perturbation improves. Compared with the 0.4-point LIBERO average gain in Table 1, this is the clearest evidence that the recipe helps under the particular distribution shifts evaluated here.
Where the evidence stops. Figure 1(b) labels these same average endpoints as +9.8; Table 2 and Section 4.2 support +8.8. No confidence intervals or seed variation are supplied. This table supports robustness to its listed perturbations, without establishing arbitrary real-world generalization.
Table 6. Physical-task completion improves overall but remains rare. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each task has two columns: Progress measures the proportion of subtasks completed, while SR measures full-task success. Section 4.5 evaluates each model for 50 trials per task after joint training on 550 demonstrations. The four tasks exercise size-matched cylinder insertion, deformable towel manipulation, wrench retrieval from a toolbox, and grasping a red light from a rotating turntable. The backbone here is Lingbot-VA. Read the task pairs before the printed averages: cylinder progress rises markedly, but its full success rate is only 2.0%; rotating-light SR is unchanged at 4.0 while progress decreases. Figure 5 and Appendix B document the physical setup. e14e16
What it supports. Average full-task success increases from 1.0% to 5.5%. The per-task success rates show improvement on cylinders, towel, and wrench retrieval, with no increase on rotating-light manipulation. This is evidence of some physical transfer, yet most evaluated long-horizon trials still fail to complete.
Where the evidence stops. The printed Progress averages, 20.3 and 31.8, are not arithmetic means of their displayed task columns; the aggregation rule is unspecified. Rotating-light progress falls from 25.0 to 23.8, so Section 4.5's statement that every task improves is too broad.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Plus robustness FastWAM-Joint backbone; trained on LIBERO only, evaluated under seven unseen perturbation categories. | 71.01 average; camera 45.15; noise 69.08. Success rate (%) | FastWAM-Joint: 62.21 average, camera 27.89, noise 51.09. All seven columns improve. The average gain is 8.80 percentage points; Figure 1's +9.8 annotation conflicts with Table 2 and Section 4.2. e10e18 |
| In-distribution manipulation FastWAM-Joint backbone; RoboTwin clean/random and the four standard LIBERO suites. | RoboTwin clean/random/average: 91.92/90.76/91.34; LIBERO average: 98.6. Success rate (%) | Base: 90.84/90.32/90.58; LIBERO average: 98.2. Gains are modest; LIBERO Spatial decreases from 99.6 to 98.8. Table 1 reports Object 99.5, whereas Table 5's full configuration reports 99.6. e09e13 |
| RoboTwin Clean2Rand transfer Lingbot-VA backbone; clean-only demonstrations, evaluated on clean and randomized splits. | Clean 81.5; randomized 41.8; average 61.7. Success rate (%) | Lingbot-VA: 80.7/34.6/57.7; pi0.5 randomized: 46.0. Randomized success improves by 7.2 points over the base but remains below pi0.5; the best two-split average does not imply the best randomized score. e11 |
| OOD future-observation prediction LIBERO-Plus; 100 episodes per perturbation type; policy execution in simulation, comparing predictions with simulator-rendered observations. | 21.44 PSNR; 0.83 SSIM. PSNR and SSIM (higher is better) | FastWAM-Joint: 19.22 and 0.75. Supports better rendered-observation prediction under this protocol; these metrics do not themselves measure executed task completion. e12 |
| Alignment objective and layer ablation FastWAM-based LIBERO evaluation following Table 1. | Layer 20 with both objectives: 98.6; without destination or without motion: 98.0 each. Average success rate (%) | Layers 16/18/22/24: 97.9/97.4/98.5/97.8. Supports the combined setting among tested variants. Either single loss is below the 98.2 base average; uncertainty is unreported. The prose claim that later layers consistently exceed 98.5 is contradicted by the table. e09e13 |
| Four real-world long-horizon tasks ARX LIFT2; Lingbot-VA backbone; jointly trained on 550 demonstrations; 50 trials per task. | Average SR 5.5; per-task SR: cylinders 2.0, towel 4.0, wrench 12.0, rotating light 4.0. Success rate (%) and reported progress | Base average SR 1.0; per-task SR 0.0/0.0/0.0/4.0. Completion remains rare. Rotating-light progress falls from 25.0 to 23.8, contradicting improvement on every task. Printed average progress 20.3/31.8 does not equal the arithmetic mean of displayed task values; aggregation is unresolved. e14e16 |
| Frozen-representation trajectory probing Frozen layer-20 features; correspondence targets from Trace Anything; LIBERO and LIBERO-Plus. | LIBERO: 0.4487 and 4.07; LIBERO-Plus: 0.2625 and 10.91. Destination Top-1 accuracy (fraction) and mean rank | Post-trained FastWAM-Joint: 0.4150/4.66 and 0.2487/11.38. Teacher-target correspondence is more accessible. Table 8 calls the probe linear, but Appendix C specifies lightweight MLPs; this ambiguity prevents a firm linear-decodability claim. e17 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 2. Temporal differences offer a more compatible alignment signal in the illustrated tasks. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel relative to its own horizontal zero line. The five task groups are move stapler pad, put bottles dustbin, hanging mug, turn switch, and open microwave. Panel (a) compares pi0.5 with VGGT-derived 3D features; panel (b) compares Lingbot-VA hidden states with Trace Anything 4D features. Panel (c) instead compares the temporal differences of the WAM and teacher representations. The horizontal axis traverses hidden layers within task groups. Pink areas lie above zero and blue areas below it. This distinction matters: the bottom panel has more positive regions but also retains negative intervals, especially for the microwave task. e03e05
What it supports. Section 3.1 uses this contrast to motivate comparing how representations change across frames. The plotted examples make temporal alignment plausible where absolute hidden states disagree. They support the choice of a supervision signal, rather than establishing a quantitative relationship between similarity and downstream success for every task.
Where the evidence stops. Only zero is numerically labeled on the vertical axes, so precise similarity magnitudes cannot be recovered from this crop. Panel (a) also changes both model and teacher relative to panel (b); it is not a controlled comparison of alignment objectives.
Table 5. The combined layer-20 configuration leads the tested variants by a small margin. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The upper block changes the DiT layer where alignment is applied, while the lower block removes one objective at a time. Read the four LIBERO suite columns separately from the final average: different settings can lead on individual suites without leading overall. The highlighted layer-20 row is the full configuration, with a 98.6 average. Both 'w/o Destination' and 'w/o Motion' reach 98.0. To compare with no alignment, consult Table 1's FastWAM-Joint row, which reports 98.2. Section 4.4 states that the ablations follow the Table 1 evaluation setting; the crop itself contains no no-alignment row. e09e13
What it supports. Using both losses at layer 20 performs best among the printed configurations, while either loss alone falls below the base model's reported average. This pattern is compatible with complementary objectives, but the table does not resolve whether the difference is stable across training seeds or explained by loss-weight interactions.
Where the evidence stops. The prose says alignment at or beyond layer 20 consistently exceeds 98.5; layer 24 actually reports 97.8, and layer 22 is 98.5. Table 5 also gives Object 99.6 for the full model, versus 99.5 in Table 1. These discrepancies remain unresolved.
Table 8. Aligned features expose more of the teacher's trajectory correspondence information. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the four blocks as two correspondence tasks on two datasets. Frame-to-frame prediction locates a token in the next latent frame; destination prediction locates a first-frame token in the final latent frame. Appendix C freezes the WAM and extracts layer-20 features for the probes. The asterisk denotes the original FastWAM-Joint checkpoint without post-training, not a randomly initialized model. Lower KL, cross-entropy, and rank are better; higher Top-1, Top-5, and Top-10 are better. Top-k entries are fractions. The targets are derived from Trace Anything, so all columns measure agreement with the teacher's correspondence target rather than independently measured robot trajectories. e17
What it supports. 4D-WAM leads the printed rows throughout this table. For LIBERO destination prediction, Top-1 rises from 0.4150 to 0.4487 and mean rank falls from 4.66 to 4.07 relative to post-trained FastWAM-Joint. The LIBERO-Plus destination improvement is smaller, with Top-1 moving from 0.2487 to 0.2625.
Where the evidence stops. The caption calls the probe linear, while Appendix C describes two lightweight MLP probes. That conflict leaves probe capacity unresolved. Because the same teacher supplies training supervision and probe targets, these results alone cannot establish independently accurate geometric reasoning.
7. Analysis & limitations
7.1 What the evidence leaves open
No seed variation, confidence intervals, or statistical tests accompany the main comparisons. The small saturated-benchmark gains and physical-trial success rates should be interpreted accordingly. e09e10e11e13e14
Teacher-derived probing targets resemble the training supervision. Probe gains support transfer of that representation but do not independently validate metric 3D tracking, causal dynamics, or robustness to teacher errors. e05e06e17
Figure 2's temporal similarities remain task- and layer-dependent and include negative regions. The diagnostic motivates temporal alignment without proving that all absolute-feature alignment strategies must fail. e03
7.2 Questions for discussion
- Do the two losses provide complementary information or primarily compensate for each other's optimization effects?
- Would gains persist with independently measured correspondences rather than the same teacher used for supervision?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful implementation needs the specified base checkpoints, frozen Trace Anything features, layer-20 extraction, token registration, projection initialization/dimensions, temperature, exact loss weights, and benchmark configurations. Sections 3–4 leave projector initialization, numerical dimensions, temperature, precise weights, optimizer settings, and software versions unspecified. e04e05e06e07e08
Physical reproduction needs the bimanual ARX LIFT2, two 6-DoF arms with 1-DoF grippers, three RGB cameras, and 100/150/200/100 demonstrations for cylinders/towel/wrench/light. Task progress aggregation needs clarification before comparing the printed average. e14e16
Reader-proposed checks: compare both losses against each single loss and temporally shuffled teacher features under matched training budgets; separately compare linear and MLP probes against independent correspondence labels. Neither experiment was run here. e05e06e13e17
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether temporal correspondence explains the joint-loss gain
Reader-proposed experiment, not performed: start from the same FastWAM-Joint checkpoint and hold the LIBERO training data, layer 20, batch size, step budget, token mapping, and random seeds fixed. Compare no alignment, motion only, destination only, both losses, and both losses with teacher frame order shuffled within each episode before forming targets. Resolve the paper's unspecified projection and loss hyperparameters first, then report the chosen values. Evaluate standard LIBERO and the same seven LIBERO-Plus categories, reporting seed variation and task-level outcomes. The mechanism predicts that properly matched joint supervision should outperform the shuffled control, especially under shifts. If shuffling preserves the gains, or a single loss matches the joint model within uncertainty, the temporal-correspondence or complementarity explanation would weaken. e04e05e06e07e08e09e10e13
Check 2: Disentangle probe capacity from teacher imitation
Reader-proposed experiment, not performed: freeze layer-20 representations from the original checkpoint, post-trained FastWAM-Joint, and 4D-WAM. Train separately specified linear and matched-capacity MLP probes on identical episode splits for next-frame and first-to-final correspondence, resolving the caption/body discrepancy by reporting both probe families. Use held-out episodes and evaluate both the paper's Trace Anything targets and a separately validated set of visible-point correspondences obtained independently of that teacher. Keep token candidates and metric definitions fixed; report Top-1, Top-5, Top-10, mean rank, and target-distribution metrics where distributions are available. If 4D-WAM's advantage survives probe choice and independent labels, that supports transferable trajectory information. If it appears only for teacher targets or one probe family, the stronger geometric interpretation is not supported. e04e05e06e17
8.3 Reading coverage
Visual audit: All eleven original PDF pages were rendered and visually inspected, including the title/author/version block, Figures 1–7, Tables 1–8, Equations 1–9, training setup, and Appendices A–C. All six final crops were separately viewed at their native generated resolutions; labels, table headers, arrows, freeze markers, and complete plotted regions are retained. Figure 3 was cross-checked against Equations 3–8 and Appendix A. Supporting uncropped pages include page 5 for base results and training, page 6 for Clean2Rand/video prediction, and page 8 for hardware, caching, and physical data. Pages 10–11 cover the end of Appendix C and references; cited outside works were not separately read. Source inconsistencies in Figure 1, Tables 1/5/6/7, and the Table 8 probe description are disclosed. Separate supplemental material was not supplied or verified.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work, including 2.1 and 2.2
- 3 Methodology, including 3.1–3.4
- 4 Experiments, including 4.1–4.5
- 5 Conclusion
- A Auxiliary Training Cost
- B Real-world Task and Dataset
- C Probing Experiments: Mechanism of 4D Awareness
- References
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All five supplied text chunks and all eleven PDF pages were read; original figure images and table layouts were also visually inspected. No separate supplement was supplied.
- Identity/version scope: the inspected title page identifies arXiv:2608.08023v2 [cs.RO], 12 Aug 2026. Its title and all 17 authors match the catalog after name-order normalization. No earlier edition or revision history was supplied, so changes from v1 cannot be established.
- The linked code was not inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author/affiliation block and arXiv margin
The title matches the catalog; the 17 listed authors run from Lishan Yang to Haoang Li. Six affiliations are printed. The margin identifies arXiv:2608.08023v2 [cs.RO], 12 Aug 2026.
Go to primary source ↓e02PDF pp. 1–2, Abstract, Section 1 and Figure 1(a)
The paper motivates trajectory-field representation alignment as a training recipe for the gap between 2D video modeling and evolving 3D manipulation structure.
Go to primary source ↓e03PDF p. 3, Section 3.1 and Figure 2(a–c)
Five RoboTwin tasks compare VLA/3D feature similarities, Lingbot-VA/Trace Anything absolute similarities, and their temporal differences. Temporal curves are more positive in many regions but include negative intervals.
Go to primary source ↓e04PDF pp. 3–4, Section 3.2, Equations 1–3
Corresponding frame/spatial tokens from H and Z are mapped through separate two-layer MLPs into shared dimension d and normalized. P_z is frozen; exact projection sizes and initialization are not supplied.
Go to primary source ↓e05PDF p. 4, Section 3.2, Equations 3–4 and Figure 3 branch (1)
Motion loss averages cosine distance between adjacent-frame differences of normalized student and teacher tokens over episodes, frame pairs, and tokens; epsilon stabilizes the denominator.
Go to primary source ↓e06PDF pp. 4–5, Section 3.3, Equations 5–7 and Figure 3 branch (2)
Student source queries and teacher source queries share teacher destination keys. Temperature-scaled similarities yield distributions matched through D_KL(q_z || p_h); first/final frames are used in the implementation.
Go to primary source ↓e07PDF p. 5, Section 3.4, Equation 8 and coefficient paragraph
The full loss combines latent denoising, action prediction, motion alignment, and destination alignment. Both balancing coefficients are recommended below 0.1; exact experimental values are not stated.
Go to primary source ↓e08PDF p. 5, Section 4, Experimental Setup
Alignment is at layer 20 using eight H100 GPUs. FastWAM-Joint uses batch 1024/30K steps on RoboTwin and batch 512/25K on LIBERO; Lingbot-VA Clean2Rand uses batch 64/20K steps.
Go to primary source ↓e09PDF p. 5, Section 4.1 and Table 1, FastWAM-Joint and 4D-WAM rows
Training data cover 50 RoboTwin tasks with 500 randomized and 50 clean demonstrations per task, and 40 LIBERO tasks with 100 each. Table 1 reports RoboTwin averages 90.58/91.34 and LIBERO averages 98.2/98.6; Spatial declines 99.6 to 98.8.
Go to primary source ↓e10PDF p. 6, Table 2, all perturbation columns; Section 4.2
LIBERO-trained FastWAM-Joint/4D-WAM score 62.21/71.01 average on LIBERO-Plus. Camera scores are 27.89/45.15 and noise scores 51.09/69.08. All seven categories improve.
Go to primary source ↓e11PDF pp. 5–6, Experimental Setup, Section 4.2 and Table 3, Clean/Rand./Average rows
Clean2Rand uses 50 clean demonstrations per task. Lingbot-VA/4D-WAM score 80.7/81.5 clean, 34.6/41.8 randomized, and 57.7/61.7 average. pi0.5 scores 46.0 randomized.
Go to primary source ↓e12PDF pp. 6–7, Section 4.3; p. 6, Table 4 and Figure 4
Policies execute in simulation and predictions are compared with rendered observations. Table 4 specifies 100 episodes per perturbation type and reports PSNR 19.22/21.44 and SSIM 0.75/0.83. Figure 4 shows three selected four-frame comparisons.
Go to primary source ↓e13PDF p. 7, Table 5 and Section 4.4; p. 5, Table 1 Object column
Layer-20 full average is 98.6 versus 98.0 for either removed objective; layer 24 is 97.8 despite the prose's later-layer generalization. Full Object is 99.6 in Table 5 but 99.5 in Table 1.
Go to primary source ↓e14PDF p. 7, Table 6, all task/average columns; Section 4.5 and Figure 5
Each real task has 50 trials. Success averages are 1.0/5.5. Listed progress values are 28.1/2.5/6.7/25.0 for Lingbot-VA and 50.6/7.5/8.3/23.8 for 4D-WAM, while printed progress averages are 20.3/31.8. Progress is defined as proportion of subtasks completed.
Go to primary source ↓e15PDF p. 8, Appendix A, Equation 9 and Table 7
Frozen teacher features can be cached or extracted online. Eight H100 80GiB GPUs train LIBERO for 25K steps at batch 512. Table 7 reports 7.3/7.5/8.4 seconds per step, 77.3/82.7/96.0 peak-memory percentages, 51/52/59 training-time entries, and 56 MiB extra trainable storage. Teacher/projectors are removed at inference; training-time unit wording is inconsistent.
Go to primary source ↓e16PDF pp. 7–9, Section 4.5, Appendix B and Figures 5–7
The ARX LIFT2 has two 6-DoF arms with 1-DoF grippers and three RGB cameras. The four tasks use 100/150/200/100 demonstrations for cylinders/towel/toolbox wrench/rotating red light, totaling 550.
Go to primary source ↓e17PDF pp. 9–10, Appendix C; p. 9, Table 8 and caption
Frozen layer-20 features predict next-frame and final-frame correspondences using Trace Anything targets. Table 8 calls the probe linear; the body specifies two lightweight MLPs. The asterisk denotes the original checkpoint without post-training. Destination Top-1/rank are 0.4150/4.66 versus 0.4487/4.07 on LIBERO and 0.2487/11.38 versus 0.2625/10.91 on LIBERO-Plus.
Go to primary source ↓e18PDF p. 2, Figure 1(b), LIBERO-Plus bars; p. 6, Table 2 and Section 4.2
Figure 1 prints 62.21 and 71.01 with a +9.8 annotation. Table 2 repeats these scores, and Section 4.2 describes an 8.8-point gain.
Go to primary source ↓e19PDF p. 4, Figure 3 loss banner and frozen/trainable markers; p. 5, Equation 8
Fire symbols mark WAM/student projector training and snowflakes mark the teacher side. The figure's video-loss banner omits coefficient weights and action loss, which appear in the complete Equation 8.
Go to primary source ↓8.5 Primary sources
4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields ↗
PDF · 6,099 extracted words
Source fingerprint
ae919cd5694bc0b07c847aa8057ff54f2c0f313f269529f7f65964396bf90584