Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation
1. Paper overview
In one sentence: Joint RGB-D and action denoising improves offline navigation planning, while endpoint regularization and visual refinement bring distinct benefits and unresolved execution limits. e-identitye-framinge-architecturee-denoisee-trajectoriese-ablatione-protocole-implementatione-long
| At a glance | What to know |
|---|---|
| Research problem | Author claim Visual goal navigation must connect observed scenes with feasible motion. The authors argue that proposing actions externally and then ranking imagined videos makes planning depend on candidate coverage, repeated rollouts, and imperfect action–image consistency. SWAM conditions trajectory synthesis itself on the goal. e-framing |
| Core mechanism | Source description One latent diffusion transformer co-generates RGB, depth, and actions. VGAR uses generated visual tokens to refine action readout; TSR supervises cumulative displacement during training. e-architecturee-vgare-objective |
| A key reported result | Offline trajectory prediction on RECON, SCAND, and TartanDrive: RECON 0.93 / 0.43; SCAND 1.15 / 0.34; TartanDrive 1.55 / 0.68 ATE / RPE, unit grids; lower is better. NWM-standardized evaluation subsets; pose-derived normalized actions; fixed nine-frame model; three-run averages. Exact subset sizes and uncertainty are not reported. NWM+NoMaD ×16: 1.53 / 0.49, 2.18 / 0.46, 6.23 / 1.30 respectively. CogVideoX (Joint): 2.09 / 0.73, 2.25 / 0.67, 4.90 / 1.07. SWAM improves both trajectory metrics on all three datasets in Table 1. These are dataset predictions, not executed navigation trials. e-protocole-trajectoriese-implementation |
| Reading caution | Author claim The authors identify traversable weeds mistaken for obstacles and sharp-turn failures associated with displacement-only actions lacking explicit orientation dynamics. e-failures |
Core contributions
- Source description
One latent diffusion transformer co-generates RGB, depth, and actions. VGAR uses generated visual tokens to refine action readout; TSR supervises cumulative displacement during training. e-architecturee-vgare-objective
- Reader analysis
A matched CogVideoX joint-generation baseline helps separate navigation-specific additions from video pretraining, although the full model's gains cannot all be attributed to one component. e-protocole-ablation
Figure 2. A shared denoising backbone links predicted views to predicted motion. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read left to right. The two endpoint image/depth pairs become fixed conditioning latents, while outlined RGB, depth, and action boxes denote noisy targets. The central DiT produces all three target modalities. On the right, visual latents feed the VAE decoder and the refinement branch; action tokens also connect to action decoding. Equations (2)–(4) specify the refinement more precisely than the schematic: actions supply queries, visual tokens supply keys and values, and a gated residual updates actions. The dashed connection to TSR denotes a training loss on integrated displacement, not an additional online controller. The figure's single DiT block summarizes an iterative diffusion process. e-architecturee-vgare-objectivee-denoisee-framing
What it supports. The architecture supports the core One Model × Joint prediction classification because visual and action targets share a denoising sequence and backbone. VGAR adds a final route from predicted visual evidence to action readout; it does not first generate a video and then invoke a separately trained inverse-dynamics policy.
Where the evidence stops. The abstract states monocular RGB test input, but Figure 2 shows DA3 endpoint depths and Equation (1) retains their latents. The exact inference-time depth construction is not separately documented. The schematic also omits the explicit gate shown in the equations.
2. Motivation
2.1 The problem and the proposed response
Visual goal navigation must connect observed scenes with feasible motion. The authors argue that proposing actions externally and then ranking imagined videos makes planning depend on candidate coverage, repeated rollouts, and imperfect action–image consistency. SWAM conditions trajectory synthesis itself on the goal. e-framing
2.2 What this reading follows
A goal image says where a robot should arrive, but a planner must still connect that destination to plausible motion and changing views. SWAM learns to generate these together: a shared diffusion transformer predicts a visual path and planar displacements, then visual tokens refine the action readout. Its strongest evidence is lower offline trajectory error than candidate-ranking world-model planners, with less serial inference time. Read the figures alongside their boundaries: diffusion remains iterative, the ablation gains are mixed, and the reported success rates concern dataset trajectories. The supplied arXiv v1 includes a supplement whose longer-sequence model is separate from the main quantitative evaluation. e-identitye-framinge-architecturee-denoisee-trajectoriese-ablatione-protocole-implementatione-long
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The core Q1 classification is supported architecturally: one DiT jointly denoises future visual and action targets, with a refinement head rather than a separate inverse-dynamics policy. Frozen encoding and a depth estimator do not make the predictive system two independent models. The secondary '3D multiview modeling' label is broader than the demonstrated monocular RGB-D representation; explicit multiview reconstruction is not established. e-architecturee-denoisee-vgar
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate what conditions the plan from what the model must invent
SWAM starts with endpoint observations rather than an externally proposed action sequence. Section 3.2 separates their RGB/depth latents from the noisy targets: intermediate visual latents and action tokens are generated together. This matters when reading the architecture, because a goal-conditioning arrow is not evidence that the intermediate route is known. The frozen VAE provides the visual latent representation, while a learned MLP puts two-dimensional local displacements into token form. One DiT then iteratively predicts noise for the combined target sequence. After denoising, visual latents are decoded and actions are refined before readout. The abstract promises monocular RGB input, but the formulation still includes endpoint depth latents; their exact inference construction remains an implementation question. Nothing in this information flow supplies an explicit feedback controller for physical execution. e-architecturee-denoisee-vgare-framinge-long
5.2 Understand why endpoint regularization and visual refinement solve different problems
TSR sums denoised displacement predictions and compares the resulting endpoint with the local goal position. Its unsquared Euclidean penalty, divided by the action horizon, supplements the diffusion loss during training. VGAR acts elsewhere: immediately before decoding, action queries attend to generated visual tokens, and a sigmoid gate controls an additive update. Reader analysis: these mechanisms address different failure opportunities. Small local errors can accumulate into endpoint drift, while an action readout may fail to exploit visual information already present in the backbone. The ablation supports useful contributions without proving that either remedy is universally beneficial. Adding VGAR to depth plus TSR improves TartanDrive ATE while worsening its RPE, and also worsens SCAND trajectory errors. TSR is not an inference-time goal optimizer, and VGAR's generated evidence is not a new observation from the environment. e-objectivee-vgare-ablation
5.3 Keep the evidence for accuracy, transfer, and execution separate
The main experiment compares predicted trajectories and frames on standardized dataset subsets, using a fixed nine-frame model. Table 1 supports lower trajectory errors; Figure 3 adds an endpoint-threshold view; Table 2 measures RGB similarity. None substitutes for the others. HuRoN transfer is encouraging: SWAM reports 2.94 ATE and 0.85 RPE without HuRoN training, against 3.73 and 0.95 for a comparator trained there. That is a different training-exposure comparison, not matched zero-shot evaluation. The supplement's 64-step predictions come from a separately trained variable-length model, and the authors caution that more frames need not mean farther travel. Reader analysis: the evidence is strongest for offline high-level planning. Claims about robust closed-loop operation would require an execution protocol, feedback handling, and measured outcomes beyond these predictions and selected visual examples. e-protocole-trajectoriese-successe-videoe-transfere-implementatione-longe-qualitative
5.4 Training and inference
During training
DDPM noise prediction trains the joint latent targets. TSR adds normalized endpoint error. The authors describe zero-weight initialization of newly added modules to preserve pretrained behavior initially; the VAE remains frozen. e-objectivee-inite-architecture
Main comparisons use eight observation–action pairs, nine frames total. A separate visualization model samples 9/17/33/65-frame segments with RoPE interpolation; it is excluded from main quantitative comparisons. The supplement reports bf16, gradient checkpointing, batch size 1, Adam learning rate 10^-4, 1,000 warm-up steps, gradient clipping 1.0, and epsilon 10^-15. e-implementation
During inference
The source's 'single-pass' wording means one joint generative planning process without candidate expansion: Equation (1) still iterates over T diffusion steps. Generated trajectories are high-level plans; a feedback controller, replanning policy, and physical execution evaluation are not provided. e-framinge-denoisee-protocole-long
Monocular RGB is the stated external test input. Figure 2 nevertheless shows DepthAnything V3 at the endpoints, and the equations condition on endpoint depth latents. Their precise inference-time construction is not separately specified; RGB-only sensing should not be read as proof of depth-free internal computation. e-framinge-architecturee-denoise
5.5 Implementation flow
- Encode paired modalities
The frozen CogVideoX 3D VAE separately encodes RGB and estimated depth. A learned MLP embeds each local-frame action, a_n = (Δx_n, Δy_n), into action tokens. e-architecture
- Denoise a shared sequence
Intermediate RGB, depth, and action tokens form the noisy target sequence. Start/goal latents remain conditions. The DiT jointly predicts noise residuals over iterative denoising steps, allowing visual and motion predictions to interact. e-architecturee-denoise
- Refine and decode
VGAR takes action tokens as cross-attention queries and final visual tokens as keys and values. A learned sigmoid gate scales a residual update before the action head; the VAE decodes visual latents. e-vgare-denoise
6. Experiments & results
SWAM turns start and goal images into a jointly generated RGB-D path and planar action sequence. A shared diffusion transformer, visual-guided action refinement, and endpoint regularization improve offline trajectory accuracy over candidate-ranking planners. Its evidence supports high-level planning, with mixed component gains and unresolved implementation details; it does not establish closed-loop robot execution.
6.1 Read the original evidence
Table 1. SWAM reduces trajectory errors while avoiding the cost of many candidate rollouts. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each dataset contributes an ATE/RPE pair; both measure trajectory error and are reported in unit grids under the NWM-compatible protocol. Lower is better. The final column measures seconds per episode without parallelization. Read the four NWM rows as increasing candidate budgets, not larger models. Then compare the final two rows: CogVideoX (Joint) shares the pretrained backbone and training protocol, but lacks depth, TSR, and VGAR. Bold marks the best value and underlining the second best. Keep the fast GNM and NoMaD rows in view when interpreting efficiency: SWAM improves over the rollout planners but does not minimize latency across all methods. e-trajectoriese-protocole-denoisee-ablatione-implementation
What it supports. On TartanDrive, SWAM reports 1.55 ATE and 0.68 RPE, compared with 6.23 and 1.30 for NWM+NoMaD ×16. Serial time is 16.91 versus 245.98 seconds. SWAM also improves both error metrics on RECON and SCAND, providing broader support than a single favorable dataset.
Where the evidence stops. These are offline predictions averaged over three runs, without uncertainty or identified timing hardware. Table 4 gives slightly different RECON values. The latency result does not establish real-time execution or a matched hardware-and-sampling-budget speedup.
Figure 3. Threshold sweeps reveal endpoint precision beyond average trajectory error. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The horizontal axis relaxes the allowed goal distance from 0.25 to 1.0 unit grids; the vertical axis is success rate in percent. Compare methods vertically at the same threshold. The red dashed stars indicate SWAM, blue circles NoMaD, and the remaining markers NWM with different candidate budgets. A high score at the rightmost threshold permits more endpoint error than a high score at the leftmost threshold. Notice also that increasing NWM's candidate budget does not give a uniformly ordered set of curves. The source does not provide a table of exact marker values, so use this plot primarily to assess the threshold-dependent pattern. e-successe-protocole-trajectories
What it supports. SWAM remains above every plotted comparator at each threshold. The advantage therefore concerns both coarse goal matching and stricter endpoint precision, complementing Table 1's average trajectory errors. It remains an offline goal-distance evaluation, rather than a measured rate of successful robot deployments.
Where the evidence stops. Section 4.2 states a 2.1× gain over NWM ×16 at 0.25, but the plotted marker heights do not corroborate that ratio. The crop is preserved faithfully; exact percentages and a corrected ratio remain unresolved.
Table 2. Visual fidelity mostly improves, with a clear perceptual-metric exception. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the arrows before comparing values: higher PSNR and SSIM are preferred, whereas lower LPIPS is preferred. Each dataset repeats this three-column group. The metrics compare generated RGB frames with reference RGB observations under the paper's evaluation protocol; they do not directly measure the quality of the generated depth. Start with the CogVideoX (Joint) row to distinguish SWAM's additions from the already strong video backbone, then inspect the NWM rows. Bold and underlined entries indicate the best and second-best scores. The last LPIPS column is particularly useful because it prevents the general improvement in the other columns from becoming an unsupported claim of universal superiority. e-videoe-protocole-depthe-ablation
What it supports. SWAM improves RECON PSNR/SSIM/LPIPS to 17.31/0.653/0.328 versus 16.65/0.633/0.366 for the joint baseline. On TartanDrive its PSNR and SSIM also improve, but LPIPS worsens from 0.330 to 0.335. Under the reported evaluation protocol, the table supports broad visual improvements with a specific exception.
Where the evidence stops. RGB similarity cannot certify collision-free motion or metric depth accuracy. Supplementary depth LPIPS uses pseudo-depth references. Full-model TartanDrive PSNR is 18.11 here but 18.15 in Table 4, an unexplained source difference.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Offline trajectory prediction on RECON, SCAND, and TartanDrive NWM-standardized evaluation subsets; pose-derived normalized actions; fixed nine-frame model; three-run averages. Exact subset sizes and uncertainty are not reported. | RECON 0.93 / 0.43; SCAND 1.15 / 0.34; TartanDrive 1.55 / 0.68 ATE / RPE, unit grids; lower is better | NWM+NoMaD ×16: 1.53 / 0.49, 2.18 / 0.46, 6.23 / 1.30 respectively. CogVideoX (Joint): 2.09 / 0.73, 2.25 / 0.67, 4.90 / 1.07. SWAM improves both trajectory metrics on all three datasets in Table 1. These are dataset predictions, not executed navigation trials. e-protocole-trajectoriese-implementation |
| Serial inference cost per planning episode Table 1; no parallelization; hardware and denoising-step budget unspecified | 16.91 Seconds; lower is better | NWM+NoMaD ×16: 245.98; ×2: 31.37; CogVideoX (Joint): 14.12; NoMaD: 0.21; GNM: 0.12 Faster than candidate-ranking world-model baselines, but slower than the joint backbone baseline and direct policies. This does not establish real-time control. e-protocole-trajectoriese-denoisee-implementation |
| Goal-reaching threshold evaluation on RECON Figure 3; thresholds 0.25, 0.5, and 1.0 unit grids | SWAM's plotted curve is highest at all three thresholds; exact rates are not tabulated Success rate; higher is better | NoMaD and NWM candidate budgets ×2/×4/×8/×16 Supports better offline endpoint precision. The prose's 2.1× claim at 0.25 does not match the apparent plotted ratio; it is not adopted as a verified quantitative result. e-protocole-success |
| RGB path generation on RECON, SCAND, and TartanDrive Predicted versus ground-truth RGB frames under the main evaluation protocol | 17.31 / 0.653 / 0.328; 16.11 / 0.619 / 0.325; 18.11 / 0.532 / 0.335, respectively PSNR / SSIM / LPIPS; higher / higher / lower is better | CogVideoX (Joint): 16.65 / 0.633 / 0.366; 15.04 / 0.601 / 0.351; 17.20 / 0.524 / 0.330 PSNR and SSIM improve across datasets, but TartanDrive LPIPS is worse than the joint baseline. Image similarity does not certify traversability. e-protocole-video |
| Zero-shot trajectory prediction on HuRoN SWAM receives no HuRoN training or fine-tuning; comparator explicitly trained on HuRoN | 2.94 / 0.85 ATE / RPE, unit grids; lower is better | NWM+NoMaD ×16: 3.73 / 0.95 Encouraging transfer under different training exposure, not a matched zero-shot comparison between both methods. e-protocole-transfer |
| Component ablation on TartanDrive Shared CogVideoX baseline; Table 4 component configurations | Baseline 4.90 / 1.07; depth 4.61 / 0.97; TSR 2.63 / 0.94; depth+TSR 1.94 / 0.53; full 1.55 / 0.68 ATE / RPE, unit grids; lower is better | VGAR improves ATE over depth+TSR while worsening RPE; SCAND also worsens from 1.12 / 0.30 to 1.15 / 0.34. Component benefits depend on the dataset and metric; the table does not support uniformly better refinement. e-ablatione-protocol |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. The component rows show complementary gains and metric-dependent tradeoffs. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the checkmarks. The first row is the joint CogVideoX baseline; subsequent rows enable depth alone, TSR alone, depth plus TSR, and all three additions. Hold a dataset column fixed while moving down the rows. Lower ATE/RPE and higher PSNR are preferred, with bold and underlining marking the best and second-best entries. Comparing the final two rows isolates adding VGAR on top of depth and TSR within the reported design. It does not establish VGAR's effect without those components, because such a row is absent. Read individual metrics instead of treating the final row as automatically strongest: several best values occur above it. e-ablatione-protocole-trajectories
What it supports. On TartanDrive, TSR alone lowers ATE from 4.90 to 2.63, and depth plus TSR reaches 1.94. VGAR then reaches 1.55 ATE but raises RPE from 0.53 to 0.68. On SCAND, VGAR worsens both trajectory metrics. Its benefit is therefore conditional rather than uniform.
Where the evidence stops. No uncertainty is shown, and this is not a complete factorial ablation. RECON full-model ATE and baseline RPE differ from Table 1. The prose's broad refinement claim must be qualified by the actual row comparisons.
Figure 6. Selected failures expose traversability ambiguity and missing heading dynamics. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each example contains an observation, a goal image, and a trajectory plot. Green marks the prediction and blue the ground truth; unlike the quantitative tables' grid units, these plot axes explicitly use meters. In the vegetation example, compare the predicted path's extent with the reference path. In the pedestrian example, compare direction as well as endpoint: the two paths separate as the ground-truth motion turns. Section 4.6 supplies the authors' interpretation of these examples, connecting the first to weeds treated as obstacles and the second to the action representation's lack of explicit orientation dynamics. The images alone cannot isolate either causal explanation. e-failurese-architecturee-protocole-long
What it supports. The paper's own failures show that goal conditioning and generated depth do not settle all navigation decisions. Semantically ambiguous terrain can disturb traversability judgments, and planar displacement prediction can miss sharp heading changes. These examples bound the practical interpretation of the favorable aggregate trajectory results.
Where the evidence stops. These are selected diagnostic examples, not a measured failure rate or a controlled test of semantics versus orientation. The plots show predicted trajectories; they do not document physical collisions or executed avoidance behavior.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors identify traversable weeds mistaken for obstacles and sharp-turn failures associated with displacement-only actions lacking explicit orientation dynamics. e-failures
Internal reporting differences remain unresolved: full-model RECON ATE is 0.93 in Table 1 versus 0.94 in Table 4; baseline RECON RPE is 0.73 versus 0.70; full-model TartanDrive PSNR is 18.11 in Table 2 versus 18.15 in Table 4. No uncertainties explain these differences. e-trajectoriese-videoe-ablatione-protocol
Depth LPIPS compares generated maps with pseudo-depth references, not measured geometry. The 64-step examples use a separate variable-length model, and the supplement explicitly distinguishes sequence length from physical travel distance. e-depthe-implementatione-long
7.2 Questions for discussion
- Does VGAR use sample-specific geometry, or mainly provide another learned action transformation?
- How would endpoint accuracy change under closed-loop execution with explicit heading dynamics?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the CogVideoX backbone/VAE, DepthAnything V3 pseudo-labels, pose processing, and NWM-compatible subsets. The source omits the exact backbone checkpoint, input resolution, training duration, hardware, software versions, diffusion sampling budget, TSR weight, detailed normalization, and subset counts; reported optimizer settings alone are insufficient. e-architecturee-denoisee-objectivee-protocole-implementation
Proposed checks: repeat matched depth/TSR/VGAR ablations with paired seeds and uncertainty, then test whether disturbing generated visual tokens specifically removes VGAR's action benefit. Neither check was run. e-vgare-ablatione-protocol
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Repeat the component rows under one fully specified protocol
Reader-proposed, not performed: train the five Table 4 configurations from the same identified CogVideoX checkpoint, using identical nine-frame splits, action normalization, optimizer schedule, and three paired seeds. Fix and disclose the TSR weight, diffusion sampler, hardware, and runtime measurement. Report per-seed ATE/RPE and paired uncertainty on all three datasets. The discriminating observation is whether VGAR again lowers TartanDrive ATE while worsening RPE and SCAND errors. If those differences reverse or are indistinguishable from run variation, the reported component tradeoff is not stable. Record Table 1 and Table 4 metrics from the same saved outputs to resolve their discrepancies. e-ablatione-trajectoriese-protocole-implementatione-objective
Check 2: Test whether VGAR needs the correct scene's visual tokens
Reader-proposed, not performed: hold a trained model's denoised action tokens fixed, and compare VGAR with its aligned RGB-D tokens, tokens substituted from a different episode of matching shape, and its residual gate set to zero. Keep the backbone output, endpoints, sampler, and action decoder identical across conditions. Measure paired changes in ATE/RPE and endpoint error, including the turning and vegetation cases motivating the limitations. If aligned tokens consistently outperform both controls, that supports sample-specific visual guidance. If mismatched tokens work equally well or disabling the residual helps, the claimed refinement mechanism needs qualification; altered-input distribution effects should also be reported. e-vgare-architecturee-ablatione-failurese-objectivee-protocol
8.3 Reading coverage
Visual audit: Original PDF pages 1–15 and 20–23 were rendered and visually inspected, including both title blocks, every main and supplementary figure/table, equations, evaluation details, failure discussion, and supplementary implementation settings. Reference-only pages 16–19 were read in the complete text chunks but were not visually rendered. Each of the six final original crops was separately viewed; legends, axes, headers, component checkmarks and all relevant rows are retained. Figure 2's arrows were checked against its caption and Equations (1)–(7): the equations clarify iterative denoising, VGAR's gated action-query update, and TSR's training-only role. The endpoint-depth inference description remains incomplete. Figure 3's printed 2.1× prose claim and the plotted ratio remain inconsistent; numerical differences between Tables 1/2 and 4 are preserved. Embedded supplementary material was inspected; availability of separate supplementary material was not verified.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 20, 21, 22, 23. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; 1 Introduction
- 2 Related Work, including 2.1–2.3
- 3 Method, including 3.1–3.5 and Equations (1)–(7)
- 4 Experiments, including 4.1–4.6
- 5 Conclusion; Acknowledgements
- References, PDF pp. 16–19
- Embedded Supplementary Material: 1 Implementation Details; 2 More Results, PDF pp. 20–23
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity: PDF p. 1 displays arXiv:2606.29908v1 [cs.RO], 29 Jun 2026. The observed title and all 13 authors match the supplied catalog after name-order normalization; the embedded supplement repeats their identity. No later revision or venue edition was supplied or compared. The catalog's ECCV 2026 venue label is not independently established by this artifact.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This supplied extraction limitation was addressed by inspecting original PDF pages and all six final crops.
- Separate supplemental material availability has not been fully verified. The supplement embedded on PDF pp. 20–23 was fully read; no separate supplement was supplied.
- All five supplied text chunks were read individually, covering all 23 PDF pages. Visual inspection covered pp. 1–15 and 20–23; reference-only pp. 16–19 were read as text. Referenced external works were not inspected.
- Code was not inspected and experiments were not reproduced. No external source or implementation was acquired.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author block and arXiv margin; p. 20, supplementary title block
Title and 13 authors agree with the catalog. The primary margin identifies arXiv:2606.29908v1, 29 Jun 2026; three institutions and cities are printed.
Go to primary source ↓e-framingPDF p. 1, Figure 1 and caption; pp. 2–3, Abstract and Section 1; p. 4, Section 2.2
The paper contrasts direct policies, candidate-ranked world-model rollouts, and joint RGB-D/action generation; it states monocular RGB input at test time and calls the planning process single-pass.
Go to primary source ↓e-architecturePDF p. 5, Figure 2 and caption; p. 6, Sections 3.1–3.2
The figure depicts DA3 endpoint depths, VAE encoding, one DiT, visual/action decoding, VGAR and TSR. Text defines local planar actions, frozen CogVideoX 3D VAE, MLP action tokens, joint targets, and separate endpoint conditioning.
Go to primary source ↓e-denoisePDF p. 7, Section 3.2, Equation (1) and following paragraph
The model iteratively denoises over T steps conditioned on start/goal RGB and depth latents, then decodes visuals and refines action tokens; no numerical T is specified.
Go to primary source ↓e-vgarPDF p. 7, Section 3.3, Equations (2)–(4)
Action queries cross-attend to final-layer visual keys/values; a sigmoid MLP gate and projection produce an additive residual before action decoding.
Go to primary source ↓e-objectivePDF pp. 7–8, Section 3.4, Equations (5)–(7)
The objective sums DDPM noise-prediction MSE and weighted TSR. TSR is the unsquared L2 endpoint error divided by N, with endpoint defined by summing denoised actions; its weight is symbolic only.
Go to primary source ↓e-initPDF p. 8, Section 3.5
The authors describe zero-weight initialization for newly added modules and progressive adaptation while preserving pretrained video priors.
Go to primary source ↓e-protocolPDF pp. 9–10, Section 4.1, Datasets, Baselines and Evaluation Metrics
Pose-derived actions use a unified scale. Evaluation follows NWM preprocessing and standardized subsets, with grid-unit ATE/RPE, threshold success, RGB similarity metrics, three-run averages, and serial per-episode timing. Baseline ×N denotes candidate budget. Exact split counts and uncertainty are absent.
Go to primary source ↓e-trajectoriesPDF p. 10, Table 1, all dataset and Time(s) columns
SWAM's ATE/RPE pairs are 0.93/0.43, 1.15/0.34 and 1.55/0.68 versus ×16's 1.53/0.49, 2.18/0.46 and 6.23/1.30. Timing is 16.91 seconds versus 245.98; direct policies and CogVideoX Joint are faster than SWAM.
Go to primary source ↓e-successPDF p. 10, Section 4.2, Goal-reaching Performance; p. 11, Figure 3 and caption
Figure 3 plots RECON success at 0.25/0.5/1 grids, with SWAM highest. Text states 2.1× over ×16 at 0.25, whereas plotted markers appear around 33% and 12–13%; exact values are not tabulated, so the ratio is unresolved.
Go to primary source ↓e-videoPDF p. 13, Table 2, all dataset columns; p. 11, Section 4.2, Video Generation Quality
SWAM has best PSNR/SSIM across the three datasets. TartanDrive LPIPS is 0.335, worse than CogVideoX Joint's 0.330, qualifying the prose's general best-quality claim. Main-table TartanDrive PSNR is 18.11.
Go to primary source ↓e-transferPDF p. 13, Table 3 and Section 4.4
SWAM without HuRoN training/fine-tuning obtains 2.94 ATE and 0.85 RPE; NWM+NoMaD ×16 explicitly trained on HuRoN obtains 3.73 and 0.95.
Go to primary source ↓e-ablationPDF p. 14, Table 4, Depth/TSR/VGAR rows and Section 4.5
Rows isolate depth, TSR, depth+TSR and the full model. VGAR lowers TartanDrive ATE from 1.94 to 1.55 but raises RPE from 0.53 to 0.68; SCAND ATE/RPE also worsen. Table 4 reports RECON baseline RPE 0.70, full ATE 0.94 and full TartanDrive PSNR 18.15.
Go to primary source ↓e-failuresPDF pp. 14–15, Section 4.6; p. 15, Figure 6
The authors attribute weed-related avoidance/drift to missing semantic traversability reasoning, and sharp-turn errors to planar displacements without explicit orientation. Figure 6 contrasts green predicted trajectories with blue ground truth.
Go to primary source ↓e-implementationPDF p. 20, Supplementary Material Section 1, Implementation Details
Main experiments use eight observation–action pairs and nine frames. A distinct visualization variant samples 9/17/33/65 frames with RoPE rescaling. Reported optimization uses bf16, checkpointing, batch size 1, Adam at 10^-4, 1,000-step warm-up, norm clipping 1.0 and epsilon 10^-15; hardware, software versions and training duration are absent.
Go to primary source ↓e-depthPDF pp. 20–21, Supplementary Material Section 2, Depth Estimation Results and Table 1
Depth LPIPS is 0.224/0.243/0.182 on RECON/SCAND/TartanDrive. References are pretrained-model pseudo-depth rather than ground-truth measurements, motivating the authors' perceptual evaluation.
Go to primary source ↓e-longPDF p. 20, Supplement Section 1; pp. 21 and 23, Section 2, Longer-sequence Visualization Results; p. 23, Figure 3
The separately trained variable-length model illustrates predictions up to 64 steps. The supplement explicitly distinguishes sequence length from travel distance and describes SWAM as a high-level planner rather than a local executor.
Go to primary source ↓e-qualitativePDF pp. 11–12, Figures 4–5 and Section 4.3; pp. 21–23, Supplement Figures 1–3 and Section 2
Qualitative panels compare trajectory scale and video evolution, show HuRoN transfer, and provide RGB-D examples across environments. These are selected predictions rather than measured execution outcomes or failure frequencies.
Go to primary source ↓8.5 Primary sources
Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation ↗
PDF · 7,656 extracted words
Source fingerprint
889439c2df2da1c8aaa769c24fad3b616b464e25ee3ceec051d3058ef0dd28f0