NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation
1. Paper overview
In one sentence: NavWAM turns a video diffusion transformer into an action-producing navigation policy, with evidence that future-image supervision helps offline accuracy and limited-scale evidence of physical deployment. formulationarchitecturefuture-ablationrobot-resultsvalue-definitionhyperparameters
| At a glance | What to know |
|---|---|
| Research problem | Source description A robot must connect a distant goal image to local motion under partial observability. An action-conditioned navigation world model can imagine candidate futures, but an external planner must select actions. NavWAM learns this action-producing role within the same predictive representation. formulation |
| Core mechanism | Source description The central contribution is a shared video-latent interface for visual and nonvisual navigation variables, using direct frame readout instead of separate action/value heads. architecture |
| A key reported result | Closed-loop Diablo image-goal navigation: 19/24; 79.2%; [59.5, 90.8]%. Success count/rate; 95% Wilson interval. 24 shared start/goal episodes per method in four indoor environments; randomized method order; success within 1 m. OmniVLA 14/24, 58.3%, [38.8, 75.5]%; NWM 4/24, 16.7%, [6.7, 35.9]%. Physical execution supports feasibility, with limited sample size. Marginal intervals do not establish paired significance; failure labels include operator safety stops. robot-resultsfailure-countsevaluation |
| Reading caution | Source description Evaluation covers static indoor image goals on one platform. Dynamic obstacles, long multi-room routes, alternative goal modalities and cross-platform transfer remain unverified. Friction-induced over-rotation is a proposed cause of the illustrated failure, not a measured causal intervention. limitationsfailure-rollout |
Core contributions
- Source description
The central contribution is a shared video-latent interface for visual and nonvisual navigation variables, using direct frame readout instead of separate action/value heads. architecture
- Reader analysis
The authors argue that joint future, action and progress prediction makes visual foresight useful for control. The controlled future-loss ablation supports a training benefit, while the specific causal explanation remains open. future-ablation
Figure 2. One latent canvas carries the observations and the variables needed for action. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read from the lower conditioning row through the central Cosmos Predict2 2B block to the upper generated row. F0 is temporal padding; F1 is current state; F2 and F3 carry the goal and current images. F4–F8 then represent the action chunk, future state, two future views and value. The VAE blocks belong to image encoding and decoding. The state, action and value arrows indicate direct readout from denoised latent entries. Section 4.1 and Table S1 confirm this observed/generated partition; the graphic does not specify an attention mask or a separate planning loop. architecturecanvasformulationinference
What it supports. The figure supports a shared-transformer architecture: actions and future images occupy the same denoising sequence. This is the basis for the catalog's One Model × Joint prediction assessment. Action extraction uses the denoised action frame, so it does not require a separate inverse-dynamics network.
Where the evidence stops. Equation (3) abbreviates conditioning to observation and goal, while this figure and Table S1 also require current state. Future images and value are jointly produced, but their decoded outputs need not be consulted to execute the default policy.
2. Motivation
2.1 The problem and the proposed response
A robot must connect a distant goal image to local motion under partial observability. An action-conditioned navigation world model can imagine candidate futures, but an external planner must select actions. NavWAM learns this action-producing role within the same predictive representation. formulation
2.2 What this reading follows
A navigation world model can predict the view after a candidate motion, but the robot still needs an action-selection rule. NavWAM moves action generation into the predictive model by placing actions, future views, state and goal progress in one latent canvas. This reading follows that information flow, distinguishes training supervision from inference-time execution, and then separates three kinds of evidence: offline trajectory error, a controlled future-loss ablation and physical goal reaching. The most useful reproduction targets are the matched supervision comparison and the value-label definition. The source also leaves important implementation details unresolved, especially diffusion-step counts and the interpretation of its future-indexed value target. formulationarchitecturefuture-ablationrobot-resultsvalue-definitionhyperparameters
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded One Model × Joint prediction classification is supported by one diffusion transformer jointly denoising action, state, future-image and value frames. Direct action-frame readout is architectural evidence beyond joint training alone. A causal VAE and physical controller remain surrounding components; action extraction is not an inverse-dynamics model, and main inference does not use an external visual planner. architecturecanvasinferenceplatform
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn video frames into a policy interface
Start with the obstacle posed by a predictive world model: it answers what might be seen after an action, while a separate procedure chooses the action. NavWAM instead allocates parts of the video model's input/output sequence to the action itself. The goal and current images are encoded into their observed frames; state occupies another observed frame. A common transformer then denoises the action, future state, future images and value. Spatial averaging converts non-image frames back into low-dimensional variables. This explains why the architecture can be classified as joint prediction without invoking an inverse-dynamics decoder. As a reader interpretation, the shared canvas offers a route for visual supervision to shape the action-generating representation; shared placement alone does not establish exactly which internal features influence each action. Table S5 supplies the empirical intervention needed to go beyond that architectural possibility. formulationarchitecturecanvasfuture-ablation
5.2 Separate joint training from action execution
The training masks determine which prediction problem a sample teaches. Policy samples observe Frames 0–3 and generate the remaining five frames. World-model samples also observe the action frame, and value samples observe everything except the value. The 50/25/25 mixture trains all three conditional roles in one set of weights. However, the reported main controller uses only the first role: generate a chunk, execute it, then obtain another observation. Future images and value accompany the action but are not necessary readouts for execution. The controlled future-loss ablation is therefore evidence about what supervision teaches the policy, not evidence that a separate image-based evaluator selects its moves. Optional best-of-N would introduce such an explicit selection stage using predicted values, but the paper does not demonstrate a benefit from that optional mode. objectiveinferencestate-actionfuture-ablation
5.3 Evaluate progress labels before trusting progress scores
The value frame is described as predicting progress at the future state, yet Equation (5) computes its target from the current position and the recorded trajectory endpoint. This is a concrete indexing ambiguity, not something to repair silently. Another question is how that endpoint relates to the later trajectory image sampled as the training goal. The cap is described only as an upper percentile, leaving the numeric normalization unresolved. Figure S3 shows one rollout whose values dip when the goal leaves view and later recover; the authors explicitly advise qualitative interpretation. As reader analysis, that temporal association cannot establish probability calibration or prove that the score will rank alternative actions correctly. Before implementing value-guided sampling, a reproduction should pin down the target index, goal correspondence and scale, then test ranking against observed outcomes. value-definitioncanvasqualitativeinference
5.4 Training and inference
During training
Training starts with success-only HM3D trajectories (802 scenes; 185,000 trajectories), then adapts on RECON/SACSON/SCAND (11,835/2,000/372 trajectories). Simulation uses H=16; adaptation resets H=4. Optional in-domain fine-tuning uses 3,544 GO STANFORD trajectories, disjoint from 30 test episodes. curriculumdatasetsstate-action
Each sample selects policy, world-model or value conditioning with probabilities 50/25/25 percent. These observe Frames 0–3, 0–4 or 0–7 respectively. Generated-frame denoising uses an action multiplier of 5. AdamW uses learning rate 10^-4, effective batch 32 on four RTX PRO 6000 GPUs, bfloat16 and no EMA. objectivecurriculumhyperparameters
During inference
Main evaluations use one policy-mode denoising chain without CEM or best-of-N. The robot consumes the predicted chunk and then queries on a fresh observation. Co-predicted images and value are available for interpretation but are not needed to execute the action output. inferencestate-action
Optional value-guided candidate selection is described, but its robustness benefit is untested. A single denoising chain must not be equated with a single transformer evaluation or proven onboard latency. inferenceefficiencylimitations
5.5 Implementation flow
- Build the canvas
Cosmos Predict2 2B provides the diffusion transformer and causal VAE. Observed Frames 0–3 contain blank padding, current state, goal image and current image. Generated Frames 4–8 hold actions, future state, two future images and value. This nine-frame latent sequence corresponds to 33 raw VAE frames. architecturecanvas
- Encode and read out
Image frames use the VAE. Normalized state, action and value vectors are broadcast over the spatial grid and recovered by averaging denoised entries. State uses position divided by 100 and yaw divided by π. Actions contain local translation and yaw increments; the post-adaptation chunk has H=4. architecturestate-action
- Keep variants distinct
The image-goal variant places a later trajectory image in Frame 2 during training. The WM variant instead uses the previous observation and the backbone's text interface. Its description mentions language goals, but the paper explicitly leaves language-goal evaluation untested. canvaslimitations
6. Experiments & results
NavWAM adapts a video diffusion transformer into an image-goal navigation policy by jointly denoising actions, future views, state and goal progress. Its strongest mechanism evidence is a controlled future-image-loss ablation; its physical deployment reports 19/24 successes. The default policy avoids candidate-action search, but the study does not establish calibrated foresight or broad real-world robustness.
6.1 Read the original evidence
Table 1. Direct policy output lowers offline trajectory error, with another gain from in-domain adaptation. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the two lower-is-better columns, then compare the untuned and fine-tuned NavWAM rows separately. The first two rows use future prediction with CEM action selection, whereas NavWAM uses its default policy mode. The untuned row means no GO STANFORD-specific fine-tuning, not absence of earlier real-robot adaptation. Appendix B describes a 30-episode held-out GO STANFORD subset and a disjoint training split for the extra fine-tune. Table S4 supplies the corresponding horizon-8 setting, although Table 1 itself does not label the horizon. This is an offline trajectory comparison, distinct from the Diablo deployment. offline-resultsdatasetscurriculumevaluationinference
What it supports. NavWAM reports ATE/RPE of 0.324/0.099 before in-domain fine-tuning and 0.192/0.070 afterward. Table 1 reports NWM at 0.453/0.107. The results favor the action-producing policy in this evaluation, and the fine-tuning difference shows why the two NavWAM rows must remain separate.
Where the evidence stops. The source gives no uncertainty for these offline entries. Table S4 reports NWM ATE 0.452 at N=120 instead of Table 1's 0.453. Neither value is silently corrected here, and neither is a physical success rate.
Table 4. The physical deployment reaches 19 of 24 goals under a shared episode protocol. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each environment entry as successes divided by attempted episodes. The denominators differ across environments, so the final success-rate column aggregates episode counts rather than averaging four percentages. Appendix C.3 states that methods received the same start and goal image, with their execution order randomized within the session. Success means reaching within one meter of the operator-marked goal. The table shows that NavWAM's overall advantage includes gains in Office, Storage and Meeting; it ties OmniVLA in Hallway. The nearby rollout photos are selected examples, while these counts cover the full reported deployment set. robot-resultsfailure-countsevaluationlimitations
What it supports. The total is 19/24 for NavWAM, compared with 14/24 for OmniVLA and 4/24 for NWM. The accompanying text reports a 95% Wilson interval of 59.5–90.8% for NavWAM. These are executed robot episodes, providing stronger deployment evidence than future-image similarity alone.
Where the evidence stops. Twenty-four episodes on one robot in static indoor scenes leave substantial uncertainty. Per-method Wilson intervals are not a paired significance test. Appendix failure labels count both obstacle contact and an operator safety stop as collision.
Table S4. The reported cost advantage is specific to one hardware and timing protocol. Original paper, p. 18 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the NWM block vertically to see candidate budget N change, then compare the Cosmos Predict2+CEM row with NavWAM, which uses the same named 2B backbone family. ATE and RPE describe trajectory error; the remaining columns report FLOPs per executed action in teraflops, latency in milliseconds and peak allocated GPU memory. The source measures on one RTX PRO 6000 with 96 GB, using bfloat16 and 224×224 inputs, averaging 100 steps after 20 warmup steps. Candidate rollout and goal scoring are included for CEM; NavWAM includes its full denoising chain. These timing conditions come from Appendix C.1, outside the crop. efficiencyhyperparametersplatform
What it supports. NavWAM reports 205.7 ms, 4.45 TF per action and 4.82 GB, while NWM at N=120 reports 233,831 ms, 14,521 TF and 51.65 GB. The table supports a large implementation-specific cost separation and shows that the reported NavWAM cost does not depend on a CEM candidate budget.
Where the evidence stops. The denoising-step count is not provided, limiting exact replication. The robot platform includes a Jetson AGX Orin, but this table measures a desktop GPU. Its very large planner latencies should be reproduced before treating the gap as generally representative.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| GO STANFORD offline image-goal navigation 30 held-out episodes; corresponding horizon 8 in Table S4; policy mode with or without in-domain fine-tuning. | NavWAM 0.324 / 0.099; with FT 0.192 / 0.070. ATE / RPE, lower is better | Table 1: NWM+CEM 0.453 / 0.107; Cosmos Predict2+CEM 0.455 / 0.109. Fine-tuning materially improves accuracy. Table S4 instead prints NWM ATE 0.452; this discrepancy is preserved. Offline errors do not measure physical goal reaching. offline-resultsdatasetsinference |
| GO STANFORD future-view consistency Figure 3 comparison accompanying the offline benchmark. | NavWAM 0.668; with FT 0.635. Subject consistency, higher is better | NWM 0.524. Fine-tuning improves trajectory metrics while lowering this visual metric. Feature similarity does not certify action execution or dynamics accuracy. visual-consistencyevaluationoffline-results |
| SIT held-out direct-policy comparison 1,400 episodes across 14 official test segments; h=4/8. | NavWAM ATE 0.077/0.144; SR 46.3/15.9. ATE; SR@1.0m (%) | OmniVLA ATE 0.086/0.162; SR 45.4/12.1. Observed gains are modest; no uncertainty is reported. SIT avoids known GO STANFORD training overlap, but the 2B-versus-7B comparison does not isolate architecture or data effects. sit-resultsdatasetsevaluation |
| Closed-loop Diablo image-goal navigation 24 shared start/goal episodes per method in four indoor environments; randomized method order; success within 1 m. | 19/24; 79.2%; [59.5, 90.8]%. Success count/rate; 95% Wilson interval | OmniVLA 14/24, 58.3%, [38.8, 75.5]%; NWM 4/24, 16.7%, [6.7, 35.9]%. Physical execution supports feasibility, with limited sample size. Marginal intervals do not establish paired significance; failure labels include operator safety stops. robot-resultsfailure-countsevaluation |
| Effect of future-image supervision Same backbone, canvas, data, three-phase curriculum and step budget; 30 GO STANFORD episodes; h=4/8. | With future loss: ATE 0.076/0.192; RPE 0.037/0.070. ATE / RPE, lower is better | Without future loss: ATE 0.090/0.262; RPE 0.045/0.103. A cleaner mechanism test than Table 2: future frames remain generated while only their loss changes. It supports supervision's benefit without proving inference-time causal use of decoded images. future-ablationtarget-ablationinference |
| Inference efficiency against CEM planners One RTX PRO 6000 96 GB; bfloat16; 224×224; 100 steps after 20 warmup; per-executed-action normalization. | NavWAM: 4.45; 205.7; 4.82. FLOPs/action (TF); latency (ms); peak allocated GPU memory (GB) | NWM, N=120: 14,521; 233,831; 51.65. Cosmos Predict2+CEM, N=120: 18,114; 887,606; 20.04. These are author-reported measurements under this implementation. The extreme gap warrants timing replication; the desktop-GPU result is not a Jetson measurement. efficiencyplatform |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table S5. Keeping the canvas fixed makes future-image loss the controlled intervention. Original paper, p. 18 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The left group names supervised targets, not separate neural heads: image futures, action, state and value. Both rows keep the same nine-frame canvas and still generate Frames 6/7. Only the future-image denoising loss is zeroed in the upper row. Appendix C.2 states that backbone, curriculum, datasets, optimizer and training-step budget are shared. Compare each column vertically, taking care to distinguish h=4 from h=8 and ATE from RPE. Unlike Table 2's image-only-planning comparison, this experiment does not switch between an external planner and direct policy inference, making it a more focused test of future-image supervision. future-ablationtarget-ablationarchitectureinference
What it supports. At h=8, adding the future loss changes ATE from 0.262 to 0.192 and RPE from 0.103 to 0.070. Both h=4 metrics also improve. This is the clearest evidence that future-view supervision benefits the trained policy in the evaluated setting.
Where the evidence stops. No across-seed variability is given. The comparison cannot separate useful transition learning from generic representation regularization, and it does not prove that decoded future pixels causally guide action selection at inference.
Figure S4. A coherent-looking predicted future can accompany a trajectory that ends in failure. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The top row records actual egocentric views; the lower row shows predictions made at successive decision points. A prediction labeled t=8→12 should be compared with the observed t=12 view, not merely the image directly above it. Continue this offset comparison through t=16→20, and keep the red goal panel separate from predicted observations. The last observed panel carries a red cross even though its border says End. Appendix D.1 describes an over-rotation near t=12, followed by inconsistent prediction and a collision. This visual illustrates the described sequence; it does not itself measure wheel friction or the robot's pose. failure-rolloutfailure-countsstate-actioninference
What it supports. The example exposes a failure boundary of joint prediction: physical execution can depart from the model's expected trajectory, and re-querying from the resulting view does not guarantee recovery. Future images are useful diagnostic outputs, but visual plausibility alone does not verify that a chunk will be safely executed.
Where the evidence stops. The authors suggest friction as a possible cause; they do not establish it with a controlled perturbation. This selected rollout cannot estimate failure frequency. The aggregate failure table assigns one dominant label even when drift and collision compound.
7. Analysis & limitations
7.1 What the evidence leaves open
Evaluation covers static indoor image goals on one platform. Dynamic obstacles, long multi-room routes, alternative goal modalities and cross-platform transfer remain unverified. Friction-induced over-rotation is a proposed cause of the illustrated failure, not a measured causal intervention. limitationsfailure-rollout
Table 2 changes targets and inference procedure together. Table S5 better isolates future supervision, but supplies no seed variation. The single value rollout is qualitative; Figure S2 also compares unequal prediction horizons. Neither establishes calibrated uncertainty. target-ablationfuture-ablationqualitative
Table S6 records NavWAM drift/collision counts of 3/2 versus OmniVLA's 3/7: drift is tied, despite prose implying superiority in both modes. Success examples and failure explanations should therefore be read alongside the actual counts. failure-counts
7.2 Questions for discussion
- How much of the future-loss gain reflects transition learning versus preservation of a useful video representation?
- Would future-indexed, goal-matched value targets improve candidate ranking beyond the default policy?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the stated pretrained backbone, trajectory data/splits, frame packing and dataset-specific normalization, plus the Diablo action interface for deployment. The source specifies ROS 2 cmd_vel and operator-supervised hardware stopping, but does not fully specify the waypoint-to-controller conversion or deployment state estimator. architecturecurriculumdatasetsplatformlimitations
Unresolved details include phase training-step and warmup counts, denoising-step count, frozen-module policy, exact d_max percentile, complete action rescaling bounds, native control rates, timeout duration and exact episode identifiers. Noise ranges and waypoint spacings are supplied; they do not resolve these gaps. hyperparameterscurriculumvalue-definitionstate-actionevaluationdatasets
Proposed first check: repeat the matched future-loss ablation across seeds with paired episode errors. Separately compare printed current-position value labels with future-position labels; this is a reader-designed clarification experiment, not a reported reproduction. future-ablationvalue-definition
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does correctly aligned future supervision add more than a dense image loss?
Reader-proposed experiment, not run: repeat Table S5 with identical initialization, nine-frame canvas, data, schedule and policy inference across several seeds. Compare zero future loss, the correct future-image loss, and an equal-weight control whose future targets are shuffled within the same training scene. Keep evaluation episodes disjoint and fixed. Report paired ATE/RPE differences at h=4 and h=8, seed variation and future-image consistency. Correctly aligned targets should consistently outperform both controls if useful transition supervision contributes beyond merely retaining a dense image objective. Similar performance for correct and shuffled targets would weaken that interpretation, even if both beat zero loss. future-ablationdatasetsvisual-consistencyobjective
Check 2: Resolve the future-value index and test action ranking
Reader-proposed experiment, not run: on fixed held-out trajectories, compare labels computed literally with current p_t against labels using the future position p_(t+H). Use the same sampled goal images and explicitly test goal-matched endpoints versus recorded trajectory endpoints; predeclare and record the d_max percentile. Keep the training budget and candidate action set fixed. Measure label agreement, predicted-value ranking against realized goal-distance reduction, and any success change from optional best-of-N under a fixed compute budget. If the future-position, goal-matched definition improves ranking and selection, it supports the stated future-progress interpretation; if not, the value may mainly describe current scene/goal proximity. Obtain the original label convention before claiming an exact reproduction. value-definitioncanvasinferencequalitativehyperparameters
8.3 Reading coverage
Visual audit: The title/author/version page, every main-paper figure and table, all appendix figures and tables, equations, training configuration, dataset counts, evaluation protocols, robot hardware, efficiency setup and limitations were visually inspected. Final crops were each inspected: Figure 2, Tables 1 and 4, Tables S4 and S5, and Figure S4. All supporting method and numerical pages are declared, including uncropped appendix pages. Pages 9–12 contain references and were read as supplied text; no external cited work or separate supplement was opened.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF pp. 1–8: Abstract; Sections 1–6, including complete method, training, inference, experimental setup, results and conclusion
- PDF pp. 9–12: References
- PDF pp. 13–14: Appendix A.1–A.5, canvas layout, value, state, actions and conditioning modes
- PDF pp. 15–17: Appendix B.1–B.5, hyperparameters, curriculum, datasets and platform
- PDF pp. 17–21: Appendix C.1–C.4.2, efficiency, controlled ablation, failure counts, future views and value rollout
- PDF pp. 21–22: Appendix D.1–D.2, failure mechanisms, unverified regimes, limitations and deployment
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity notes: the inspected title page is arXiv:2606.13494v1 [cs.RO], dated 11 June 2026. Title and all nine authors match the catalog. The catalog affiliation string is incomplete; the inspected affiliations are recorded in metadata. No other edition or revision chain was supplied or compared.
- The supplied extraction did not reconstruct figure images; this limitation was addressed by inspecting all original figures and tables in the retained PDF.
- Separate supplemental material availability has not been fully verified. The embedded Appendices A–D were supplied and fully read.
- No project website, code, checkpoints, external references or separate supplements were inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title block and arXiv margin
The title and nine authors match the catalog; the observed artifact is arXiv:2606.13494v1 [cs.RO], 11 June 2026. Affiliations are The University of Tokyo, National Institute of Informatics, AIRoA and ATR.
Go to primary source ↓formulationPDF pp. 2–4, Figure 1, Sections 3–3.2, Eqs. (1)–(3)
Direct policies predict actions; navigation world models predict observations conditioned on candidate actions and require an external selector. NavWAM formulates joint action, future-state, future-observation and value prediction.
Go to primary source ↓architecturePDF pp. 4–5, Section 4 and Section 4.1, Figure 2
Cosmos Predict2 supplies the causal VAE and shared diffusion transformer. Frames 0–3 condition Frames 4–8. Images use the VAE; normalized non-image variables are broadcast and recovered by averaging, without separate action/value heads.
Go to primary source ↓canvasPDF p. 13, Appendix A.1, Table S1
Nine latent frames correspond to a 33-raw-frame causal-VAE chunk. Frame 2 contains a goal image in the IG variant or the previous observation in the WM variant, which uses the text interface for goal specification. IG training goals are sampled from later steps of the same trajectory.
Go to primary source ↓state-actionPDF p. 14, Appendix A.3–A.4, Eqs. (6)–(7)
State is [x_t/100,y_t/100,psi_t/pi] using recorded-frame positions in meters and yaw in radians. Actions are H local-frame waypoint increments with translation and yaw. H=16 in simulation pretraining and H=4 thereafter; online execution consumes the chunk at the dataset-native rate before querying again.
Go to primary source ↓objectivePDF p. 5, Section 4.2, Eq. (4); PDF p. 14, Appendix A.5, Eq. (8) and mode list
Weighted denoising error is applied to generated frames. Per-sample policy/world-model/value modes occur with probabilities 50/25/25 percent; their observed frames are 0–3, 0–4 and 0–7 respectively.
Go to primary source ↓inferencePDF p. 6, Sections 4.2–4.3
Main results use policy mode without best-of-N or CEM. Future images and value are co-predicted but not required for execution. Optional best-of-N would sample candidates and use auxiliary modes to select the highest predicted value.
Go to primary source ↓value-definitionPDF pp. 13–14, Appendix A.2, Eq. (5) and its definitions
The printed future-indexed value uses clipped one-minus-distance from current position p_t to trajectory endpoint p_end, divided by d_max. The cap is an unspecified upper percentile of training trajectory lengths. HM3D uses geodesic distance from the Habitat shortest-path API.
Go to primary source ↓curriculumPDF p. 15, Appendix B.1–B.3
The action multiplier is 5; waypoint spacings are RECON 0.25 m, SACSON 0.255 m, SCAND 0.38 m and GO STANFORD 0.12 m. Training proceeds through success-only HM3D, joint RECON/SACSON/SCAND adaptation, and optional GO STANFORD fine-tuning. Step counts are not supplied in the phase descriptions.
Go to primary source ↓datasetsPDF pp. 15–16, Appendix B.3, Table S2
HM3D has 802 scenes/185,000 trajectories; Phase 2 uses 11,835 RECON, 2,000 SACSON and 372 SCAND trajectories. GO STANFORD fine-tuning has 3,544 trajectories with a disjoint 30-episode test subset. SIT uses 14 test segments and 1,400 episodes.
Go to primary source ↓hyperparametersPDF p. 16, Appendix B.4, Table S3
The backbone is Cosmos-Predict2 2B Video2World. Training uses AdamW at 10^-4, batch 8 on each of four RTX PRO 6000 GPUs, effective batch 32, bfloat16, cosine scheduling with linear warmup and no EMA. The table gives noise ranges but no denoising-step count; its caption refers to phase step counts/warmup not enumerated in B.2.
Go to primary source ↓evaluationPDF p. 6, Section 5.1, datasets, baselines, metrics and real-world setup
Evaluation uses image goals; ATE and RPE are lower-is-better at horizons 4 and 8, SR@1.0m measures goal reaching, and subject consistency is feature similarity to ground-truth futures. SIT avoids OmniVLA's GO STANFORD training overlap. Real-robot inputs are 224x224 RGB, with goal-distance or fixed-timeout termination.
Go to primary source ↓offline-resultsPDF p. 7, Table 1 and Section 5.2; PDF p. 18, Table S4 caption and accuracy columns
Table 1 reports ATE/RPE 0.455/0.109 for Cosmos Predict2, 0.453/0.107 for NWM, 0.324/0.099 for NavWAM, and 0.192/0.070 with fine-tuning. Table S4 identifies the corresponding evaluation horizon as 8 and reports NWM ATE 0.452 at N=120, a discrepancy from Table 1.
Go to primary source ↓visual-consistencyPDF p. 7, Figure 3, Figure 4 and Section 5.2, Preserving Visual Foresight
Subject consistency is 0.524 for NWM, 0.668 for NavWAM and 0.635 after fine-tuning. Figure 4 shows a selected GO STANFORD prediction comparison at H=4.
Go to primary source ↓target-ablationPDF p. 7, Table 2 and Section 5.2, Learning Useful Futures for Control
Image-only supervision with CEM N=120 gives ATE 0.326/0.569 at h=4/8. Adding action/state supervision and switching to policy gives 0.107/0.287. Adding value gives 0.076/0.192; RPE becomes 0.037/0.070. The first comparison changes both targets and inference procedure.
Go to primary source ↓sit-resultsPDF pp. 7–8, Section 5.2, Predictive Policies vs. Direct Policies, Table 3
On SIT, NavWAM ATE is 0.077/0.144 and SR is 46.3/15.9 percent at h=4/8. OmniVLA gives 0.086/0.162 and 45.4/12.1 percent. The compared backbones are 2B and 7B parameters; training and architecture are not controlled by that comparison.
Go to primary source ↓robot-resultsPDF pp. 7–8, Section 5.2, Closed-loop Real-Robot Deployment, Table 4 and Figure 5
NavWAM succeeds in 19/24 episodes (79.2%, 95% Wilson interval 59.5–90.8), OmniVLA in 14/24 (58.3%, 38.8–75.5), and NWM in 4/24 (16.7%, 6.7–35.9). NavWAM environment counts are Office 6/8, Storage 6/6, Meeting 4/6 and Hallway 3/4.
Go to primary source ↓failure-countsPDF p. 19, Table S6 and Appendix C.3; PDF p. 20, opening continuation
Methods share start/goal pairs and randomized within-session order. Failures receive a dominant drift or collision label; collision includes operator safety stops. Aggregate drift/collision counts are NWM 13/7, OmniVLA 3/7 and NavWAM 3/2. NavWAM ties OmniVLA on drift despite prose describing the lowest count in both modes.
Go to primary source ↓efficiencyPDF pp. 17–18, Appendix C.1, Table S4
Measurements use one RTX PRO 6000 96 GB, bfloat16 and 224x224 inputs, averaged over 100 steps after 20 warmup steps and normalized per executed action. NavWAM reports 4.45 TF, 205.7 ms and 4.82 GB. NWM at N=120 reports 14,521 TF, 233,831 ms and 51.65 GB; Cosmos Predict2+CEM reports 18,114 TF, 887,606 ms and 20.04 GB.
Go to primary source ↓future-ablationPDF p. 18, Table S5 and Appendix C.2; PDF p. 19, Why Future-View Supervision Helps
Both variants share backbone, nine-frame canvas, curriculum, data and training-step budget. Turning on the Frames 6/7 denoising loss changes ATE from 0.090/0.262 to 0.076/0.192 and RPE from 0.045/0.103 to 0.037/0.070 at h=4/8 on 30 GO STANFORD episodes. Proposed representation/foresight explanations are not separately isolated.
Go to primary source ↓platformPDF pp. 16–17, Appendix B.5 and Figure S1
The Diablo platform carries a RealSense D455 RGB-D camera, Livox Mid-360 LiDAR and Jetson AGX Orin, and executes local-frame chunks using ROS 2 cmd_vel. These platform components do not establish that the desktop-GPU timing measurement runs on the Orin.
Go to primary source ↓qualitativePDF p. 8, Figure 6; PDF pp. 20–21, Appendix C.4, Figures S2–S3
Figure 6 aligns observed execution views with H=4 predictions. Figure S2 compares NWM at step 8 with NavWAM at step 4, so horizons differ. Figure S3 shows one value rollout with a dip at t=8 and later recovery; the text says values should be read qualitatively, not as calibrated progress probabilities.
Go to primary source ↓failure-rolloutPDF pp. 21–22, Appendix D.1, Figure S4
The authors describe over-rotation near t=12, possibly from wheel friction, followed by inconsistent future prediction, drift and collision. The original panel pairs executed observations with predicted future views and marks a failed ending.
Go to primary source ↓limitationsPDF p. 8, Conclusion limitations; PDF p. 22, Appendix D.1–D.2 and Safety and Deployment
Dynamic obstacles, multi-room/multi-floor navigation, language/object goals and other robot platforms are untested. Real-world evaluation is limited to 24 static indoor episodes. Diffusion-chain shortening is unstudied; value scales are dataset-dependent. Deployment uses operator supervision and a hardware safety stop.
Go to primary source ↓8.5 Primary sources
NavWAM: A Navigation World Action Model for Goal-Conditioned Visual Navigation ↗
PDF · 9,984 extracted words
Source fingerprint
fd35fd9d82e5f098a61fbb8ae486ac3359234b93d29172f4553a4ce1f697b559