FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation
1. Paper overview
In one sentence: FutureNav strengthens a shared navigation policy with spatial features and auxiliary transition learning, while leaving future-prediction heads inactive during default action decoding. problemarchitectureencodingtraining-inferencemain-resultsobjective-ablationtraining-progressqualitative
| At a glance | What to know |
|---|---|
| Research problem | Source description Continuous vision-and-language navigation must connect instructions and egocentric RGB history to low-level movements while retaining spatial progress. The authors argue that action imitation weakly supervises how actions change the environment. FutureNav adds spatial-state and transition supervision to the action-decoding representation. problemarchitecture |
| Core mechanism | Source description A shared VLM combines language, visual tokens and frozen geometry-aware features through residual fusion, giving spatial information to the language backbone that predicts executable actions. architectureencoding |
| A key reported result | R2R-CE navigation, full-data 4B: 65.4% SR; 61.3 SPL; 4.24 NE SR / SPL (higher better); NE (lower better). Habitat/Matterport3D val-unseen; 1,839 trajectories; single RGB; 10.457M external samples JanusVLN-7B: 60.5% SR, 56.8 SPL, 4.78 NE; 10.692M external samples. SR improves 4.9 percentage points, equivalent to the reported 8.1% relative gain. Similar data volume does not establish identical recipes. setupmain-results |
| Reading caution | Reader analysis Results lack confidence intervals or repeated-seed variability. Ablations support useful auxiliary supervision but do not measure counterfactual dynamics accuracy or long-horizon rollout fidelity. The source provides no dedicated limitations section. objective-ablationlatent-ablationreproduction-gaps |
Core contributions
- Source description
A shared VLM combines language, visual tokens and frozen geometry-aware features through residual fusion, giving spatial information to the language backbone that predicts executable actions. architectureencoding
- Source description
Forward dynamics, inverse dynamics and action-free future prediction supplement action-token training. Default inference does not imagine candidate futures or use inverse dynamics to select the next action. forwardinversegenerationtraining-inference
Figure 2. Spatial input fusion and four training tasks share one navigation backbone. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read downward from observations and instruction to their token streams. Snowflakes mark frozen encoders; flame symbols mark trainable components. The central plus represents residual addition of projected spatial features to VLM visual tokens, formalized by Eq. (2). The lower branches separate action policy, inverse dynamics, action-conditioned forward prediction and action-free future generation. Only the policy is marked Train & Infer. The diagram's spatial-feature head inputs are schematic: Eqs. (4), (7) and (9) specify LLM hidden-state inputs. Likewise, its inverse pair is labeled t,t+1, while Eq. (7) uses previous/current frames t-1,t. Use the equations when constructing tensors. architectureencodingforwardinversegenerationtraining-inference
What it supports. The architecture supports the One Model judgment because the action policy and auxiliary heads use the same language-model representation. The frozen spatial encoder supplies both input features and prediction targets. Auxiliary learning can therefore change the acting representation even when the prediction heads are absent from the default inference computation.
Where the evidence stops. Do not read the scene illustrations as generated images or explicit 3D reconstruction outputs. Predictions are pooled spatial latents. The inverse action/frame indexing differs between the diagram and text and needs an explicit implementation convention.
2. Motivation
2.1 The problem and the proposed response
Continuous vision-and-language navigation must connect instructions and egocentric RGB history to low-level movements while retaining spatial progress. The authors argue that action imitation weakly supervises how actions change the environment. FutureNav adds spatial-state and transition supervision to the action-decoding representation. problemarchitecture
2.2 What this reading follows
Consider a navigation instruction that requires a turn at a landmark and a precise stop. Correctly recognizing the landmark is only part of the task: the agent must also retain where it has moved and connect movement to changing observations. FutureNav adds this supervision to a vision-language action policy in two places. Frozen spatial features enrich the visual input, and three auxiliary heads learn from adjacent observations during training. Follow the architecture first, then separate the two data regimes in the main table. The ablations reveal which training signals help; the training curve and physical examples require narrower interpretations than a general speed or reliability claim. problemarchitectureencodingtraining-inferencemain-resultsobjective-ablationtraining-progressqualitative
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
One Model is supported architecturally: action decoding and auxiliary heads share a language backbone and fused tokens. Auxiliary inverse dynamics is not the default inference action-extraction mechanism, supporting Other mechanisms/Outside quadrants. Navigation and latent prediction are explicit; the combined latent-prediction/JEPA tag is supported on its latent-prediction side, without establishing a specific JEPA implementation. architectureencodingforwardinversegenerationtraining-inference
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Inject geometry before choosing an action
Start with the two routes from an RGB frame into the shared backbone. The VLM vision tower provides visual embeddings, while frozen VGGT extracts spatial patch features using the current image and cached history. The trainable merger groups patches, normalizes them and maps them into the VLM hidden space; interpolation handles token-count mismatch. Residual addition with weight 0.2 then puts spatial information into the same token positions consumed by the language model. This mechanism matters at inference as well as training: disabling the auxiliary heads does not remove spatial input encoding. Reader interpretation: the policy can benefit from geometry without asking a separate planner to rank imagined trajectories, but this architectural convenience alone cannot establish equal total inference cost to a policy without VGGT. architectureencodingsetuptraining-inference
5.2 Distinguish three questions about an adjacent transition
The auxiliary heads ask different questions about nearby observations. Forward dynamics receives the current observation hidden state and a teacher-forced action hidden state, then predicts the pooled next spatial target. Inverse dynamics observes an adjacent pair through mean-pooled LLM visual states, their difference and pre-response context, and classifies the intervening action. Future generation receives context and current visual state without the explicit action-token input, predicting the same next target. The full loss adds these three objectives to action-token cross entropy, each with weight 0.1. This is why the generation head should not be described as an action-conditioned rollout model. The inverse notation also requires care: the prose specifies a previous-action label while writing a_t, and the architecture diagram shifts the pair's frame indices. policyforwardinversegenerationtraining-inferencesetuparchitecture
5.3 Use the ablations to narrow the explanation
The headline table establishes improved navigation under the reported comparisons, but the component table is more informative about the proposed mechanism. Within expert-only R2R-CE training, forward dynamics produces the largest individual auxiliary gain, and all three losses produce the strongest combined score. The target-representation table then asks a different question: what should the prediction heads learn to predict? Spatial features outperform VAE, VLM visual and DINO alternatives. Reader interpretation: these two tables jointly favor transition supervision with spatial targets, but they do not prove accurate counterfactual dynamics. Missing pairwise ablations, target-matching details and seed variation leave alternative explanations open. Finally, the default inference branch uses the trained representation for direct action decoding; improved navigation is not evidence that explicit future rollouts were used during evaluation. main-resultsobjective-ablationlatent-ablationtraining-inferencereproduction-gaps
5.4 Training and inference
During training
The language model, merger and heads are optimized while both encoders remain frozen. Qwen3-VL-4B and 8B are evaluated. Expert-only training uses approximately 2.43M samples: 631K R2R-CE and 1.80M RxR-CE. The full recipe adds ScaleVLN and DAgger rollouts; “0K” means zero external data, not zero navigation training. training-inferencesetup
During inference
The policy processes the instruction and RGB history and decodes an action. Auxiliary heads are disabled in the main setting; spatial input encoding remains active. Physical deployment streams D435i RGB from a Go2 to an H20 GPU server and executes the returned action. architecturetraining-inferencedeployment
Removing auxiliary heads explains the absence of their inference work, but no timing benchmark establishes total latency parity with a VLM lacking spatial encoding. Optional look-ahead is mentioned without an evaluated planning protocol. problemtraining-inference
5.5 Implementation flow
- Encode and align spatial information
The frozen VLM vision tower and frozen VGGT process observations. VGGT uses the current frame with cached history. Spatial patches undergo grouping, RMS normalization and a two-layer merger; token-count mismatches are linearly interpolated before residual fusion. Figure 2 labels an eight-frame history window. architectureencoding
- Learn the action interface
A chat-style sequence places fused embeddings at image placeholders alongside instruction tokens. A tied vocabulary head autoregressively decodes action text; masking system/user labels with -100 leaves assistant action tokens under the policy loss. policy
- Predict action-conditioned transitions
The forward MLP takes the last image-token hidden state and the last teacher-forced ground-truth action-token hidden state. Its target is the pooled next-frame VGGT spatial feature, rather than a rendered image. forward
- Recover the intervening action
Inverse dynamics combines pre-response context, mean-pooled previous/current image hidden states and their difference. Cross entropy supervises the action producing that transition. The source calls this the previous-action label but writes a_t; its indexing needs care. inverse
- Predict an action-free future
The generation MLP combines pre-response context with the mean-pooled current image hidden state and regresses the same next spatial target. Excluding the explicit action-token input distinguishes it from forward dynamics. generation
6. Experiments & results
FutureNav teaches a shared navigation VLM to predict actions and understand adjacent spatial transitions through three auxiliary tasks. Frozen VGGT features enrich its input and provide future-state targets. Default inference decodes actions without running the auxiliary heads. Simulated navigation and objective ablations provide quantitative evidence; physical transfer is illustrated qualitatively. Evidence: architecture, encoding, training-inference, main-results, objective-ablation and qualitative.
6.1 Read the original evidence
Table 1. Compare FutureNav within each data regime before comparing headline scores. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the far-right external-training-data column. Dagger rows use expert VLN-CE trajectories only; 0K does not mean untrained. Then check model size and observation columns before reading R2R and RxR metrics. NE is navigation error and is lower-is-better; OS is oracle success, SR success rate, SPL success weighted by path length, and nDTW normalized dynamic time warping. The source's notes above the table are retained because they define the dagger, starred baselines and relative-gain rows. For the full-data R2R comparison, align FutureNav-4B with JanusVLN rather than its dagger variant. The purple delta rows report relative percentages. setupmain-results
What it supports. FutureNav-4B reaches R2R SR 65.4 and SPL 61.3, versus JanusVLN's 60.5 and 56.8. Its 4.9-percentage-point SR improvement corresponds to 8.1% relative gain. On RxR, FutureNav-8B reaches SR 63.9 and SPL 54.8. These are validation-unseen simulation results under the listed training budgets.
Where the evidence stops. Backbones, pretraining, sensor access and data recipes differ across rows; external-data counts alone do not control them. The table reports no seed uncertainty. These simulated success rates are not physical-robot success rates.
Figure 6. Physical navigation examples retain the paper's success labels; the bottom row's stated destination is unresolved. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each strip left to right, comparing its instruction with the visible route and endpoint. The top strip names a yellow roadblock and coffee-shop entrance. The middle names a chair and vending machine, which is visible in its final frame. The bottom repeats that instruction but ends near a counter/cabinet; no vending machine is identifiable there. All three carry source-provided success badges, so the bottom badge cannot independently establish destination adherence. Section 4.1 describes a Go2 streaming D435i RGB to an H20 GPU server and executing returned actions. Section 4.4 reports deployment without real-world navigation fine-tuning. These are external views of physical demonstrations, not auxiliary-head predictions. qualitativedeploymentfigure-6-mismatch
What it supports. Together with the deployment description, the images document physical execution by a navigation policy that the authors report deploying without real-world fine-tuning. The middle endpoint visibly matches its vending-machine instruction. The bottom sequence documents robot movement, but its repeated instruction and ambiguous endpoint prevent verifying that it reached the specified destination.
Where the evidence stops. The source does not explain the bottom row's instruction/endpoint mismatch; preserve it as unresolved rather than infer a corrected instruction or successful arrival. Success badges also provide no trial denominator. Failure distribution, repeated trials, communication latency and action durations remain unreported.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| R2R-CE navigation, full-data 4B Habitat/Matterport3D val-unseen; 1,839 trajectories; single RGB; 10.457M external samples | 65.4% SR; 61.3 SPL; 4.24 NE SR / SPL (higher better); NE (lower better) | JanusVLN-7B: 60.5% SR, 56.8 SPL, 4.78 NE; 10.692M external samples. SR improves 4.9 percentage points, equivalent to the reported 8.1% relative gain. Similar data volume does not establish identical recipes. setupmain-results |
| RxR-CE navigation, full-data 8B Val-unseen; 3,669 trajectories; single RGB; 10.435M external samples | 63.9% SR; 54.8 SPL; 69.1 nDTW; 4.26 NE SR / SPL / nDTW (higher better); NE (lower better) | JanusVLN-7B: 56.2% SR, 47.5 SPL, 62.1 nDTW, 6.06 NE. SR improves 7.7 percentage points. The 8B model leads this split, while 4B leads R2R in the reported full-data comparison. setupmain-results |
| Expert-only navigation, 4B R2R-CE and RxR-CE val-unseen; dagger/0K regime | R2R: 55.1% / 50.1; RxR: 54.5% / 46.0 SR / SPL | JanusVLN dagger: R2R 52.8% / 49.2; RxR 51.4% / 44.3. SR gains are 2.3 and 3.1 percentage points without external navigation data; backbone/pretraining differences remain. setupmain-results |
| World-action objective ablation R2R-CE val-unseen; 0K | All objectives: 55.1% / 50.1 SR / SPL | Policy only: 50.1% / 45.1; inverse: 50.8% / 45.8; forward: 53.0% / 48.3; generation: 51.6% / 46.9. All auxiliaries add 5.0 points on both metrics; forward gives the largest single-objective gain. Pairwise combinations and seed uncertainty are absent. objective-ablation |
| Latent-target ablation R2R-CE val-unseen; no external data | Spatial latent: 55.1% / 50.1 SR / SPL | VAE: 51.0% / 46.4; VLM visual: 51.4% / 46.8; DINO: 51.8% / 47.1. Spatial targets lead the reported alternatives; incomplete representation and normalization details limit causal attribution. latent-ablation |
| Navigation accuracy during training R2R-CE val-unseen; matched reported training schedule | At about 42%: 42.3% / 37.8; at about 63%: SR 50.2%. SR / SPL versus schedule progress | Policy-only at about 42%: 33.2% / 27.9; final policy-only SR is 50.1%. The plotted 1.58x compares schedule progress, not measured wall-clock acceleration. Axis/caption terminology conflicts. training-progressobjective-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Forward dynamics gives the largest individual gain; all auxiliary objectives perform best. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the checkmarks before the scores. Every row retains policy training. The next three rows activate inverse dynamics, forward dynamics or future generation individually; the highlighted final row activates all three. This is the R2R-CE validation-unseen 0K comparison, so changes should be interpreted within expert-only training. Compare each single-objective row against the first row to identify its incremental benefit, then compare the full combination with the strongest individual row. SR and SPL are the primary metrics; NE decreases when navigation error improves. The table has no pairwise-objective rows, which limits what can be inferred about interactions. objective-ablationsetupforwardinversegeneration
What it supports. Policy-only training gives SR/SPL 50.1/45.1. Forward dynamics raises them to 53.0/48.3, larger than the individual inverse or generation gains. All three auxiliaries reach 55.1/50.1 and reduce NE from 6.00 to 5.13. This supports auxiliary supervision as a useful addition under the reported regime.
Where the evidence stops. The best combined row does not establish that every auxiliary is necessary once the others are present. Missing pairwise combinations and repeated-seed variation leave those marginal contributions unresolved.
Table 3. The prediction target matters: spatial latents lead the listed alternatives. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left column as a choice of representation for world modeling. The first row removes latent modeling; subsequent rows use VAE, VLM visual, DINO semantic or spatial features. All scores concern R2R-CE validation-unseen without external data. First compare each representation with the no-modeling baseline, then compare spatial features with the strongest alternative, DINO. Section 3.2 identifies the proposed spatial encoder as frozen VGGT; Table 3 does not fully specify corresponding encoder versions or normalization for the alternatives. The scores measure downstream navigation, rather than reconstruction quality or the accuracy of the predicted future features themselves. latent-ablationencodingsetup
What it supports. Spatial latents achieve SR/SPL 55.1/50.1, compared with DINO's 51.8/47.1 and VLM visual latents' 51.4/46.8. All listed latent alternatives exceed the no-modeling baseline. The result favors geometry-aware supervision in this implementation, with a 3.3-point SR advantage over the strongest listed alternative.
Where the evidence stops. Different targets may also differ in scale, capacity and preprocessing. The paper does not document enough matching details to attribute the entire advantage solely to spatial content or to claim a controlled comparison of representation families.
Figure 4. FutureNav reaches the baseline's final accuracy earlier in the reported schedule. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the left panel for SR and the right panel for SPL. No color legend appears in the source graphic; the caption and Section 4.3 identify the higher purple/red curves with FutureNav and the lower blue/orange curves with the policy-only baseline. The horizontal arrows connect roughly 63.2% progress for FutureNav with the baseline's full schedule. Preserve the wording discrepancy: both axes say Training Data (%), while the caption describes completed training steps and the prose describes a shared schedule. The defensible reading is earlier attainment of comparable navigation accuracy under that reported schedule, without asserting independently reduced dataset size. training-progressobjective-ablationtraining-inferencereproduction-gaps
What it supports. The prose reports SR/SPL 42.3/37.8 for FutureNav at about 42% progress, versus 33.2/27.9 for policy-only training. At about 63%, FutureNav reaches SR 50.2, approximately the baseline's final 50.1. The plot's 1.58x label summarizes this progress comparison rather than a timed hardware measurement.
Where the evidence stops. The source supplies neither a color legend nor an absolute step/time schedule. Auxiliary training changes computation per step, so the 1.58x annotation cannot establish wall-clock speedup or lower total training cost.
7. Analysis & limitations
7.1 What the evidence leaves open
Results lack confidence intervals or repeated-seed variability. Ablations support useful auxiliary supervision but do not measure counterfactual dynamics accuracy or long-horizon rollout fidelity. The source provides no dedicated limitations section. objective-ablationlatent-ablationreproduction-gaps
Real-world evidence comprises selected sequences labeled successful, without a trial denominator or aggregate success rate. Figure 6's bottom strip repeats the middle strip's vending-machine instruction, but its endpoint resembles a counter/cabinet and is not identifiable as a vending machine. Physical execution is documented; that strip's destination adherence remains unresolved. qualitativedeploymentfigure-6-mismatch
Figure 2 labels prediction inputs as spatial features and its inverse pair as t,t+1; Eqs. (4), (7) and (9) use specific LLM hidden states and a previous/current inverse pair. Follow the equations for implementation and resolve action/frame indexing explicitly. architectureforwardinversegeneration
7.2 Questions for discussion
- Does forward supervision encode counterfactual action effects, or mainly improve representations along expert trajectories?
- Would spatial targets retain their advantage with matched scaling, encoder capacity and repeated seeds?
- How much robot latency comes from spatial encoding, communication and action execution?
8. Reproducibility audit
8.1 Requirements and known gaps
Required components include Qwen3-VL, VGGT, Habitat/Matterport3D, expert trajectories and, for full-data comparisons, ScaleVLN/DAgger data. Reconstruct assistant-token masks, image-token grouping, target pooling and frozen modules before comparing Table 2 variants. policyencodingsetupobjective-ablation
Missing optimizer, learning rate, batch size, absolute schedule, training hardware/time, software versions, exact spatial layer/cache settings and alternative-target normalization prevent exact recipe reconstruction. Motion magnitudes/durations are also unstated. The title page promises code/models without establishing release status. reproduction-gapslatent-ablationdeploymentidentity
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether forward supervision depends on the correct transition
Reader-proposed check, not an executed experiment: under the 0K R2R-CE setup, compare policy-only, policy plus correct forward targets, and policy plus next-spatial targets shuffled within action classes. Freeze the same encoders, retain identical spatial input fusion, and match seeds, batches, optimizer, loss weight and training steps. Evaluate held-out SR/SPL and aligned next-target error. Correct targets should outperform shuffled targets if learning the actual transition drives the gain. Similar navigation gains from shuffled targets would weaken that explanation. Record the exact frame/action alignment first, and treat this as a diagnostic beyond the published ablation. forwardencodingsetupobjective-ablationinversereproduction-gaps
Check 2: Measure what policy-only inference actually saves
Reader-proposed check, not an executed experiment: on a fixed H20 server and identical RGB histories, benchmark the same FutureNav checkpoint with auxiliary heads disabled and enabled, while action decoding remains identical. Report warmed-up median and tail latency, peak memory, spatial-encoder time and action-token decoding time. Add a separately identified policy without spatial fusion to measure the input-encoder contribution; record its accuracy rather than assuming equivalence. The first pair tests the cost avoided by disabling heads. A measurable spatial-encoding overhead in the second comparison would narrow the broad no-additional-inference-cost claim even if default navigation remains faster than explicit look-ahead. training-inferencearchitectureencodingdeploymentreproduction-gaps
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/version on p. 1; overview on p. 2; introductory training/inference claims on p. 3; architecture and input formulation on p. 4; spatial encoding, policy, forward/inverse equations on p. 5; generation, combined loss, inference diagram, benchmark setup, training settings and robot hardware on p. 6; full quantitative table on p. 7; both ablation tables and training curves on p. 8; simulated and physical examples plus training-progress continuation on p. 9; and qualitative analysis/conclusion on p. 10. All six final crops were viewed at their native rendered resolution. The two narrow ablation tables were rendered at 400 DPI; other crops use 200 DPI. Table 1 retains its source notes because they define comparison regimes. Figure 2's schematic head inputs/frame indexing and Figure 4's absent color legend and axis/caption discrepancy are disclosed in the guides. Figure 6's repeated bottom-row vending-machine instruction, cabinet-like endpoint and source-provided success badge were inspected on p. 9 and in the final crop; the unexplained mismatch with the caption and p. 10 prose is preserved explicitly. All method/numerical/reproduction evidence pages are included above. References on pp. 11–14 were read in all five complete text chunks but were not visually inspected; no appendix or separate supplement was supplied.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title metadata (p. 1)
- 1 Introduction (pp. 1–3)
- 2 Related Work, including 2.1–2.2 (p. 3)
- 3 Methodology, including 3.1–3.4 and Eqs. (1)–(11) (pp. 4–6)
- 4 Experiments, including 4.1–4.4 (pp. 6–10)
- 5 Conclusion (p. 10)
- References [1]–[79] (pp. 10–14)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This extraction limitation was addressed by inspecting PDF pages 1–10 and all six final crops.
- Separate supplemental material availability has not been fully verified.
- No separate supplement was supplied. No appendix appears in the 14-page PDF.
- Identity note: the exact title and all ten authors match. The PDF margin identifies arXiv:2606.30367v1 dated 29 June 2026; its title-block date is 30 June 2026. The catalog submission date is 29 June. No later revision was supplied or substituted.
- Code, project pages and cited works were not opened; experiments were not reproduced. References on pp. 11–14 were read as text without page-image inspection.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author/affiliation block, date and arXiv margin stamp
The title and ten authors match the catalog. The artifact is stamped arXiv:2606.30367v1, 29 Jun 2026; its title block gives June 30, 2026. The abstract promises future code/model release.
Go to primary source ↓problemPDF pp. 1–3, Abstract and Section 1
Continuous VLN motivates instruction grounding, history, spatial understanding and executable actions. The introduction describes default policy-only inference and optional auxiliary world-state/look-ahead use without an evaluation protocol for the latter.
Go to primary source ↓architecturePDF p. 4, Figure 2 and Sections 3.1–3.2
A shared VLM receives instructions and spatially augmented RGB tokens. Figure 2 marks action Train & Infer and the other branches Train Only; its history label states window size 8. Diagram inputs are schematic; implementation is detailed on pp. 5–6.
Go to primary source ↓encodingPDF p. 5, Section 3.2, Eqs. (1)–(2)
Frozen VGGT supplies intermediate-layer spatial patches of width 2048 using current image and cached history. Grouping, RMS normalization, a two-layer MLP and optional token-axis interpolation precede residual fusion.
Go to primary source ↓policyPDF pp. 4–5, Sections 3.1–3.3, Eq. (3)
The policy autoregressively generates four discrete action names as assistant text. System/user labels are masked with -100; cross entropy supervises assistant action tokens.
Go to primary source ↓forwardPDF p. 5, Section 3.3, Eqs. (4)–(5)
The forward MLP concatenates the last image-token hidden state and last unmasked teacher-forced action-token hidden state, regressing a pooled next-observation spatial feature of width 2048.
Go to primary source ↓inversePDF p. 5, Section 3.3, Eqs. (6)–(8); PDF p. 4, Figure 2 inverse branch
The inverse classifier uses pre-response context, mean-pooled previous/current image-token groups and their difference. The previous-action label is written a_t. Text frames are t-1,t, while the figure labels an adjacent pair t,t+1.
Go to primary source ↓generationPDF p. 6, Section 3.3, Eqs. (9)–(10)
The generation MLP uses pre-response context and mean-pooled current image state, regressing the same next spatial target without the teacher-forced action-token input.
Go to primary source ↓training-inferencePDF p. 6, Figure 3, Section 3.4, Eq. (11), and Section 4.1 Implementation Details
Language components, merger and heads are optimized; both encoders remain frozen. Default inference omits auxiliary heads. The source provides no measured latency/throughput comparison.
Go to primary source ↓setupPDF p. 6, Section 4.1 Dataset and Evaluation Metrics; Implementation Details
Habitat/Matterport3D validation-unseen evaluation uses 1,839 R2R-CE and 3,669 RxR-CE trajectories. Metrics are navigation error (NE), oracle success (OS), success rate (SR), success weighted by path length (SPL), and RxR normalized dynamic time warping (nDTW). Qwen3-VL-4B/8B use alpha=0.2 and auxiliary weights 0.1. Expert-only data are approximately 2.43M samples; full data add ScaleVLN/DAgger.
Go to primary source ↓main-resultsPDF p. 7, Table 1, FutureNav-4B/8B, dagger variants and JanusVLN rows; caption and Section 4.2
Full-data 4B gives R2R SR/SPL 65.4/61.3 using 10457K external samples; 8B gives RxR 63.9/54.8 using 10435K. Dagger 4B gives R2R 55.1/50.1 and RxR 54.5/46.0. JanusVLN full-data SR/SPL are R2R 60.5/56.8 and RxR 56.2/47.5, with 10692K external samples. Delta rows are relative gains.
Go to primary source ↓objective-ablationPDF p. 8, Table 2 and Section 4.3 Effect of World-Action Objectives
R2R-CE 0K policy-only SR/SPL are 50.1/45.1; inverse gives 50.8/45.8, forward 53.0/48.3, generation 51.6/46.9, and all objectives 55.1/50.1. All-objective NE is 5.13 versus 6.00. Pairwise combinations and uncertainty estimates are absent.
Go to primary source ↓latent-ablationPDF p. 8, Table 3 and Section 4.3 Effect of Spatial Latent Representation
Without external data, SR/SPL are VAE 51.0/46.4, VLM visual 51.4/46.8, DINO semantic 51.8/47.1, spatial 55.1/50.1, and no latent modeling 50.1/45.1. Alternative encoder versions, target normalization and fully matched compute are unspecified.
Go to primary source ↓training-progressPDF p. 8, Figure 4 and Section 4.3 Training Efficiency; PDF p. 9 bottom and p. 10 opening continuation
The axis says Training Data (%), whereas caption/prose describe completed training steps. No color legend is supplied. Prose gives about 42% progress SR/SPL 42.3/37.8 versus 33.2/27.9; at about 63%, FutureNav SR is 50.2, matching final policy-only accuracy. The plot marks 1.58x.
Go to primary source ↓qualitativePDF p. 9, Figures 5–6 and captions; PDF p. 10, Section 4.4
Figure 5 contrasts selected simulation trajectories with JanusVLN. Figure 6 presents three physical sequences with source-provided success badges; its caption and Section 4.4 claim destination adherence and indoor/outdoor zero-shot transfer without real-world fine-tuning. The bottom strip's destination is visually unresolved. A trial denominator, aggregate real-world success rate and failure distribution are absent.
Go to primary source ↓figure-6-mismatchPDF p. 9, Figure 6, middle and bottom instruction strips and final frames; Figure 6 caption; PDF p. 10, Section 4.4 Real-world Qualitative Analysis
The middle and bottom strips print the same instruction ending at a vending machine. A vending machine is visible in the middle strip's final frame. The bottom endpoint resembles a counter/cabinet and is not identifiable as a vending machine, despite its success badge. The caption and prose claim arrival at specified destinations but do not explain this mismatch; the intended bottom-row destination cannot be established from the supplied source.
Go to primary source ↓deploymentPDF p. 6, Section 4.1 Real-world Deployment
A Go2 with D435i camera streams egocentric RGB to an H20 GPU inference server; returned low-level actions are physically executed. Action duration, motion magnitudes and communication latency are unspecified.
Go to primary source ↓reproduction-gapsPDF pp. 4–6, Sections 3.1–3.4 and 4.1; PDF pp. 7–10, Sections 4.2–4.4
Descriptions omit optimizer, learning rate, batch size, absolute training schedule, training hardware/time, software versions, precise VGGT layer/cache settings and random-seed uncertainty. The conclusion is followed by references, without a limitations section or appendix.
Go to primary source ↓8.5 Primary sources
FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation ↗
PDF · 8,185 extracted words
Source fingerprint
008de466ef776348f9e78ecf939b03b56a66d8d59c38b32c62429623a1146b2b