Scaling Data Generation in Vision-and-Language Navigation
1. Paper overview
In one sentence: ScaleVLN improves navigation by generating diverse training routes with traversable graphs, repaired observations and synthetic instructions, but the benefit depends on data quality and how training uses it. e-pipelinee-pairse-graph-teste-sizee-imagee-mixturee-r2re-missing
| At a glance | What to know |
|---|---|
| Research problem | Source description VLN agents must ground language in observations while choosing routes through unfamiliar buildings. Limited training environments encourage scene-specific behavior. ScaleVLN asks how to generate diverse, usable supervision automatically and retain its benefit during downstream fine-tuning. e-probleme-agents |
| Core mechanism | Source description The pipeline uses 800 HM3D training scenes and 491 Gibson training/validation scenes to create 4,941,710 R2R-style instruction–trajectory pairs through graph construction, image repair and an existing instruction speaker. e-scenese-pairse-recovery |
| A key reported result | R2R single-run navigation with EnvEdit and CLIP ViT-H/14: 80%; 70%; 2.27 m SR; SPL; NE. R2R test-unseen; starred DUET + ScaleVLN in Table 9 BEVBert: 73% SR, 62% SPL; original DUET: 69% SR, 59% SPL. The gain is 7 percentage points SR over BEVBert and 11 over DUET. The abstract’s +11-point previous-SoTA wording does not apply to BEVBert. Extra visual components prevent attributing the headline entirely to data. e-r2re-problem |
| Reading caution | Reader analysis Graph comparisons change density and collisions together, so they do not isolate collision removal. At fixed 800 scenes, SR is not monotonic with sample count despite the broad scaling interpretation. The tables supply no seed-level uncertainty. e-graph-teste-size |
Core contributions
- Source description
The pipeline uses 800 HM3D training scenes and 491 Gibson training/validation scenes to create 4,941,710 R2R-style instruction–trajectory pairs through graph construction, image repair and an existing instruction speaker. e-scenese-pairse-recovery
Figure 2. Scans become supervision through graph construction, visual repair and instruction generation. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the large arrows from the scans at left to the top-down navigation graph. Above that graph, the rendered-image and recovered-image panels show the Co-Mod GAN repair step. The path-sampling panel supplies a sequence of observations to the Speaker; its speech bubble represents generated training language. The pink dataset block then feeds pre-training and fine-tuning, with downstream task examples grouped at right. These arrows depict offline data preparation and agent training. They do not show a runtime loop that imagines future observations before selecting actions. The caption and Section 3.2 support this information flow. e-pipelinee-recoverye-pairse-agentse-missing
What it supports. The figure explains how unannotated scans become paired visual-language supervision. Its central product is a dataset, subsequently used by DUET or HAMT. Repairing observations and generating instructions are separate components, so the pipeline does not establish a single learned model that jointly predicts future world states and actions.
Where the evidence stops. The shared diagram includes REVERIE, but Section 3 specifies that the described route/instruction procedure is R2R-style. REVERIE-specific object-directed collection and training are deferred to an appendix absent from the supplied artifact.
2. Motivation
2.1 The problem and the proposed response
VLN agents must ground language in observations while choosing routes through unfamiliar buildings. Limited training environments encourage scene-specific behavior. ScaleVLN asks how to generate diverse, usable supervision automatically and retain its benefit during downstream fine-tuning. e-probleme-agents
2.2 What this reading follows
An indoor scan is not yet a useful navigation demonstration. It must support plausible routes, observations that a visual encoder can interpret, and language aligned with those routes. ScaleVLN assembles these ingredients into nearly five million training pairs and then studies how existing followers should learn from them. Read the pipeline first, then use the ablations to distinguish environment diversity from sample volume and image quality. Finally, separate the standard R2R result from the enhanced configuration behind the 80% headline. The supplied main paper supports this analysis, while its referenced appendix and full implementation details remain unavailable. e-pipelinee-pairse-graph-teste-sizee-imagee-mixturee-r2re-missing
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Datasets |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The dataset/synthetic-navigation classification fits the central contribution: generated language–trajectory data trains existing followers. Architecture, prediction-paradigm and quadrant labels are appropriately not applicable to the data resource. Image repair and auxiliary pre-training objectives imply neither a single world-and-action predictor nor inverse-dynamics control. e-pipelinee-agentse-training
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
5. Method in detail
5.1 Build physically meaningful routes before generating language
The route graph defines which observation sequences can become demonstrations. Section 3.2 first samples many navigable positions, merges neighboring positions with agglomerative clustering, and creates a rough graph before applying a cited refinement procedure. The separation and clustering thresholds control how densely the open space is represented; refinement is needed because connecting nearby points is not by itself a guarantee of traversability. Shortest routes with three to five intermediate nodes then supply the path samples. Table 1 makes the practical issue visible: AutoVLN’s added data barely changes unseen success relative to the no-HM3D baseline, whereas the proposed graphs improve it. That comparison supports the overall graph construction choice, but it changes both density and obstacle-crossing edges, leaving their individual causal contributions unresolved. e-graphse-pairse-graph-test
5.2 Treat repaired images and generated instructions as coupled supervision
A path is paired with language only after the environment can provide usable observations. Co-Mod GAN learns image recovery from Matterport3D render/camera pairs, and EnvDrop Speaker supplies instructions from the sampled routes. Table 4 shows why visual repair belongs in the data pipeline: adding faulty Gibson images is less useful than adding recovered ones. Table 5 then tests a different bottleneck, comparing an LSTM speaker with GPT-2. The LSTM has higher BLEU-4 and slightly better downstream unseen success in this experiment. Reader interpretation: greater language-model capacity alone does not guarantee more useful navigation supervision. However, these results do not certify the faithfulness of every generated instruction or repaired landmark. The authors themselves identify instruction quality as an open avenue for improvement. e-recoverye-pairse-imagee-speaker
5.3 Preserve diverse supervision through the training transition
The paper separates pre-training from imitation-learning fine-tuning because the two stages need not benefit from identical data mixtures. Its final recommendation pre-trains with MLM and SAP on R2R, PREV and the combined HM3D/Gibson ScaleVLN data, then fine-tunes on R2R plus HM-E. Table 6 supports retaining HM-E at the second stage; adding PREV there reduces unseen performance relative to HM-E alone. Table 7 reports complementary MLM/SAP benefits and a weaker result after adding MRM, although its reference to the Table 6 configuration is internally inconsistent. At deployment, the trained follower uses observations and its map or history to choose actions. The offline repair GAN and speaker do not become a world-model planning loop, and continuous navigation additionally relies on a waypoint predictor. e-traininge-mixturee-proxye-agentse-pipelinee-ce
5.4 Training and inference
During training
The recommended recipe pre-trains masked language modeling (MLM) and single-action prediction (SAP) on R2R, PREV and ScaleVLN, then fine-tunes on R2R plus HM-E. Masked region modeling (MRM) is tested but not retained. DAGGER fine-tuning samples policy actions and supervises against ground truth; the body supplies no explicit loss equation. e-traininge-mixturee-proxy
Default features are CLIP ViT-B/16. DUET pre-training uses 20,000 iterations, batch 256, learning rate 5 × 10⁻⁵ and two NVIDIA Tesla A100 GPUs, reported as about 72 GPU-hours. R2R fine-tuning uses 200,000 iterations, batch 16 and one GPU, with about 48 GPU-hours to peak performance. CVDN fine-tunes the pre-trained model on its own dataset. e-training
During inference
The follower grounds language in current and accumulated observations to select navigation actions. Neither the speaker nor the repair GAN is described as an inference-time planning loop. For R2R-CE, a candidate waypoint predictor supplies navigable locations to HAMT’s high-level decisions. e-pipelinee-agentse-ce
R2R runs avoid pre-exploration, beam search and ensembling. Success rate (SR) requires stopping within 3 m of the goal; navigation error (NE) measures final distance and SPL penalizes inefficient paths. These metrics assess executed simulator navigation, not language fluency. e-probleme-metricse-r2r
5.5 Implementation flow
- Discretize traversable space
Sample Habitat positions with pairwise geodesic separation greater than 0.4 m. Agglomerative clustering at 1.0 m merges nearby viewpoints. Randomly connect positions within 5.0 m with an initial maximum degree of five, then apply a cited graph-refinement procedure. Final mean edge length is 1.41 m and mean degree 4.55; the full refinement algorithm is not restated. e-scenese-graphs
- Recover observations
Train Co-Mod GAN on Matterport3D rendered/camera-image pairs, then repair faulty HM3D/Gibson renders. This is conditional image recovery for training data; no action-conditioned future-state prediction is introduced. e-recoverye-pipeline
- Pair sampled routes with language
Collect shortest routes with three to five intermediate nodes: 2,890,267 HM3D paths and 2,051,443 Gibson paths. The off-the-shelf LSTM EnvDrop Speaker, trained on R2R, generates one instruction per path. R2R-style pairs support R2R, CVDN and R2R-CE; REVERIE-specific generation is deferred to the missing appendix. e-pairse-missing
- Train an existing follower
DUET builds a topological map online, extending choices to navigable directions previously encountered. HAMT stores past observations for vision–instruction alignment. The study uses DUET for R2R/CVDN/REVERIE and HAMT for R2R-CE; it does not specify a new unified controller. e-agents
6. Experiments & results
ScaleVLN turns scanned indoor environments into synthetic instruction–trajectory supervision for existing navigation agents. Graph connectivity, image recovery and training mixtures matter alongside scale. Standard DUET with ScaleVLN reaches 77% R2R test-unseen success; the headline 80% additionally uses EnvEdit and CLIP ViT-H/14. These are simulator navigation results, including continuous simulation.
6.1 Read the original evidence
Table 9. Read the star footnote before interpreting the 80% success headline. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Focus first on the Test-Unseen SR and SPL columns at the far right. Compare original DUET with the unstarred DUET+ScaleVLN row, then compare the unstarred and starred rows. The retained footnote identifies the star as EnvEdit image augmentation plus CLIP ViT-H14 features; the dagger symbols describe pre-training and additional visual data. NE and TL are distances, while the reported SR/SPL values use percentage-style scores. Keep the validation groups separate from the test group. This ordering prevents a change in both training data and visual representation from being mistaken for an isolated data effect. e-r2re-metricse-humane-problem
What it supports. Standard ScaleVLN reaches 77% SR and 68% SPL versus DUET’s 69% and 59%. The enhanced configuration reaches 80% and 70%, exceeding BEVBert’s 73% and 62%. The table therefore supports a seven-point SR gain over BEVBert for the headline result, while eleven points is the difference from DUET.
Where the evidence stops. Historical rows do not control architecture, compute or all data sources. The human row also uses egocentric observations rather than the model’s panoramas. Neither the headline nor the reduced validation gap establishes human-equivalent real-world navigation.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| R2R single-run navigation with EnvEdit and CLIP ViT-H/14 R2R test-unseen; starred DUET + ScaleVLN in Table 9 | 80%; 70%; 2.27 m SR; SPL; NE | BEVBert: 73% SR, 62% SPL; original DUET: 69% SR, 59% SPL. The gain is 7 percentage points SR over BEVBert and 11 over DUET. The abstract’s +11-point previous-SoTA wording does not apply to BEVBert. Extra visual components prevent attributing the headline entirely to data. e-r2re-problem |
| R2R single-run navigation with default visual features R2R test-unseen; unstarred DUET + ScaleVLN | 77%; 68% SR; SPL | DUET: 69%; 59%. Displayed rows imply +8 points SR and +9 SPL; Section 4.4 describes both gains as eight. This is not a compute-matched architecture comparison. e-r2r |
| REVERIE navigation Test-unseen; DUET + ScaleVLN; navigation metrics only | 56.13%; 39.52% SR; SPL | AutoVLN: 55.17%; 38.88%. Table arithmetic gives +0.96 points SR and +0.64 SPL; prose states +0.94 SR. Object-localization success is absent from this table, so navigation results do not establish full remote-object grounding. e-reveriee-tasks |
| CVDN dialogue-conditioned navigation Test-unseen; DUET pre-trained with R2R-style data and fine-tuned on CVDN | 6.97 m Goal progress (GP), meters; higher is better | HAMT: 5.58 m. Displayed values imply +1.39 m; text reports +1.41 m and 25.26%. Retain the table values and discrepancy. The comparison changes the follower as well as data. e-cvdne-training |
| R2R-CE continuous navigation Test-unseen; HAMT + ScaleVLN with candidate waypoint predictor | 55%; 50%; 5.11 m SR; SPL; NE | VLN-BERT + Ego2-Map: 47% SR, 41% SPL, 5.54 m NE. Discrete supervision transfers to continuous simulation. Baselines differ in model/visual representation; physical robot deployment is not evaluated. e-ce |
| R2R fine-tuning mixture ablation Table 6, rows 4 and 6; same R2R/PREV/HM-E pre-training; add HM-E in fine-tuning | 76.59% → 79.10%; row 6 gap 0.92 percentage points Val-unseen SR; seen–unseen SR gap | Row 4 fine-tunes on R2R alone; row 6 has 80.02% val-seen SR. New environments during fine-tuning improve unseen SR by 2.51 points. Adding PREV too gives 77.10% in row 7, so more fine-tuning data is not uniformly beneficial. e-mixture |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 1. Better unseen navigation accompanies denser graphs with traversable edges. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the two HM3D graph columns: Density counts nodes per square meter, while Collision is the fraction of edges crossing obstacles. The latter is a dataset-graph property, not a measured collision rate during test navigation. Both augmented graph methods use the same 800 HM3D scenes without image recovery. Then move to the R2R Val-Unseen columns on the right, where lower NE and higher SR/SPL are preferred. The None row trains only with R2R and PREV. Comparing it with AutoVLN shows why adding graph-derived data is not automatically useful. e-graph-teste-graphs
What it supports. ScaleVLN’s graphs have density 1.16 versus 0.36 and zero reported obstacle-crossing edges versus 29.35%. Unseen SR rises from AutoVLN’s 72.75% to 76.01%; the no-HM3D baseline is 72.92%. Thus graph quality is consequential for the usefulness of additional scenes, rather than merely for the visual appearance of sampled paths.
Where the evidence stops. Density and obstacle-crossing rate change together, so this comparison cannot identify which graph property causes the gain. Zero graph-edge collisions also does not establish collision-free behavior in continuous simulation or on a physical robot.
Table 2. Environment diversity and sample volume are separate experimental axes. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the first two columns to construct two comparisons. Hold samples at 700k and compare 200, 400 and 800 scenes; this changes the variety of buildings while keeping the number of pairs fixed. Next hold scenes at 800 and compare 700k, 1400k and 2890k samples; this increases route coverage within the same environment set. The caption specifies pre-training on R2R, PREV and HM-E, followed by R2R-only fine-tuning. Read Val-Unseen SR and SPL separately, rather than treating every metric as the same measure of generalization. The zero-data row supplies the baseline. e-size
What it supports. At 700k pairs, unseen SR increases from 74.84% to 75.22% to 76.25% as scene count rises; SPL also increases. At 800 scenes, the largest sample set has the best unseen SR and SPL. These comparisons support acquiring varied environments as well as covering their routes more thoroughly.
Where the evidence stops. The sample-count SR sequence is 76.25%, 76.01%, 76.59%, so it is not monotonic despite the text’s gradual-increase wording. The table gives no seeds or uncertainty, and its tested ranges do not establish a universal scaling law.
Table 4. Additional scenes help most when their rendered observations are repaired. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The F/R key below the table is essential: F means faulty rendered images and R means recovered images. HM-E and Gib-E refer to HM3D and Gibson pairs whose instructions come from EnvDrop Speaker. First compare the two HM-E-only rows to see the recovery effect within one environment source. Then compare the final two rows: both retain recovered HM-E and add Gibson, but Gibson’s images switch from faulty to recovered. Read the unseen columns on the right for generalization; the bold values in the seen columns answer a different question. e-imagee-pairse-recovery
What it supports. Recovering HM-E changes unseen SR from 76.01% to 76.59%. Adding faulty Gibson then gives 76.33%, whereas recovering Gibson gives 78.08% and improves SPL from 66.97% to 68.31%. More environments can therefore disappoint when their rendered observations are poor; their quality mediates the value of their quantity.
Where the evidence stops. These are follower-performance measurements, not direct guarantees of pixel fidelity or preserved scene semantics. A repaired panorama can look plausible without proving every landmark is accurate; the table does not independently measure that risk.
Table 6. Retaining new environments during fine-tuning reduces the seen–unseen gap. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the checkmarks before the scores: the first group names pre-training data, and the second group names fine-tuning data. Rows 4 through 7 all pre-train on R2R, PREV and HM-E, making them the clearest comparison of fine-tuning mixtures. Row 4 uses R2R alone; row 5 adds PREV; row 6 instead adds HM-E; row 7 includes all three. The best unseen SR is in row 6, whereas the largest seen SR is in row 7. This difference illustrates why selecting a recipe solely from seen-environment performance can be misleading. e-mixturee-proxy
What it supports. Adding HM-E during fine-tuning raises unseen SR from 76.59% to 79.10%, a 2.51-point gain. Row 6’s 80.02% seen SR leaves a 0.92-point gap. Including PREV as well lowers unseen SR to 77.10%, showing that data useful for pre-training need not help in the same way during fine-tuning.
Where the evidence stops. Do not silently attach Table 7’s proxy ablation to row 6: its text names Method 6, but its MLM+SAP scores repeat row 4. The supplied artifact does not resolve that training-mixture discrepancy.
7. Analysis & limitations
7.1 What the evidence leaves open
Graph comparisons change density and collisions together, so they do not isolate collision removal. At fixed 800 scenes, SR is not monotonic with sample count despite the broad scaling interpretation. The tables supply no seed-level uncertainty. e-graph-teste-size
Only two speakers are compared. The authors identify generated instructions as low quality; EnvDrop nevertheless reaches 76.59% unseen SR versus GPT-2’s 75.82%. BLEU and follower success do not establish faithful grounding of every instruction. e-speaker
Table 7 says its proxy ablation uses Table 6 Method 6, but its MLM+SAP row repeats Method 4’s 76.59% SR/67.74% SPL. That mixture is unresolved. The human comparison also uses egocentric views versus model panoramas. e-proxye-mixturee-human
7.2 Questions for discussion
- Would graph improvements remain after matching density and route counts while varying obstacle-crossing edges?
- Does image recovery improve geometric grounding or partly help the visual encoder through appearance changes?
- Which mixture produced Table 7, and would its objective ranking persist under the final fine-tuning recipe?
8. Reproducibility audit
8.1 Requirements and known gaps
Rebuilding needs the specified scenes, Habitat sampling/refinement, Matterport3D paired images, Co-Mod GAN, EnvDrop Speaker, CLIP and DUET/HAMT. Graph thresholds and agent compute are given, but the body defers implementation details to an absent appendix. Optimizer details, mixture ratios, frozen-module policy and complete GAN training settings remain unresolved here. A project link alone does not establish reproducibility. e-scenese-graphse-recoverye-pairse-traininge-missing
Audit the PREV manifest: Section 3.2 calls ScaleVLN 4.62 times PREVALENT, Section 4.1 describes PREV as 178,270 samples, and Figure 1 labels PREVALENT 1.0M. These cannot safely be treated as one identical training set. Scene-overlap checks and data/license access terms are not established by the supplied body. e-pairse-prev
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate graph density from obstacle-crossing edges
Reader-proposed check, not performed: use the same HM3D scenes, rendered images, speaker and training budget in a factorial comparison of low/high node density and retained/removed obstacle-crossing edges. Match route counts and intermediate-node distributions across conditions, and include a R2R/PREV-only control. Repeat with five declared seeds and report unseen SR/SPL with uncertainty. If edge removal helps at both matched densities, that supports traversability as an independent mechanism; if benefits appear only after increasing density, Table 1’s collision-based explanation needs qualification. e-graphse-graph-teste-pairse-size
Check 2: Test recovery while holding language and routes fixed
Reader-proposed check, not performed: keep recovered HM-E constant and compare adding Gibson with faulty versus recovered images, using identical trajectories, stored instructions, sample counts, visual encoder and training mixture. Repeat across seeds and evaluate unseen SR/SPL. On a fixed paired-image subset, independently check whether recognizable landmarks survive recovery. A consistent follower gain with preserved landmarks supports useful image repair; no gain under fixed language, or gains accompanied by altered landmarks, would challenge a simple fidelity-based explanation and motivate separating appearance effects from instruction changes. e-recoverye-imagee-pairse-training
8.3 Reading coverage
Visual audit: Visually inspected the title/author/affiliation block, CVF edition note and Figure 1 on p. 1; introductory protocol and human-view caveat on p. 2; graph and recovery method on p. 3; Figure 2, generation counts and task descriptions on p. 4; training hardware/configuration, metrics and Table 1 on p. 5; Tables 2–5 and data-use discussion on p. 6; Tables 6–8, proxy-task discussion and result commentary on p. 7; Tables 9–11, symbol keys and conclusions on p. 8; final reference page/footer on p. 12. Every final crop was separately viewed. Figure 2 arrows were checked against its caption and Sections 3–3.2; Table 6 checkmarks and Table 9 star were checked against their text. The shared diagram’s REVERIE scope is qualified, and the Table 7 cross-reference discrepancy is disclosed. All six text chunks, including references, were read. Reference pages 9–11 were not visually inspected; no numerical, methodological or reproduction claim depends on their layout. The referenced appendix is absent and no external material was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 12. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Works
- 3. Scaling Data for Learning VLN
- 3.1. Resources for VLN Training
- 3.2. Generating Augmented Data
- 4. Experiments
- 4.1. Experimental Setup
- 4.2. Scale VLN Data, What Really Matters?
- 4.3. How to Utilize Large-Scale Data?
- 4.4. Evaluate on Various VLN Tasks
- 5. Conclusion
- 6. Acknowlegement
- References, PDF pp. 9–12
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- All six supplied chunks were read individually, covering all 12 PDF pages. The main paper is complete; its referenced appendix is absent from this artifact and was not read.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by inspecting the original figures and tables on PDF pp. 1–8 and every final crop.
- Separate supplemental material availability has not been fully verified.
- Identity/edition: the observed title and all nine authors match the catalog. PDF p. 1 identifies the CVF ICCV 2023 Open Access version and states that it is identical to the accepted version except for the watermark. The final IEEE artifact was not supplied. No separate revision identifier/date is established. Printed pagination is 12009–12020, unlike the catalog BibTeX range 11975–11986; the catalog was not changed.
- Linked code, external datasets and referenced papers were not inspected; no experiments were reproduced. Reference pages 9–11 were read as text, without image inspection. Appendix material on graph visualizations, REVERIE collection/training and proxy-task implementation remains unavailable.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, CVF watermark and footer; PDF p. 12, footer
The observed title and nine authors match the catalog; affiliations are printed beneath them. The CVF note identifies the open-access accepted-version artifact. Printed first/last pages are 12009/12020.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Section 1
The paper motivates environment diversity and synthetic supervision, reports 80% SR with an abstract +11-point comparison, and describes single-run evaluation without search, pre-exploration or ensembling.
Go to primary source ↓e-scenesPDF p. 3, Section 3.2, Collecting Environments
The pipeline uses 800 HM3D training scenes and 491 Gibson training/validation scenes.
Go to primary source ↓e-graphsPDF p. 3, Section 3.2, Constructing Navigation Graphs
Sampling separation exceeds 0.4 m; clustering threshold is 1.0 m; initial connections are within 5.0 m with maximum degree five. A cited refinement produces final graphs; mean edge length/degree are 1.41 m/4.55.
Go to primary source ↓e-recoveryPDF pp. 3–4, Recovering Faulty Rendered Images; Figure 2
Co-Mod GAN learns rendered-to-camera recovery on paired Matterport3D images and repairs HM3D/Gibson views.
Go to primary source ↓e-pipelinePDF p. 4, Figure 2 and caption; PDF p. 3, Section 3 introductory paragraph
The diagram proceeds from scans to graphs/recovered views, sampled paths and speaker instructions, then a dataset for pre-training/fine-tuning. The text distinguishes R2R-style from REVERIE-specific generation.
Go to primary source ↓e-pairsPDF p. 4, Sample Trajectories and Generate Navigational Instructions
Routes have three to five intermediate nodes. HM3D/Gibson yield 2,890,267/2,051,443 paths; EnvDrop Speaker generates instructions. Total pairs are 4,941,710; the stated PREVALENT multiplier is 4.62.
Go to primary source ↓e-tasksPDF p. 4, Section 4.1, Datasets
R2R uses detailed instructions; REVERIE requires reaching and localizing a described object; CVDN uses dialogue history; R2R-CE transfers paths to continuous Habitat scenes.
Go to primary source ↓e-agentsPDF p. 5, Baseline VLN Models
DUET builds an online topological map and is used for R2R/CVDN/REVERIE. HAMT stores historical observations and is used for R2R-CE.
Go to primary source ↓e-trainingPDF p. 5, Training and Implementation Details; PDF p. 7, end of Section 4.3
The paper tests MLM/MRM/SAP, fine-tunes using DAGGER and gives CLIP ViT-B/16 and DUET iteration/batch/learning-rate/GPU details. The final recipe uses MLM+SAP with ScaleVLN, then R2R+HM-E fine-tuning.
Go to primary source ↓e-metricsPDF p. 5, Evaluation Metrics
SR counts stops within 3 m; NE is final-to-target distance; TL is path length; SPL penalizes path length; nDTW measures alignment. CVDN uses GP.
Go to primary source ↓e-graph-testPDF p. 5, Table 1 and caption; Section 4.2, Effect of Navigation Graphs
Both graph variants use 800 HM3D scenes without image recovery. AutoVLN/Ours density is 0.36/1.16 nodes per square meter and obstacle-crossing rate 29.35%/0.00%. Unseen SR is 72.75/76.01 and SPL 62.56/66.94; no-HM3D baseline SR is 72.92.
Go to primary source ↓e-sizePDF p. 6, Table 2, all rows and caption; Effect of More Data
At 700k samples, 200/400/800 scenes give unseen SR 74.84/75.22/76.25 and SPL 64.59/65.32/66.00. At 800 scenes, 700k/1400k/2890k samples give SR 76.25/76.01/76.59 and SPL 66.00/66.56/67.74. All fine-tune on R2R; no uncertainty is tabulated.
Go to primary source ↓e-pretrainPDF p. 6, Table 3, all rows
After R2R fine-tuning, unseen SR rises from 72.92 to 76.59 to 78.08 when adding HM-E and then Gibson to R2R/PREV pre-training. Without fine-tuning the values are 47.42, 55.09 and 57.00.
Go to primary source ↓e-imagePDF p. 6, Table 4 and F/R caption; Effect of Image Quality
F/R mean faulty/recovered images. HM-E recovery changes unseen SR 76.01→76.59. Adding faulty Gibson gives 76.33; recovering both gives 78.08. Last-two-row SPL values are 66.97/68.31.
Go to primary source ↓e-speakerPDF p. 6, Table 5 and Effect of Augmented Instruction
GPT-2/EnvDrop BLEU-4 scores are 24.36/27.66 and unseen SR 75.82/76.59. Authors only tested two speakers and describe generated instructions as low quality.
Go to primary source ↓e-mixturePDF p. 7, Table 6, rows 4–7; PDF p. 6, Section 4.3; PDF p. 7, end of Section 4.3
Identical R2R/PREV/HM-E pre-training followed by R2R, R2R+PREV, R2R+HM-E or all-three fine-tuning gives unseen SR 76.59/75.78/79.10/77.10. Row 6 seen SR is 80.02. Final text recommends excluding Gibson from fine-tuning.
Go to primary source ↓e-proxyPDF p. 7, Table 7 and Effect of Pre-training Tasks; Table 6 rows 4 and 6
No pre-training, MLM, SAP, MLM+SAP and all objectives give unseen SPL 51.25/57.47/65.40/67.74/66.62. The text names Method 6, but the MLM+SAP row repeats Table 6 Method 4 numerically.
Go to primary source ↓e-r2rPDF p. 8, Table 9, DUET, BEVBert and ScaleVLN rows, caption/star footnote; PDF p. 7, Section 4.4, R2R
Test SR/SPL are DUET 69/59, BEVBert 73/62, unstarred ScaleVLN 77/68 and starred ScaleVLN 80/70. The star adds EnvEdit and CLIP ViT-H14. Starred NE is 2.27 m. Prose describes the unstarred SR/SPL gains as eight points each.
Go to primary source ↓e-humanPDF p. 2, footnote 3; PDF p. 8, Table 9, Human row
Humans use egocentric observations whereas the model uses panoramas. Human test SR is 86 under that different observation protocol.
Go to primary source ↓e-reveriePDF p. 7, Table 8, AutoVLN and DUET+ScaleVLN test-unseen columns; Section 4.4, REVERIE
AutoVLN has SR/SPL 55.17/38.88; ScaleVLN has 56.13/39.52. Only navigation OSR/SR/SPL are shown. Prose reports a 0.94-point SR gain, differing from the table’s 0.96.
Go to primary source ↓e-cvdnPDF p. 8, Table 10, HAMT and DUET+ScaleVLN test-unseen GP; PDF p. 7, Section 4.4, CVDN
HAMT/ScaleVLN GP values are 5.58/6.97 m. Prose states +1.41 m and +25.26%, inconsistent with arithmetic from the displayed values.
Go to primary source ↓e-cePDF p. 5, Implementation Details; PDF p. 8, Table 11 and R2R-CE paragraph
HAMT uses a candidate waypoint predictor. Test-unseen NE/SR/SPL are 5.11/55/50 for ScaleVLN and 5.54/47/41 for VLN-BERT+Ego2-Map. Continuous Habitat evaluation uses discrete synthetic training pairs.
Go to primary source ↓e-prevPDF p. 1, Figure 1 legend; PDF p. 4, end of Section 3.2 and Section 4.1 PREV paragraph
Figure 1 labels PREVALENT 1.0M; Section 3.2 states a 4.62-times size ratio; Section 4.1 describes PREV as 178,270 samples. These descriptions are not reconciled.
Go to primary source ↓e-missingPDF p. 3, Section 3 introduction and Constructing Navigation Graphs; PDF p. 5, Training and Implementation Details; PDF p. 1, project link
REVERIE-specific collection/training, graph visualizations, objective implementation and further training details are deferred to an appendix. A project URL is printed; the body does not supply a complete implementation specification.
Go to primary source ↓8.5 Primary sources
Scaling Data Generation in Vision-and-Language Navigation ↗
PDF · 9,686 extracted words
Source fingerprint
4b3616d4881aa332eb527228d486f09f91cc9a37e65bd7df6af7586d25a1be25