StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling
1. Paper overview
In one sentence: StreamVLN makes navigation dialogue efficient by reusing recent KV states and compressing older visual history, trading complete context for bounded working context and occasional window-transition latency. identitymotivationarchitecturecontextpruningdata-ablationcache-latencyscore-discrepancies
| At a glance | What to know |
|---|---|
| Research problem | Source description Continuous vision-and-language navigation needs detailed current observations, useful history and responsive action generation. Re-prefilling the entire dialogue wastes computation, while unrestricted KV caching grows memory. StreamVLN addresses this tension by separating recent dialogue from older visual memory. motivationcontext |
| Core mechanism | Source description A single vision-language-action pathway combines an active dialogue window with slower visual-memory updates, enabling cross-turn KV reuse. architecturecontext |
| A key reported result | VLN-CE navigation with extra VLN data: StreamVLN: R2R 56.4 / 50.2; RxR 54.4 / 45.4. SR / SPL (%). R2R-CE and RxR-CE Val-Unseen; Table I dagger-marked RGB-only models. NaVILA: R2R 54.0 / 49.0; RxR 49.3 / 44.0. These are table values, not the conflicting headline values in Section IV-C. Extra-data membership does not imply identical datasets. evaluationnavigationscore-discrepancies |
| Reading caution | Reader analysis The supplied revision contains unresolved numerical inconsistencies: Section IV-C headline navigation scores disagree with Table I, and several data-ablation gains disagree with Table III. Table values are retained explicitly rather than silently harmonized. score-discrepancies |
Core contributions
- Source description
A single vision-language-action pathway combines an active dialogue window with slower visual-memory updates, enabling cross-turn KV reuse. architecturecontext
- Source description
Training-free voxel pruning selects existing visual token states instead of learning a feature compressor; it requires geometry for back-projection. pruningnavigation
Figure 2. Recent dialogue remains detailed; inactive visual context is sampled and spatially pruned. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the filmstrip and follow the arrow through the vision encoder and projector into the observation-token stream. The grey token denotes the instruction; the action icons below the LLM are generated responses. Solid outlines identify the current window, while dashed outlines identify inactive windows. Follow the lower branch in order: temporal sampling removes some observations, then voxel-based spatial pruning selects remaining patch states. The dashed token styles mean pruned tokens, not an attention mask. Section III-B explains that old prompt and action states are discarded when a window is consolidated; the decoder subsequently conditions on retained visual memory and current-window KV states. architecturecontextpruningnavigationpruning-resultscache-latency
What it supports. The figure supports a single action-generating model with two context-update rates. Its memory is selected past observation information, not a generated future scene. Reusing recent KV states avoids repeating all history prefill, while consolidation reduces the context carried between windows; Figure 7 tests the resulting latency pattern.
Where the evidence stops. The RGB-only schematic does not show the depth and odometry required by optional pruning. Algorithm 1 and Table I's footnote supply that requirement. Prune-All also prunes streaming-frame KV tokens; the inactive-window illustration alone does not describe every evaluated variant.
2. Motivation
2.1 The problem and the proposed response
Continuous vision-and-language navigation needs detailed current observations, useful history and responsive action generation. Re-prefilling the entire dialogue wastes computation, while unrestricted KV caching grows memory. StreamVLN addresses this tension by separating recent dialogue from older visual memory. motivationcontext
2.2 What this reading follows
A navigation agent sees many nearly repeated views while moving through a room, yet it must remember earlier landmarks and respond to the next observation. StreamVLN treats this as a dialogue-management problem. A recent window preserves detailed observation/action context; older observations become a smaller memory, optionally pruned using 3D geometry. The same Video-LLM produces action responses and supports visual question answering. Read the architecture alongside the latency curve, then use the navigation and ablation tables to distinguish evidence for the memory mechanism from gains due to training data. This edition covers the supplied July 2026 v2 PDF and preserves its unresolved numerical inconsistencies. identitymotivationarchitecturecontextpruningdata-ablationcache-latencyscore-discrepancies
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | One Model |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded VLA, autoregressive and memory/long-horizon labels fit the architecture: one vision-encoder/projector/LLM pathway directly emits actions from language and observation history. SlowFast denotes two context-update rates, not two learned policies. No explicit future-observation prediction, joint future/action objective or inverse-dynamics action extraction is presented; the world-model prediction paradigm and quadrant are therefore appropriately not applicable. architecturecontextpruning
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow one observation through two context timescales
Imagine the robot has just advanced toward a doorway. Its new image is encoded into visual tokens, appended to the active dialogue, and used to decode an action response. Recent observations and actions already have cached key/value states, so the model need not encode that same prefix again on every turn. The important event comes when the window fills. The method discards old non-observation dialogue states and carries forward selected visual history as memory for the next window. That transition costs prefill time again, which explains the red spikes in Figure 7. The architecture has one action-producing LLM; 'slow' and 'fast' describe when context is updated. Memory supplies past evidence to the decoder, rather than simulating the consequences of candidate actions. architecturecontextcache-latency
5.2 Understand what the voxel mask keeps and deletes
Temporal sampling cannot remove all repetition because successive views can still show the same wall or doorway. StreamVLN uses depth to place image patches in a shared voxel space. Algorithm 1 first ignores invalid voxel indices, then keys its latest-token map by both a temporal group and a voxel. The temporal group is floor(t/K), where t is the time index and K is the stride; therefore selection is not one token per voxel over the entire episode. It marks the latest token for each group, then deletes an entire frame's mask when its surviving count falls strictly below the threshold fraction. These are selection operations on existing states. The geometric pruner is training-free, while the underlying vision-language model is fine-tuned. Table I makes the additional depth and odometry requirement explicit. pruningtrainingnavigation
5.3 Separate architectural evidence from recipe and deployment evidence
The main navigation table evaluates complete trained systems, so its ranking combines context design, model initialization and data. For a closer mechanism test, Table IV varies memory and window settings within oracle-only training, while Figure 7 compares cache reuse patterns. Table III addresses a different question: which data mixtures help the navigation policy? Its fixed-count VideoQA/MMC4 comparison is more focused than its first-to-second-row comparison, which adds a training stage and multiple data sources. This separation is a reader interpretation of the experimental design, not an additional experiment. The robot bars then establish that the system can execute useful navigation in the tested physical settings. ScanQA measures answer quality; it cannot by itself prove that improved visual reasoning caused the navigation gains. navigationmemory-ablationcache-latencydata-ablationrobot-resultsscanqa
5.4 Training and inference
During training
Training uses LLaVA-Video 7B, described as using Qwen2-7B. One oracle-navigation epoch precedes DAgger collection; another epoch mixes navigation and general multimodal data. Warm-up peak learning rates are 2e-5 for the language model and 5e-6 for the vision encoder, with 128 clips per step and about 1,500 A100 GPU-hours. The visual encoder is therefore not wholly frozen. training
Navigation data comprise 450K clips from 60 MP3D scenes, 300K clips from 700 HM3D scenes via ScaleVLN, and 240K corrective DAgger samples using Habitat's shortest-path follower. General data add 248K video/scene QA examples and 230K MMC4 interleaved image-text examples. data
The paper describes autoregressive supervised fine-tuning but supplies no explicit loss equation, loss masking, optimizer configuration or full second-stage schedule. 'Training-free' applies to voxel pruning, not the navigation model. contexttrainingpruning
During inference
New observations reuse active-window KV states; window transitions require historical-context prefill. Prune-Mem operates on eight memory frames, while Prune-All also prunes streaming-frame cache tokens. Only these pruning variants use depth and odometry in Table I. contextpruning-resultsnavigation
Physical deployment streams Go2 RGB-D observations from a RealSense D455 to a remote RTX 4090 workstation, which returns commands. The reported 0.27 s inference covers four actions; communication adds 0.2 s indoors or 1.0 s outdoors. These are separate latency components. deployment
5.5 Implementation flow
- Encode an observation and continue the dialogue
The vision encoder and projector feed visual tokens to the LLaVA-Video language model alongside instruction and action history. Each turn pairs observation o_i with response a_i. Prefill caches attention key/value states; autoregressive decoding produces the action response. architecturecontext
- Retain recent dialogue, consolidate older observations
A window retains N dialogue turns. At capacity, its states are offloaded; non-observation prompt and action states are discarded. Historical observation states are temporally sampled into memory for subsequent windows. This is stored observation context, not a predicted future world state. contextpruning
- Prune spatial repetition
Image patches are back-projected into shared 3D voxels. Algorithm 1 groups time indices by floor(t/K), retains the latest valid token per temporal group and voxel, then clears a frame's mask if fewer than theta times its patch count survive. The mask selects token states without a learned pruning module. pruning
6. Experiments & results
StreamVLN turns a Video-LLM into a streaming navigation policy: it reuses recent dialogue KV states and compresses older visual context into memory. Optional geometric pruning removes repeated spatial observations. Simulation, robot and latency experiments support the design, but inconsistent prose/table scores and missing implementation details limit precise reproduction.
6.1 Read the original evidence
Table I. Read performance together with the modality columns and extra-data markers. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First read the observation columns: panoramic input, odometry, depth and single RGB distinguish the method groups. Then compare within each benchmark, keeping the dagger marker for additional VLN training data in view. The unmarked StreamVLN row and dagger-marked StreamVLN row are different training settings. The bottom two rows apply inference-time pruning to the extra-data model. Their starred checks under odometry and depth have a specific meaning in the retained footnote: geometry is used for back-projection during pruning. Higher SR and SPL indicate better success and path-weighted success; lower NE indicates less navigation error. evaluationnavigationpruning-resultsscore-discrepancies
What it supports. In the extra-data RGB-only group, StreamVLN reports R2R SR/SPL of 56.4/50.2 and RxR 54.4/45.4, compared with NaVILA's 54.0/49.0 and 49.3/44.0. Memory pruning raises R2R to 57.4/51.1 but lowers RxR slightly to 53.9/45.1. These are distinct settings, not one universal best configuration.
Where the evidence stops. Section IV-C instead states R2R 56.9/51.9 and RxR 52.9/46.0. This unresolved discrepancy is preserved; the reading uses the visible table cells. Dagger markers do not guarantee matched training data, and pruned variants require geometry beyond RGB.
Figure 5. The physical-navigation advantage appears most clearly in the office scenario. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left chart first: each group is a physical scenario, and the legend maps the four bar colors to methods. Compare orange StreamVLN with yellow NaVILA before checking the other baselines. Then move to the corresponding group in the right chart to examine navigation error, where lower is better. The experiment text specifies twenty trials per method per scenario, so these are small scenario-specific evaluations. The platform is a Go2 robot with a RealSense D455, using a remote RTX 4090 server. The charts report completed navigation outcomes, distinct from the separate ScanQA question-answering evaluation. robot-resultsdeploymentscanqaqualitative
What it supports. StreamVLN's hallway, bedroom and office success rates are 100%, 85% and 60%; NaVILA reports 100%, 70% and 0%. All three displayed baselines have zero office success in this experiment. The clearest supported conclusion is improved physical performance on the tested harder scenarios, rather than a blanket claim of robust navigation everywhere.
Where the evidence stops. No error bars or trial-level outcomes are supplied. These bars do not evaluate mall or outdoor success rates. Reported server inference and communication times are separate measurements; the chart cannot establish end-to-end deployment latency.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| VLN-CE navigation without extra VLN data Habitat R2R-CE Val-Unseen; Table I unmarked RGB-only rows. | StreamVLN: 52.8 / 47.2. SR / SPL (%) | NaVILA: 49.7 / 45.5. Higher reported success and path-weighted success in this comparison; training and compute are not experimentally matched. evaluationnavigation |
| VLN-CE navigation with extra VLN data R2R-CE and RxR-CE Val-Unseen; Table I dagger-marked RGB-only models. | StreamVLN: R2R 56.4 / 50.2; RxR 54.4 / 45.4. SR / SPL (%) | NaVILA: R2R 54.0 / 49.0; RxR 49.3 / 44.0. These are table values, not the conflicting headline values in Section IV-C. Extra-data membership does not imply identical datasets. evaluationnavigationscore-discrepancies |
| Inference-time spatial pruning Table I dagger-marked variants, R2R and RxR Val-Unseen. | Prune-Mem: R2R 57.4 / 51.1, RxR 53.9 / 45.1; memory tokens reduced 28% / 22%. Prune-All: R2R 56.0 / 48.5, RxR 53.3 / 44.0; visual tokens reduced 32% / 30%. SR / SPL (%); token reduction (%) | Unpruned: R2R 56.4 / 50.2; RxR 54.4 / 45.4. Compression has a benchmark-dependent accuracy cost; pruned variants additionally require depth and odometry. navigationpruning-results |
| Physical navigation Go2 hallway, single bedroom and room-to-room office; 20 trials per method per scenario. | StreamVLN: 100, 85 and 60, respectively. SR (%) | NaVILA: 100, 70 and 0; NaVid: 80, 20 and 0; CMA: 25, 0 and 0. Actual robot outcomes favor StreamVLN in harder scenarios. The small evaluation reports no uncertainty intervals. deploymentrobot-results |
| ScanQA question answering ScanQA validation; 16 multi-view images per scan. | StreamVLN: 28.8. EM (reported score) | NaVILA with 16 frames: 27.4. Evidence for question answering, not a navigation success measure or a causal test of QA-to-navigation transfer. scanqa |
| Training-data ablation R2R Val-Unseen; no voxel pruning; equal total VL-data count for VideoQA versus VideoQA+MMC4. | VideoQA+MMC4: 52.8 / 47.2; adding ScaleVLN: 56.4 / 50.2. SR / SPL (%) | VideoQA-only co-training: 50.8 / 45.7. MMC4 yields +2.0 / +1.5 percentage points; ScaleVLN yields +3.6 / +3.0 by table arithmetic. Several prose deltas disagree. data-ablationscore-discrepancies |
| Memory-context ablation R2R Val-Unseen; oracle-only first-stage training; eight-turn window. | 8×196 memory: 45.5 / 41.6. SR / SPL (%) | 2×196: 37.3 / 34.2; all visual context: 40.0 / 36.4. More retained memory helps up to the tested intermediate size; retaining everything is worse. This is a different training regime from main results. memory-ablation |
| Cross-turn cache latency Figure 7; eight-turn windows; latency to generate eight action tokens over the plotted dialogue. | Sliding-window latency stays close to full-turn caching except at window-transition spikes. Latency in seconds; qualitative curve comparison | Single-turn caching repeatedly incurs higher history-prefill latency. Supports the cache-reuse mechanism; the plot does not measure peak memory or provide timing variance. cache-latency |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table III. The data recipe matters, but only some row comparisons isolate one change. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row's checkmarks before its scores. The first row is oracle-only training. The second adds the later co-training stage with DAgger and VideoQA; the third changes the VL mix to VideoQA plus MMC4. The paper holds total VL-data count fixed for that second-to-third-row comparison. The fourth adds ScaleVLN. The fifth removes DAgger from that fuller recipe, and the sixth removes RxR instead. All rows are evaluated on R2R Val-Unseen without voxel pruning. This ordering helps separate the relatively focused MMC4 comparison from the first-to-second-row change, which combines additional training and multiple data sources. data-ablationscore-discrepancies
What it supports. Replacing the VideoQA-only mix with VideoQA+MMC4 raises SR/SPL from 50.8/45.7 to 52.8/47.2. Adding ScaleVLN then reaches 56.4/50.2. Subtracting the visible table cells gives gains of 2.0/1.5 and 3.6/3.0 percentage points, respectively. This supports the recipe's contribution alongside the context design.
Where the evidence stops. Several claimed gains in the surrounding prose disagree with the cells, including the ScaleVLN increment. The first-to-second-row gain cannot be attributed solely to DAgger, because the training stage and VL data also change. Uncertainty and repeated-seed results are absent.
Table IV. An intermediate memory budget beats both sparse memory and retaining all visual context. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Separate the table into two comparisons. The first four rows keep Window at eight dialogue turns and change Memory, using the paper's budget notation 2×196, 4×196, 8×196 and all. Here, all means the entire visual context. The last two rows keep 8×196 memory but shorten the window to four or two turns. Read these as oracle-only first-stage experiments, as specified in the adjacent text; their absolute performance is not the final mixed-data model's score. The same text explains that shorter windows create more training clips, which complicates a pure comparison of context length. memory-ablation
What it supports. At an eight-turn window, SR rises from 37.3 with 2×196 memory to 45.5 with 8×196; using all context yields 40.0. The 8×196/eight-turn configuration also has the best displayed SPL, 41.6. Useful history therefore matters, but simply retaining more context does not monotonically improve the tested model.
Where the evidence stops. The window comparison changes training exposure: the reported sample counts are about 450K, 815K and 1.5M for windows eight, four and two. It does not establish the best window at fixed compute or explain causally why all-context memory performs worse.
Figure 7. Sliding-window reuse preserves fast ordinary turns but pays for context refresh at boundaries. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each colored curve across conversation turns rather than comparing only its highest point. Blue full-turn caching remains nearly flat. Red sliding-window caching largely overlaps it between the spikes just after eight-turn boundaries, when historical context needs prefill again. Yellow single-turn caching repeatedly recomputes history and has higher latency as the current dialogue grows. Section IV-D specifies that this timing concerns generating eight action tokens. The caption fixes the window size at eight turns. The plot therefore makes the timing consequence of the window transition visible; it is not a direct measurement of navigation accuracy or cache memory consumption. cache-latencycontextdeployment
What it supports. The latency pattern supports the intended mechanism: cross-turn reuse removes repeated history computation during most turns, while the sliding window periodically refreshes context. Full-turn caching avoids those refresh spikes but retains more historical cache. The tradeoff is occasional higher latency in exchange for controlling retained context, not uniformly minimal latency.
Where the evidence stops. The plot provides neither memory measurements nor timing variance, and does not specify its timing hardware locally. Eight generated action tokens should not be equated with the four executable actions timed in the deployment description. End-to-end robot latency also includes communication.
7. Analysis & limitations
7.1 What the evidence leaves open
The supplied revision contains unresolved numerical inconsistencies: Section IV-C headline navigation scores disagree with Table I, and several data-ablation gains disagree with Table III. Table values are retained explicitly rather than silently harmonized. score-discrepancies
Geometric pruning's robustness to noisy depth, pose drift or moving objects is untested. Cross-method comparisons also vary sensor inputs and training data. Robot success examples in additional environments do not establish an outdoor success rate. pruningnavigationrobot-resultsqualitative
Window-size ablations change both context and training sample count; they do not isolate context length at fixed training cost. Main results and ablations lack reported error bars or seed variability. memory-ablationdata-ablationnavigation
7.2 Questions for discussion
- Does voxel-aware selection outperform a token-count-matched geometry-free mask when depth and pose are perturbed?
- Would the eight-turn window remain preferable with training exposure and compute held fixed?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the exact pretrained checkpoint, Habitat scenes and splits, clip construction, DAgger collection and multimodal sampling recipe. The paper states 300K HM3D clips but later calls the ScaleVLN subset '150k' without explaining its unit; those counts should not be equated. datatrainingreproduction-gaps
Resolve voxel size, stride K, threshold theta, depth/pose alignment, total memory retention, action vocabulary and token-to-command conversion. Section IV-B names Qwen2-7B, but reference [34] names Qwen2.5. The PDF does not resolve checkpoint identity, simulator/software versions, optimizer settings or exact timing hardware for Figure 7. pruningcontextdeploymenttrainingcache-latencyreproduction-gaps
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test cache reuse with identical effective context
Reader-proposed, not performed: replay the same observations and instructions through one checkpoint, comparing reused KV states against history recomputation while keeping retained tokens, precision, generated-token budget and hardware identical. Measure prefill time, decoding time, total latency, peak cache memory and output agreement; analyze ordinary turns separately from eight-turn transitions. Add unrestricted full-turn retention as a memory-growth reference. The proposed mechanism predicts lower ordinary-turn prefill time with reuse and distinct transition overhead. If speed improves only because fewer tokens are supplied, or output differences persist despite identical effective context, cache reuse alone has not explained the result. contextcache-latencytraining
Check 2: Compare voxel pruning with a mask that retains the same number of tokens
Reader-proposed, not performed: use a fixed extra-data checkpoint and the same R2R/RxR Val-Unseen episodes for no pruning, voxel pruning, and geometry-free selection matched to the voxel mask's retained-token count per frame. Evaluate Prune-Mem and Prune-All separately. Hold temporal sampling fixed and sweep reported implementation values of voxel size, K and theta once recovered; additionally perturb depth and pose in controlled increments. Record SR, SPL, token counts and latency across repeated runs. Accurate voxel selection should retain more task-relevant information than the matched control. Equal performance would weaken the case for geometry specifically; sharp degradation under small perturbations would expose a deployment limitation. pruningpruning-resultsnavigationevaluation
8.3 Reading coverage
Visual audit: All eight supplied PDF pages were rendered and visually inspected, covering the title/authors/version, Figures 1–7, Algorithm 1, Tables I–IV, training and deployment details, and references. All four text chunks were read completely. Every final crop was inspected: Figure 2, Table I including both footnotes, Figure 5 with both charts, Tables III and IV, and Figure 7 with axes and legend. The other figures and ScanQA table were inspected on their full pages. Figure 2's temporal-then-spatial selection was cross-checked against III-B/III-C and Algorithm 1, including the strict frame-removal inequality. Numerical prose/table conflicts and the Qwen2/Qwen2.5 citation mismatch remain disclosed. No appendix is present; separate supplements, project videos and implementation files are outside the supplied review.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF p. 1: title, authors, affiliations, abstract and I. Introduction
- PDF p. 2: II. Related Work and III. Method overview
- PDF p. 3: III-A. Multi-Turn Autoregressive Generation; III-B. Fast-Streaming Dialogue Context; III-C. Slow-Updating Memory Context; Algorithm 1
- PDF pp. 3–4: III-D. Co-Training with Multi-Source Data
- PDF p. 4: IV-A. Experimental Setup and IV-B. Implementation Details
- PDF pp. 4–6: IV-C. Comparisons with State-of-the-Arts
- PDF pp. 6–7: IV-D. Ablation Studies
- PDF p. 7: V. Conclusion and Acknowledgements
- PDF p. 8: References
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/version: the inspected title page states arXiv:2507.05240v2, 9 July 2026. The title and all twelve authors match the catalog. The catalog submission date is 7 July 2025; the earlier edition was not supplied, so revision changes cannot be compared.
- Acquisition caveat retained: Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This edition resolves that visual gap through inspection of all eight PDF pages and six original crops.
- Separate supplemental material availability has not been fully verified.
- No appendix is present in the supplied PDF. Linked project material, code, datasets and demonstration videos were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author block, affiliation footnote and arXiv margin
The exact title and twelve authors match the supplied catalog. The margin identifies arXiv:2507.05240v2 [cs.RO], 9 Jul 2026. Affiliations are Shanghai AI Lab, The University of Hong Kong, Zhejiang University and Shanghai Jiao Tong University.
Go to primary source ↓motivationPDF pp. 1–2, Abstract and I. Introduction
The paper motivates streaming VLN by the tradeoff between visual detail, history retention and repeated context computation.
Go to primary source ↓architecturePDF p. 2, Figure 2 and III. Method opening
A vision encoder and projector supply tokens to one LLM; its interleaved stream includes instruction, observations and output actions, with active/inactive windows and cached states.
Go to primary source ↓contextPDF p. 3, III-A and III-B, including the unnumbered decoder equation
Observations and action responses form dialogue turns. Prefill caches KV states; an N-turn window is offloaded at capacity, non-observation states are discarded and visual history becomes memory for future windows.
Go to primary source ↓pruningPDF p. 3, III-C and Algorithm 1, lines 1–12
Depth back-projects patches into uniform voxels. Valid voxel indices are nonnegative. The latest token is retained per floor(t/K) group and voxel; line 11 removes frames whose surviving mask count is strictly below theta times H times W.
Go to primary source ↓dataPDF pp. 3–4, III-D and Figure 3
The recipe lists 450K MP3D clips from 60 scenes; 300K HM3D clips from 700 scenes; 240K DAgger samples; 248K VQA samples; and 230K MMC4 samples. DAgger uses Habitat's shortest-path follower.
Go to primary source ↓trainingPDF p. 4, IV-B. Implementation Details
LLaVA-Video 7B is described as using Qwen2-7B. Training has one oracle epoch and one mixed-data epoch after DAgger collection, warm-up peak learning rates 2e-5 and 5e-6, 128 clips per step and about 1500 A100 GPU-hours.
Go to primary source ↓evaluationPDF p. 4, IV-A. Simulation Benchmark Setup
Evaluation uses Habitat R2R-CE and RxR-CE validation-unseen splits, 79-degree camera HFOV, and NE, SR, OS and SPL metrics.
Go to primary source ↓navigationPDF p. 5, Table I, StreamVLN, NaVILA and pruning rows, observation columns and both footnotes
Unmarked StreamVLN R2R SR/SPL is 52.8/47.2 versus NaVILA 49.7/45.5. Dagger StreamVLN is 56.4/50.2 on R2R and 54.4/45.4 on RxR. Prune-Mem is 57.4/51.1 and 53.9/45.1; Prune-All is 56.0/48.5 and 53.3/44.0. Dagger denotes extra VLN data; starred checks restrict odometry/depth use to pruning.
Go to primary source ↓pruning-resultsPDF p. 4, IV-C. Effectiveness of Voxel-Based Spatial Pruning
Prune-Mem prunes eight memory frames and reports 28%/22% memory-token reduction on R2R/RxR. Prune-All also prunes streaming-frame KV tokens and reports 32%/30% visual-token reduction.
Go to primary source ↓deploymentPDF p. 4, IV-A. Real-World Evaluation Setup
Go2 uses a RealSense D455 RGB-D camera and remote RTX 4090 inference. Average inference is 0.27 seconds for four actions; communication is 0.2 seconds indoors and 1.0 seconds outdoors.
Go to primary source ↓robot-resultsPDF p. 5, Figure 5, SR bars; PDF pp. 5–6, IV-C. Real-World Experimental Results
The protocol specifies 20 trials per method per scenario. Hallway/bedroom/office SR is StreamVLN 100/85/60, NaVILA 100/70/0, NaVid 80/20/0 and CMA 25/0/0. The chart reports no uncertainty intervals.
Go to primary source ↓scanqaPDF p. 5, IV-C. Results on Video Question Answering; PDF p. 6, Table II, EM column
StreamVLN analyzes 16 multi-view images per scan. Table II gives validation EM 28.8 for StreamVLN and 27.4 for NaVILA, both with 16 frames.
Go to primary source ↓data-ablationPDF pp. 6–7, IV-D. Data Ablation; PDF p. 7, Table III, rows 1–6
No voxel pruning is used; total VL-data count is held fixed for the VideoQA/MMC4 comparison. Row SR/SPL pairs are 45.6/42.3, 50.8/45.7, 52.8/47.2, 56.4/50.2, 50.2/47.1 and 47.9/43.6. Rows 5 and 6 omit DAgger and RxR respectively.
Go to primary source ↓memory-ablationPDF p. 7, Table IV and IV-D. Memory Context Size / Sliding Window Size
Oracle-only first-stage SR/SPL at window 8 is 37.3/34.2 for 2×196 memory, 38.9/35.4 for 4×196, 45.5/41.6 for 8×196 and 40.0/36.4 for all. With 8×196 memory, windows 4 and 2 give 41.4/37.5 and 43.7/40.3. Window sizes 8/4/2 yield about 450K/815K/1.5M training samples.
Go to primary source ↓cache-latencyPDF p. 7, Figure 7, caption and IV-D. Effectiveness of KV-Cache Reuse
The eight-turn-window comparison measures latency for eight generated action tokens. Full-turn caching stays low; sliding-window caching spikes at transitions; single-turn caching repeats history prefill. Memory overhead is discussed but not plotted.
Go to primary source ↓score-discrepanciesPDF p. 4, IV-C versus p. 5, Table I; PDF pp. 6–7, IV-D prose versus p. 7, Table III
The prose states R2R 56.9/51.9 and RxR 52.9/46.0 SR/SPL, unlike Table I. Table III arithmetic gives row 2 minus 1: +5.2/+3.4, row 4 minus 3: +3.6/+3.0, row 4 minus 5: +6.2/+3.1, row 4 minus 6: +8.5/+6.6; corresponding prose claims are +5.3/+4.1, +2.9/+3.7, +5.5/+3.8 and +7.8/+7.3.
Go to primary source ↓qualitativePDF p. 4, Figure 4; PDF p. 6, Figure 6 and IV-C. Real-World Experimental Results
The authors present visual-reasoning and navigation examples, including mall and outdoor scenes. These are qualitative examples, without a reported outdoor trial denominator.
Go to primary source ↓reproduction-gapsPDF p. 3, III-D; PDF p. 4, IV-B and IV-C; PDF p. 8, reference [34]
III-D states 300K HM3D video clips, whereas IV-C calls the ScaleVLN subset 150k without clarifying the unit. IV-B names Qwen2-7B but its citation [34] is titled Qwen2.5 technical report.
Go to primary source ↓8.5 Primary sources
StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling ↗
PDF · 5,779 extracted words
Source fingerprint
e771475ef29094fbc8b0053719b18ebf6284c4bf82a05ddcfb71f119be06c886