GeoWorldAD: Geometry World Action Model for Autonomous Driving
1. Paper overview
In one sentence: GeoWorldAD refines driving trajectories with ego-aligned present geometry and latent future geometry, improving simulated progress while leaving the causal contribution of added training and the limits of forecasting unresolved. problembackboneworldactionevaluationfuture-ablationgeometry-ablationfuture-visuals
| At a glance | What to know |
|---|---|
| Research problem | Source description A planner needs spatial constraints without becoming unnecessarily conservative around moving traffic. The authors argue that fixed-frame geometry can misalign with ego-centric trajectories, while appearance-based future prediction supplies indirect geometric guidance. Their proposed solution makes present and anticipated geometry the representations used for planning. problembackbone |
| Core mechanism | |
| A key reported result | NAVSIM v2 closed-loop planning: 90.4 EPDMS; 99.0 NC; 98.6 TTC; 89.1 EP. EPDMS ↑; NC ↑; TTC ↑; EP ↑. navtest split; reactive traffic and extended scoring as described in Section 4.1. DVGT-2: 89.6 EPDMS; EponaV2: 88.9. Highest EPDMS among listed methods, by 0.8 and 1.5 score points respectively. Different architectures and supervision prevent causal attribution from this leaderboard alone. v2evaluation |
| Reading caution | Source description The planner uses fixed-length clips. KV caching for continuous planning remains future work; a streaming reconstruction backbone does not establish streaming or real-time planning. limitations |
Core contributions
- Source description
The appendix supports progressive multi-scale aggregation: four layers and four iterations yield 89.3 PDMS, versus 87.6 for 24 layers/one iteration and 88.2 for one layer/four iterations. The final design does not maximize ego progress. aggregation
Figure 2. Present geometry reaches the planner directly and through a learned future-geometry branch. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the video strip and follow the left column upward through DINOv2, frame attention, and global attention. Geometry tokens branch toward the middle world module and the right action module. In the middle, future queries first aggregate geometry, then use causal future attention. Equation (3) specifies that each future chunk can access itself and preceding chunks; the drawing does not display the mask. On the right, trajectory queries aggregate present geometry before the final latent-future refinement. The shared trajectory decoder appears at both levels because proposals are supervised throughout refinement. Ego status supplies motion context to both the world and action modules. backboneworldaction
What it supports. The architecture provides a direct route from present geometry to trajectories and an additional route through anticipated geometry. Section 3.3 specifies four present-geometry stages followed by one future stage. This explains how future information can modify an already spatially grounded proposal, rather than serving only as an auxiliary prediction task.
Where the evidence stops. The caption omits geometry decoders, and the right-hand depth pictures are not mandatory planning-time outputs. Sections 3.2–3.3 say that future latents remain useful while auxiliary decoders can be removed. No candidate-trajectory feedback arrow into the world model is specified.
2. Motivation
2.1 The problem and the proposed response
A planner needs spatial constraints without becoming unnecessarily conservative around moving traffic. The authors argue that fixed-frame geometry can misalign with ego-centric trajectories, while appearance-based future prediction supplies indirect geometric guidance. Their proposed solution makes present and anticipated geometry the representations used for planning. problembackbone
2.2 What this reading follows
A useful driving representation must describe both where obstacles are and how available space may change. GeoWorldAD addresses these needs with a geometry backbone, a future-token module, and an iterative trajectory planner. The central distinction is between learning future depth as supervision and using future latent features during planning: the latter remains active even when depth decoding is removed. Read the architecture alongside the two ablations, then interpret the benchmark gains within their simulator protocols. The appendix offers a view of what the future tokens can decode, but its qualitative depth examples do not measure forecast accuracy or demonstrate physical vehicle safety. problembackboneworldactionevaluationfuture-ablationgeometry-ablationfuture-visuals
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Conflicts with the recorded classification
The WAM interpretation and a modular world-model-to-planner architecture are supported by distinct geometry, future-query, and action components. However, the recorded IDM label is not established by the objectives: the planner learns expert trajectories and proposal scores, rather than an explicitly defined inverse-dynamics mapping between states. Joint training does not prove a single shared model architecture, and future-conditioned planning alone does not prove IDM. Preserve the catalog snapshot pending editorial review. backboneworldaction
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Put spatial supervision in the planner's moving frame
Begin with the representation mismatch described in Section 3.1. Standard StreamVGGT expresses reconstructed points relative to the first frame, while a driving trajectory is expressed in the moving ego frame. EgoStreamVGGT instead supervises each point map in its corresponding ego-camera frame and estimates transformations between adjacent frames. The temporal relationship is retained, but the geometric target is aligned with the planner's reference convention. Features from four decoder layers then offer different representation levels to trajectory queries. Appendix A.1 makes this adaptation concrete through camera, depth, and point-map losses, including confidence weighting and gradient matching. Table 4 supports the resulting design, but it should not be read as a controlled proof that coordinate alignment alone explains the gain: the backbone also undergoes driving-data adaptation. backbonereconstructiongeometry-ablationtraining
5.2 Learn an anticipated scene without decoding it at every planning step
The world module starts from learnable future queries rather than an already observed future frame. Each future chunk receives a temporal embedding, reads present geometry and ego status, and then exchanges information only with itself and earlier future chunks. This attention mask imposes temporal structure; it does not by itself describe sequential sampling of simulated actions. Future depth makes the representation trainable: present geometry queries the future latents, and a shared DPT head decodes the resulting features under depth supervision. The paper blocks this future-depth loss from updating the head, while reconstruction remains a separate training objective. During trajectory inference, the depth decoder is dispensable but the future tokens still condition the final planner stage. Figure 8 is therefore an inspection of an auxiliary decoding route, not the full planning interface. worldactionreconstructionfuture-visuals
5.3 Separate proposal learning, learned scoring, and causal evidence
The action module refines 64 proposals through four present-geometry stages and one future-geometry stage. At each stage, the closest proposal to the expert trajectory determines the trajectory loss, with earlier stages down-weighted. A separate head predicts simulator-derived proposal scores from the final features. My interpretation is that these objectives serve different purposes: expert matching supplies a feasible imitation target, while score learning adds the benchmark's safety, compliance, comfort, and progress preferences. The paper does not explicitly spell out the last selection or controller rule. Table 3 then asks whether the combined system improves over GeoAD, but the training schedule leaves an important alternative explanation to test: additional optimization. A convincing follow-up would retain the reported progress gain under equally trained controls, while checking that safety scores and rare collision outcomes do not deteriorate. actionevaluationfuture-ablationtraining
5.4 Training and inference
During training
Stage 1 initializes from StreamVGGT and trains geometry for 23K steps on OpenScene, nuScenes, ParallelDomain, and RealDriveSim in a 10:10:1:1 ratio. Stage 2 separately trains future geometry on OpenScene for 47K steps and the present-only GeoAD planner on NAVSIM navtrain for 32K steps. training
Stage 3 merges the branches, zero-initializes the future block's output projection to preserve GeoAD's initial behavior, and trains for 64K additional navtrain steps. AdamW uses batch 64 across 32 NVIDIA H20 GPUs, cosine scheduling, and learning rates 1e-4 in Stages 1–2 and 1e-5 in Stage 3. training
Reconstruction uses Huber camera loss and confidence-weighted depth/point-map errors with gradient matching. Future-depth supervision does not update the shared DPT head; this is a loss-specific restriction, not evidence that the head is frozen to reconstruction training. reconstructionworld
During inference
Observed geometry feeds both the world module and planner; future latents remain active for final trajectory refinement. Reconstruction and future-depth decoders can be omitted. Thus decoded future images are auxiliary training outputs, while anticipated latent geometry is an inference input. actionworld
The source describes scored proposals but does not explicitly specify the final selection rule, low-level controller, or replanning cadence. Simulator rollout evaluates planning; physical vehicle execution is not established here. actionevaluation
5.5 Implementation flow
- Align geometry with planning
DINOv2 feeds a 24-block StreamVGGT decoder with frame and global attention. Geometry tokens from layers 4, 11, 17, and 23 provide multiple scales. Point maps use each timestep's ego-camera frame, and camera poses describe adjacent-frame motion rather than motion relative to the first frame. backbone
- Anticipate geometry
Four future chunks spanning two seconds contain 64 tokens each and learned temporal embeddings. At each selected scale, queries cross-attend to present geometry plus ego embeddings, then attend causally to themselves and earlier future chunks. Present tokens query these future latents to produce features decoded as future depth. world
- Refine trajectory proposals
Trajectory queries have 64 proposals, eight waypoints, and embedding dimension 1024. Four present-geometry aggregation stages precede one future-token stage. A shared MLP decodes positions and tanh-bounded headings. Each stage receives best-proposal supervision; the final features also produce learned proposal scores. action
6. Experiments & results
GeoWorldAD plans driving trajectories from video using ego-aligned present geometry and learned future geometry tokens. Its planner progressively refines proposals across geometry scales, then incorporates anticipated scene evolution. The most persuasive reported benefit is greater ego progress over the present-only GeoAD baseline. Evaluation concerns NAVSIM simulation; reconstruction quality and qualitative future depth provide supporting evidence, with important gaps in causal isolation and deployment detail.
6.1 Read the original evidence
Table 1. GeoWorldAD leads the listed geometry and world-model groups, while iPad has the higher overall table score. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the Input and Aux. Sup. columns before comparing the rightmost PDMS column. C denotes camera and L denotes lidar; supervision also differs, including maps, boxes, future states, and dense geometry. GeoWorldAD uses camera input with dense and future geometry supervision. The lower groups contain the comparisons emphasized by the caption: geometry-based DVGT-2 and future-state-based EponaV2. Keep the gray rows in view, since they still contain reported results. NC, DAC, TTC, comfort, and EP reveal which components accompany the aggregate score. The evaluation is the navtest split under the paper's four-second, non-reactive NAVSIM v1 protocol. v1evaluation
What it supports. GeoWorldAD reports PDMS 91.0, compared with 90.3 for DVGT-2 and 90.4 for EponaV2. Its NC is 99.0 and EP is 85.9. However, iPad reports PDMS 91.7 with map and box supervision. The supported best-score statement is therefore restricted to the listed geometry and world-model comparison groups.
Where the evidence stops. The rows differ in modality, supervision, and architecture; this table does not isolate future geometry's causal effect. PDMS and its components are reported benchmark scores, and the TTC column must not be read as seconds of physical collision margin.
Table 2. The stronger v2 aggregate result still contains component-level tradeoffs. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the far-right EPDMS column and compare GeoWorldAD against DVGT-2 and EponaV2, then move left to NC, EP, and TTC. These columns connect the overall result to the paper's safety-and-progress argument. The remaining columns are retained exactly as printed; the source does not fully define every abbreviation or provide the complete EPDMS formula. Section 4.1 describes v2 as using reactive traffic and additional criteria compared with v1. Consequently, use this table for comparisons within the v2 navtest evaluation. Do not compare its aggregate score numerically with the v1 PDMS table as though the two metrics were identical. v2evaluation
What it supports. GeoWorldAD reaches 90.4 EPDMS, 0.8 points above DVGT-2 and 1.5 above EponaV2. Its NC, TTC, and EP values are also higher than those two rows. This is evidence of a better reported aggregate balance in the listed v2 comparison, not a claim that every measured driving behavior improves.
Where the evidence stops. GeoWorldAD does not dominate every component column, and no uncertainty across training seeds is reported. Reactive simulator traffic adds evaluation complexity, but these results remain simulator evidence rather than validation of deployment on public roads.
Figure 8. Decoded future geometry can be inspected separately from its contribution to planning. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the filmstrip on the left as the observed input sequence. The next three columns contain future RGB references, predicted depth, and GT depth, with rows labeled t+1 through t+4. The RGB references are not generated RGB predictions. Follow the central vehicle and surrounding street structure across corresponding future rows, and compare their broad boundaries with the depth maps. Section 3.2 explains that four latent future chunks cover two seconds and that a shared depth head supplies supervision. This example makes the decoded representation visible, but the planner itself consumes future tokens rather than requiring these color images during inference. future-visualsworldactiondepth
What it supports. The predicted maps show recognizable broad scene and vehicle structure across the illustrated future steps. This qualitatively supports the interpretation that the latent branch carries geometry-related future information. It does not provide a numerical estimate of depth accuracy or show which local prediction errors matter most for trajectory selection.
Where the evidence stops. There is no numerical color scale, per-horizon error, or uncertainty display. Dense predictions and sparse GT renderings have different visual coverage. Tables 5 and 7 evaluate video-depth reconstruction, so their numbers cannot be used as future-forecast accuracy for this figure.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM v2 closed-loop planning navtest split; reactive traffic and extended scoring as described in Section 4.1. | 90.4 EPDMS; 99.0 NC; 98.6 TTC; 89.1 EP. EPDMS ↑; NC ↑; TTC ↑; EP ↑ | DVGT-2: 89.6 EPDMS; EponaV2: 88.9. Highest EPDMS among listed methods, by 0.8 and 1.5 score points respectively. Different architectures and supervision prevent causal attribution from this leaderboard alone. v2evaluation |
| NAVSIM v1 closed-loop planning navtest split; four-second non-reactive simulation at 10 Hz. | 91.0. PDMS ↑ | DVGT-2: 90.3; EponaV2: 90.4; iPad: 91.7. Leads the listed world-model and geometry baselines, but not every table entry. Input modalities and auxiliary supervision vary across methods. v1evaluation |
| Future-geometry ablation GeoAD versus GeoWorldAD; separate NAVSIM v1 and v2 evaluations in Table 3. | v1: EP 82.6→85.9, PDMS 89.3→91.0. v2: EP 86.3→89.1, EPDMS 87.6→90.4. NC 98.9→99.0 in both. EP ↑; PDMS/EPDMS ↑; NC ↑ | GeoAD uses present geometry only. Progress gains exceed the small safety-score changes. The training schedule adds Stage 3 updates; an equally continued GeoAD control is not documented. future-ablationtraining |
| Geometry representation and supervision ablation NAVSIM v1 navtest; all variants exclude future tokens. | 84.2 scratch; 84.8 StreamVGGT plus reconstruction; 87.3 EgoStreamVGGT without auxiliary reconstruction; 89.3 with reconstruction. PDMS ↑ | Compare the last two rows for the reconstruction-supervision effect within EgoStreamVGGT. Supports the combined design, but coordinate choice and backbone adaptation are not fully separated. Comfort stays 99.9 across the final two rows. geometry-ablationtraining |
| Video-depth reconstruction OpenScene, nuScenes, KITTI; Tables 5 and 7 repeat results, referring to the StreamVGGT evaluation protocol without full split details. | EgoStreamVGGT: 0.141, 0.117, 0.077 respectively. Abs Rel ↓ | StreamVGGT: 0.236, 0.265, 0.173. All reported datasets improve. These values do not quantify future-depth forecasting, for which only qualitative examples are supplied. depthfuture-visuals |
| Camera-pose reconstruction nuScenes and OpenScene; Table 8 omits units and detailed evaluation settings. | EgoStreamVGGT: nuScenes 5.78/0.63/1.31; OpenScene 4.07/0.39/0.92. ATE ↓; RPE trans ↓; RPE rot ↓ | StreamVGGT: nuScenes 14.79/1.77/0.47; OpenScene 8.66/1.00/1.53. Trajectory and translation errors improve in both datasets, but nuScenes rotation error worsens. A blanket claim of uniformly better pose accuracy would be incorrect. pose |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Future geometry accompanies larger progress gains than safety-score changes. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat GeoAD as the present-geometry planner and GeoWorldAD as the model with the additional future-geometry branch. Read each benchmark group separately: v1 ends in PDMS, while v2 ends in EPDMS. Within either group, compare EP before looking at NC and TTC. This tests the authors' specific motivation that a planner already grounded in present geometry may be overly conservative. The table is concise enough to show that NC begins close to its maximum, so the small NC movement should be distinguished from the larger progress movement. Finally, consult the staged schedule on page 7 to understand how these two models were obtained. future-ablationtraining
What it supports. Ego progress rises by 3.3 points in v1 and 2.8 in v2. Aggregate scores rise from 89.3 to 91.0 PDMS and from 87.6 to 90.4 EPDMS. NC rises only from 98.9 to 99.0 in both evaluations. The pattern supports the paper's emphasis on progress rather than a large change in collision scores.
Where the evidence stops. Stage 3 trains the combined model for 64K further steps after GeoAD is obtained. The paper does not document a GeoAD control continued for the same budget. Thus the comparison motivates, but does not replace, a matched-training test of future information.
Table 4. Ego-aligned representations and reconstruction supervision work together in the present-only planner. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two leftmost columns together: they specify both the pretrained backbone and whether auxiliary 4D reconstruction is used during planner training. All four variants exclude latent future tokens, so this table concerns the foundation of GeoAD. The first two rows show why adding vanilla geometry pretraining alone is insufficient to explain the final result. The last two rows provide the closer comparison for reconstruction supervision within EgoStreamVGGT. Compare NC, DAC, TTC, and EP as well as PDMS; comfort does not change between these final two rows. The coordinate-system change is described in Section 3.1 and the ego-centric reconstruction losses in Appendix A.1. geometry-ablationbackbonereconstructiontraining
What it supports. The final configuration reaches 89.3 PDMS versus 87.3 without auxiliary reconstruction in EgoStreamVGGT. Vanilla StreamVGGT with reconstruction reaches only 84.8 and lowers several safety-related scores relative to scratch. These results support using appropriately adapted geometry with continuing supervision, rather than assuming any pretrained geometry features automatically help planning.
Where the evidence stops. Backbone adaptation and coordinate representation are not independently crossed in a complete factorial experiment. The table therefore does not isolate a pure coordinate-transform effect. Also, the prose's claim of improvement across all planning metrics is too broad: comfort stays at 99.9.
7. Analysis & limitations
7.1 What the evidence leaves open
The planner uses fixed-length clips. KV caching for continuous planning remains future work; a streaming reconstruction backbone does not establish streaming or real-time planning. limitations
Future geometry covers two seconds while trajectories cover four. Selected depth and reconstruction images cannot establish long-horizon accuracy, calibrated uncertainty, or physical driving safety. Reported tables provide no seed variance or confidence intervals. worldtrainingv1v2future-ablationreconstruction-visualsfuture-visuals
7.2 Questions for discussion
- Would progress gains survive an equally continued GeoAD baseline?
- Which future horizons and scene types drive the benefit?
- Does ego-coordinate alignment help independently of driving-domain finetuning?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the StreamVGGT initialization, four geometry datasets, NAVSIM splits and simulator targets, and the staged schedule. Exact image resolution, geometry-label preprocessing, numerical refinement weights, confidence regularization coefficient, and software versions are not specified in the supplied method/training appendix. backbonetrainingactionreconstruction
Proposed checks should compare future tokens against equally trained controls and separate ego-coordinate alignment from finetuning. Measure progress and safety together, retain per-scenario outcomes, and report variation across seeds rather than treating small score changes as established significance. future-ablationgeometry-ablationtraining
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does correctly ordered future information outperform equal training and capacity?
Reader-proposed check, not performed: begin all variants from the same GeoAD checkpoint and continue for the same 64K-step budget, with identical batches, seeds, optimizer settings, and reconstruction supervision. Compare the complete future branch against a GeoAD continuation and a branch-matched control that permutes the temporal order of future tokens before the planner reads them, while retaining future-depth training. Report NAVSIM v1 and v2 progress, aggregate scores, NC, TTC, and per-scenario collision outcomes separately. The prediction is that meaningful future geometry should outperform both controls beyond seed variation. If equal continuation closes the gap or temporal permutation has no effect, the mechanism claim needs narrowing. trainingworldactionfuture-ablation
Check 2: Does ego-coordinate supervision help independently of driving-domain adaptation?
Reader-proposed check, not performed: train anchor-frame and ego-frame versions from the same StreamVGGT initialization using the same dataset mixture, sample order, update counts, and planner architecture. Within each coordinate convention, toggle auxiliary reconstruction during planner training, keeping future tokens absent to match Table 4's setting. Evaluate NAVSIM v1 navtest PDMS, EP, NC, DAC, and TTC, plus reconstruction and pose errors under explicitly recorded splits and units. A coordinate-specific benefit should persist when pretraining exposure and reconstruction supervision are matched. If improvements arise only from adaptation or extra loss, the coordinate explanation is weakened; track nuScenes rotation error because the source reports a regression there. backbonetrainingreconstructiongeometry-ablationdepthpose
8.3 Reading coverage
Visual audit: Visually inspected the title and version block; Figures 1–9; Tables 1–8; method equations, inference statements, training schedule and hardware; evaluation descriptions; limitations; and Appendix A.1–A.4. All six final original crops were separately viewed. Figure 2's information flow was checked against Sections 3.1–3.3 and Eq. (3): geometry feeds both branches, future aggregation precedes final trajectory refinement, and the causal mask is defined in text rather than drawn. Pages 10–15 were read as text, covering the conclusion continuation and references, but were not part of the visual pass. Table 3's training-budget boundary uses inspected page 7; all appendix facts underlying interpretation and proposed checks are covered by the declared pages. Separate supplemental availability remains unverified.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 16, 17, 18, 19, 20, 21, 22. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations, version stamp, and abstract (p. 1)
- Sections 1–2.1: introduction and related work (pp. 1–4)
- Sections 3.1–3.3: geometry backbone, world model, and action model (pp. 4–6)
- Sections 4.1–4.3.3: datasets, metrics, training, results, and ablations (pp. 6–9)
- Sections 5–6: limitations and conclusion (pp. 9–10)
- References (pp. 10–15)
- Appendix A.1: reconstruction objectives (p. 16)
- Appendix A.2: collision metrics (p. 17)
- Appendix A.3.1–A.3.2: aggregation and geometry evaluation (pp. 17–19)
- Appendix A.4.1–A.4.2: reconstruction and future-depth visualizations (pp. 20–22)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The complete 22-page supplied PDF text was read through all six chunks. Text extraction does not reconstruct figure images; the retained PDF was separately rendered and visually inspected for every figure and table.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- This report reads the supplied arXiv:2607.17521v2 PDF, stamped 23 Jul 2026. Its title and all twelve authors match the catalog. No earlier version or revision history was supplied, so differences from v1 cannot be assessed.
- Code was not supplied or inspected, and experiments were not reproduced. References were read as part of this paper, not independently verified.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author block, affiliations, and arXiv margin stamp
The title is GeoWorldAD: Geometry World Action Model for Autonomous Driving. The twelve authors and three affiliations match the catalog after name-order normalization. The PDF stamp identifies arXiv:2607.17521v2 [cs.RO], 23 Jul 2026; no earlier revision is supplied.
Go to primary source ↓problemPDF pp. 1–3, Abstract and Section 1; Figure 1 on p. 2
The paper motivates ego-aligned geometry for spatial constraints and future geometry for less conservative planning. Figure 1 contrasts modular, geometry-action, and video-world-action pipelines with the proposed geometry-based pipeline.
Go to primary source ↓backbonePDF pp. 4–5, Figure 2, Section 3.1, Eqs. (1)–(2)
DINOv2 patch features enter a 24-block StreamVGGT decoder. Layers 4, 11, 17, and 23 supply geometry tokens. EgoStreamVGGT changes point maps to each timestep's ego-camera frame and camera poses to adjacent-frame transformations.
Go to primary source ↓worldPDF p. 5, Section 3.2, Eqs. (3)–(5); Figure 2 on p. 4
Four future chunks over two seconds each contain 64 latent tokens. At four geometry scales, future queries cross-attend to present geometry and ego embeddings, then use causal self-attention over current and earlier future chunks. A shared DPT head decodes future geometry for depth supervision, whose loss does not update that head.
Go to primary source ↓actionPDF pp. 5–6, Section 3.3, Eqs. (6)–(9); Figure 2 on p. 4
The planner uses 64 proposals, eight waypoints, and embedding dimension 1024. Four present-geometry stages precede one latent-future stage. A shared MLP predicts position and heading, minimum-over-proposals L1 losses supervise each stage, and an MLP scoring head learns simulator-derived scores with binary cross-entropy. Auxiliary geometry decoders are unnecessary for planning inference.
Go to primary source ↓reconstructionPDF p. 16, Appendix A.1, Eqs. (10)–(13)
Reconstruction combines Huber camera-parameter loss with confidence-weighted depth and point-map errors and gradient matching. Negative log-confidence terms discourage trivial low confidence. The appendix reiterates ego-camera point maps and adjacent-frame camera transformations.
Go to primary source ↓trainingPDF p. 7, Section 4.2, Stages 1–3 and final paragraph
Geometry pretraining uses 23K steps and OpenScene:nuScenes:ParallelDomain:RealDriveSim sampling of 10:10:1:1. Stage 2 trains future depth on OpenScene for 47K steps and GeoAD on NAVSIM navtrain for 32K steps. Stage 3 zero-initializes the future block's output projection and trains the combined model for 64K further steps. AdamW uses global batch 64 on 32 NVIDIA H20 GPUs, cosine learning-rate scheduling, 1e-4 in Stages 1–2 and 1e-5 in Stage 3.
Go to primary source ↓evaluationPDF p. 6, Section 4.1 and Eq. (8); PDF p. 17, Appendix A.2
NAVSIM planning uses simulator metrics. The described v1 protocol is four-second non-reactive simulation at 10 Hz with PDMS; v2 adds reactive traffic and extended criteria with EPDMS. NC concerns at-fault collisions; TTC concerns collision risk margins. Tables report scores, not TTC in seconds.
Go to primary source ↓v1PDF p. 7, Table 1, GeoWorldAD, DVGT-2, EponaV2, and iPad rows
On NAVSIM v1 navtest, GeoWorldAD has NC 99.0, DAC 97.8, TTC 95.8, comfort 99.9, EP 85.9, and PDMS 91.0. DVGT-2 and EponaV2 score 90.3 and 90.4 PDMS; iPad scores 91.7 with map and box supervision. The caption restricts the best-score claim to competing world-model and geometry methods.
Go to primary source ↓v2PDF p. 8, Table 2, GeoWorldAD, DVGT-2, and EponaV2 rows; Section 4.3.1
On NAVSIM v2 navtest, GeoWorldAD reports EPDMS 90.4, NC 99.0, TTC 98.6, and EP 89.1; DVGT-2 reports EPDMS 89.6 and EponaV2 88.9. GeoWorldAD does not lead every component metric.
Go to primary source ↓future-ablationPDF p. 8, Table 3 and Section 4.3.2
GeoAD versus GeoWorldAD gives v1 EP 82.6 versus 85.9 and PDMS 89.3 versus 91.0; v2 EP 86.3 versus 89.1 and EPDMS 87.6 versus 90.4. NC changes from 98.9 to 99.0 in both versions; TTC changes from 95.7 to 95.8 in v1 and 98.3 to 98.6 in v2.
Go to primary source ↓geometry-ablationPDF p. 9, Table 4 and Section 4.3.3
Present-geometry-only NAVSIM v1 navtest variants score 84.2 PDMS from scratch, 84.8 with StreamVGGT and reconstruction supervision, 87.3 with EgoStreamVGGT without auxiliary reconstruction, and 89.3 with EgoStreamVGGT plus reconstruction. Comfort remains 99.9 across the final two rows despite prose claiming gains across all metrics.
Go to primary source ↓depthPDF p. 9, Table 5; PDF p. 18, Table 7 and Appendix A.3.2
Tables 5 and 7 repeat the same video-depth results. StreamVGGT to EgoStreamVGGT Abs Rel changes are 0.236 to 0.141 on OpenScene, 0.265 to 0.117 on nuScenes, and 0.173 to 0.077 on KITTI. Threshold accuracy increases on all three datasets. These are reconstruction results, not horizon-wise future-depth prediction errors.
Go to primary source ↓aggregationPDF p. 17, Figure 3 and Appendix A.3.1; PDF p. 18, Table 6 and continuation of A.3.1
The 24-layer/one-iteration, one-layer/four-iteration, and four-layer/four-iteration designs score 87.6, 88.2, and 89.3 PDMS respectively. The last design leads NC and TTC, but the one-layer/four-iteration variant has higher EP, 82.9 versus 82.6. The table does not restate the evaluation split.
Go to primary source ↓posePDF p. 19, Table 8; PDF pp. 18–19, Appendix A.3.2
On nuScenes, StreamVGGT to EgoStreamVGGT changes ATE 14.79 to 5.78, RPE trans 1.77 to 0.63, and RPE rot 0.47 to 1.31. OpenScene values change 8.66 to 4.07, 1.00 to 0.39, and 1.53 to 0.92. The table does not provide units or detailed split/alignment settings.
Go to primary source ↓limitationsPDF p. 9, Section 5
The authors state that trajectory planning operates on fixed-length clips despite the reconstruction backbone supporting continuous streams; KV-cached streaming planning is future work.
Go to primary source ↓reconstruction-visualsPDF pp. 20–21, Appendix A.4.1, Figures 4–7
Four qualitative comparisons show input video sequences and StreamVGGT, EgoStreamVGGT, and GT reconstruction/pose views. They provide selected visual examples without per-example numerical errors.
Go to primary source ↓future-visualsPDF p. 22, Appendix A.4.2, Figures 8–9
Each example juxtaposes input video, future RGB reference images, predicted depth, and GT depth for t+1 through t+4. No numerical depth scale, horizon-wise error table, or uncertainty visualization is supplied with these examples.
Go to primary source ↓8.5 Primary sources
GeoWorldAD: Geometry World Action Model for Autonomous Driving ↗
PDF · 9,011 extracted words
Source fingerprint
7bceb7b2414049137fb3bcec5344fd011b47db89e91d9c5f452475d1f37056c1