MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving
1. Paper overview
In one sentence: MindDrive predicts action-conditioned BEV futures to refine trajectory anchors, then uses a VLM scorer to select a plan, with stronger benchmark scores but an incomplete training recipe. e-identitye-probleme-overviewe-navteste-navharde-componentse-implementation
| At a glance | What to know |
|---|---|
| Research problem | Author claim A strong generator can lose its best candidate through weak selection, while a strong selector cannot recover an absent candidate. MindDrive addresses this imbalance by combining hypothetical future simulation with learned safety, compliance, comfort and efficiency scoring. e-problem |
| Core mechanism | |
| A key reported result | NAVSIM-v1 Navtest planning: 88.9 PDMS ↑. Non-reactive evaluation; MindDrive uses camera+LiDAR; Table 1 states a shared ResNet-34 backbone. WoTE and DIVER: 88.3; DiffusionDrive: 88.1; Human: 94.8. A 0.6-point gain over the strongest listed learned baselines. NC 98.4 and comfort 99.9 do not lead every component. e-protocole-navtest |
| Reading caution | Reader analysis Reporting is internally inconsistent: Navhard final EPDMS differs between table and prose; prose also swaps Stage-I TTC/HC values. Tables 5 and 10 disagree on EP/EC for the 72.7 configuration. These discrepancies remain unresolved. e-navharde-vlm-ablation |
Core contributions
- Source description
VLoE aligns scene, trajectory and prompt embeddings, then converts a VLM score-token representation into numerical trajectory evaluations. e-alignmente-critic
Figure 2. Future context reaches both candidate generation and candidate evaluation. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the upper left: image and LiDAR backbones form BEV features, while ego status and anchor intent supply the ego representation. FaTG builds a separate scene variant for each intent and evolves future features before decoding candidate trajectories. The current/future join indicates feature combination; Eq. (8) specifically uses concatenation. Follow the downward scene-token branch and the trajectory-token branch into LaST-Former, where they meet the prompt embedding. VLM-Critic and the score head then supply the safety, compliance, comfort and progress evaluations shown at the right. The final plan depends on candidate construction and composite selection together. e-overviewe-wame-decodere-alignmente-critice-implementation
What it supports. The architecture makes predicted scene context available at inference in two places: anchor refinement and candidate scoring. This supports reading MindDrive as a connected generation–selection pipeline. It does not, by itself, establish a single network that jointly predicts world states and executed actions.
Where the evidence stops. The green pre-transition block says Scene Decoder, but Figure 3 and Eq. (5) identify an encoder. Use that formulation. The perception snowflake suggests freezing, yet no legend or training schedule establishes its exact scope.
2. Motivation
2.1 The problem and the proposed response
A strong generator can lose its best candidate through weak selection, while a strong selector cannot recover an absent candidate. MindDrive addresses this imbalance by combining hypothetical future simulation with learned safety, compliance, comfort and efficiency scoring. e-problem
2.2 What this reading follows
Planning has two distinct failure points: the useful maneuver may never enter the candidate set, or the selector may rank it poorly. MindDrive links a future-aware generator to a vision–language evaluator so that both stages can use predicted scene context. Follow the figures from sensor fusion to hypothetical futures, anchor refinement and score-token readout. Then examine the tables as separate evaluation protocols: ordinary Navtest, synthetic-future Navhard, and component removal. The paper reports improvements across these settings, but its strongest interpretation requires care: numerical inconsistencies, ambiguous diagram details and missing training specifications remain in the supplied v2 edition. e-identitye-probleme-overviewe-navteste-navharde-componentse-implementation
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded catalog classification is unassigned. Architecture evidence favors a modular world-model-assisted planner: action-conditioned future features feed a separate anchor decoder and VLM evaluator. The paper does not establish a single shared world/action predictor merely by calling the framework all-in-one. It uses neither an explicit inverse-dynamics action extractor nor only a training-time future loss. e-overviewe-wame-decodere-critic
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn an anchor into a hypothesis about the scene
The anchor is more than a trajectory that will later receive a score. MindDrive first derives anchors from expert trajectories in the training data, encodes each anchor together with ego status, and injects that action token into the current BEV feature map. Bilinear injection makes the conditioning spatial: the token contributes at grid locations along the anchor. Consequently, the same observation can yield several scene variants corresponding to different intended maneuvers. WAM forecasts future features for these variants, and the decoder then uses those forecasts to modify the original anchors through attention and offset prediction. This ordering matters: a future is conditioned on initial intent before the final trajectory is refined. The source does not specify re-simulating every refined candidate to enforce consistency with its original conditioning anchor. e-anchorse-wame-decoder
Figure 3. A learned BEV transition is trained against simulator-derived scene targets. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the central column downward from the green current scene-variant features to the blue future features. The caption and equations assign spatial modeling to a Transformer encoder and decoder, with two Mamba blocks between them for temporal propagation. Eq. (6) defines a residual update around each pre-normalized Mamba block. The dashed training column is separate: a simulator supplies current and future BEV semantic map features, while the black Sample Projection arrows connect predictions with that supervision. The right inset expands the Mamba block's linear, convolution and state-space processing. It is a structural diagram, not an empirical visualization of forecast accuracy. e-anchorse-wame-decoder
What it supports. The model predicts scene features conditioned on an initial ego intent, providing context for the later trajectory decoder. The simulator in the training branch supplies supervision; the deployed planning path instead uses the learned feature transition. This distinction prevents confusing data-generation supervision with an online external simulator.
Where the evidence stops. The drawn shortcuts do not clearly match Eq. (6): the long left shortcut reaches the second Mamba block rather than depicting a separate addition after the first. Follow the explicit per-block residual equations; the figure alone leaves implementation ambiguity.
5.2 Separate numerical evaluation from verbal reasoning
LaST-Former makes structured driving information usable by the VLM through embeddings rather than a complete textual description of each scene. Current and future scene features are compressed; trajectory embeddings are attended by learnable metric queries; the resulting contexts replace multimodal sentinels in a prompt. A dedicated score token then provides the hidden feature read by a numerical head. Reader analysis: this design can transfer useful semantic representations into ranking without producing a visible chain of reasoning. The reported VLM ablations are consistent with a benefit from the added evaluator, but they do not establish whether that benefit specifically depends on meaningful language, extra capacity, or training differences. Nor do they validate the faithfulness of a human-readable rationale. Such claims would require additional controlled evaluations beyond the score improvements reported here. e-alignmente-critice-vlm-ablation
Figure 4. The VLM is used as a learned trajectory evaluator. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the yellow trajectory tokens and green/blue scene tokens down their separate branches. Scene tokens receive spatial convolution before projection and cross-attention; the trajectory branch projects candidates and uses metric queries described in Eq. (13). The diagram marks two cross-attention repetitions in each branch. Sentinel insertion places the resulting contexts into the language stream at the corresponding <scene> and <traj> placeholders. The red score-token arrow introduces the dedicated evaluation token. At the bottom, critic hidden states converge on the score latent feature, which the score head converts into numerical evaluations. Eqs. (14)–(15) specify the score-token readout. e-alignmente-critice-overview
What it supports. The relevant output is an evaluation feature and numerical score, rather than a displayed verbal rationale. The paper's use of a language backbone therefore establishes a multimodal scoring mechanism; it does not demonstrate that a human can inspect a faithful explanation for every chosen trajectory.
Where the evidence stops. The exact attention mask and complete evaluation prompt are absent. Token-count notation also shifts between N and K in Section 3.3.1. Figure 4's schematic alignment is insufficient to reconstruct token packing, candidate-score indexing or supervision.
5.3 Compare only scores from the same evaluation contract
Start by fixing the split, sensors and stage definition. Navtest v1 uses PDMS in a non-reactive simulation setting; Navtest v2 reports Stage-I EPDMS because it lacks the synthetic observations needed for Stage II. Navhard's two-stage table instead aggregates evaluation before and after synthetic future observations. Its final score cannot be interpreted on the same basis as the separate camera-plus-LiDAR Stage-I robustness table. Reader analysis: preserving these distinctions changes what the headline numbers establish. A high multimodal Stage-I score does not remove the Stage-II degradation visible in Table 3. Likewise, the component ablation concerns Navtest v1 only. Finally, table/prose conflicts should remain visible rather than being reconciled by choosing whichever value best supports the paper's narrative. e-protocole-navharde-robustnesse-components
5.4 Training and inference
During training
Figure 3 specifies simulator-generated current/future BEV semantic map features as WAM supervision. Tiny-LLaVA-1B is fine-tuned using LoRA. No complete trajectory/critic losses, joint-training stages or loss weights are given. e-wame-implementatione-critic
Figure 2 places a snowflake beside perception, suggesting freezing, but supplies no legend or freeze/pretraining schedule. This visual cue does not establish which parameters remain trainable throughout training. e-overviewe-implementation
During inference
The learned rollout supplies future context during planning; it is not solely an auxiliary training target. Candidate refinement precedes VLM scoring and final selection. The training simulator is distinct from this learned rollout, and no real-vehicle actuation experiment is reported. e-wame-decodere-critice-protocol
5.5 Implementation flow
- Encode observations and initial intent
Separate ResNet-34 modality features are fused through BEV cross-attention. Self-attention encodes each K-Means anchor; concatenation with ego status and an MLP produce its action token. e-overviewe-anchorse-implementation
- Inject actions into space
Anchor coordinates are projected onto the BEV grid. Each action token is added to four neighboring pixels with bilinear weights, producing one hypothetical scene feature map per intent. e-anchors
- Predict future context
A Transformer encodes spatial interactions; two pre-normalized Mamba blocks with residual updates evolve the latent; a Transformer decoder reconstructs future scene features. Recurrent predictions provide multiple horizons. e-wame-wam-ablation
- Refine and evaluate candidates
The action token queries concatenated current/future features to predict anchor offsets. LaST-Former compresses scene tokens and projects trajectory tokens, using metric queries for trajectory cross-attention. Context embeddings replace <scene>/<traj> sentinels in the prompt. e-decodere-alignment
- Select a plan
Causal VLM processing yields the <score_feature> hidden state. A score head produces evaluations used in composite trajectory selection; the paper does not provide the exact aggregation formula. e-overviewe-critic
6. Experiments & results
MindDrive couples action-conditioned BEV scene prediction with anchor refinement and a VLM trajectory scorer. Its main contribution is the connection between future-aware candidate generation and selection. NAVSIM scores support planning gains, while incomplete training specifications and inconsistent result reporting limit reproducibility.
6.1 Read the original evidence
Table 1. The best listed learned PDMS does not imply a win on every component. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the shaded PDMS column first, then return to the component columns to see how the aggregate result is supported. C denotes camera and L denotes LiDAR; MindDrive uses both. NC measures no at-fault collision, DAC drivable-area compliance, EP ego progress, TTC time-to-collision, and Comf. comfort. The caption states a shared ResNet-34 backbone, although sensor modalities differ between rows. Compare MindDrive with the strongest camera-plus-LiDAR baselines near the bottom. Keep the Human row visible as a separate reference. These values belong to NAVSIM-v1 Navtest's non-reactive evaluation, not the synthetic-future Navhard protocol. e-navteste-protocol
What it supports. MindDrive reports PDMS 88.9, versus 88.3 for WoTE and DIVER and 88.1 for DiffusionDrive. Its 97.0 DAC, 82.8 EP and 95.1 TTC lead the listed learned rows. The 0.6-point advantage over WoTE/DIVER is an aggregate planning-score improvement, without reported seed uncertainty.
Where the evidence stops. Section 4.3.1 says MindDrive surpasses previous methods across all components, but NC is 98.4 versus 98.5 for WoTE/DIVER and comfort is 99.9 versus several 100 entries. Human PDMS remains higher at 94.8.
Table 3. Read the merged EPDMS cell as a two-stage aggregate. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each method occupies two rows. Stage I evaluates a plan from real initial observations; Stage II repeats evaluation on synthetic future observations. The rightmost merged cell is the final Gaussian-weighted EPDMS, not the score of the second row alone. Compare stage-specific columns before interpreting that aggregate. NC is collision avoidance; DAC, DDC and TLC concern drivable area, driving direction and traffic lights; EP is progress; TTC concerns time to collision; LK, HC and EC concern lane keeping and comfort. The caption states a shared ResNet-34 backbone. Section 4.4 separates this setup from multimodal Stage-I-only robustness evaluation. e-navharde-protocole-robustness
What it supports. The table reports final EPDMS 30.5 for MindDrive and 27.1 for GuideFlow. MindDrive's Stage-II DAC is 79.1, but NC falls from 96.1 to 82.6 and LK from 94.4 to 49.2 across stages. The aggregate lead coexists with large weaknesses under synthetic future observations.
Where the evidence stops. Section 4.3.3 instead reports EPDMS 30.9 and misstates Stage-I TTC/HC as 99.3/94.4; the table gives 95.6/97.6. These are unresolved source conflicts. Table 5's multimodal 72.7 Stage-I score belongs to another configuration.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM-v1 Navtest planning Non-reactive evaluation; MindDrive uses camera+LiDAR; Table 1 states a shared ResNet-34 backbone. | 88.9 PDMS ↑ | WoTE and DIVER: 88.3; DiffusionDrive: 88.1; Human: 94.8. A 0.6-point gain over the strongest listed learned baselines. NC 98.4 and comfort 99.9 do not lead every component. e-protocole-navtest |
| NAVSIM-v2 Navtest planning Same Navtest scenarios; real observations only, without Stage II. | 84.2 Stage-I EPDMS ↑ | ARTEMIS: 83.1. A 1.1-point gain under the expanded metric; not a two-stage Navhard score. e-protocole-navtest-v2 |
| NAVSIM-v2 Navhard two-stage planning Real initial observations plus synthetic future observations; Gaussian-weighted stage aggregation; shared ResNet-34. | 30.5 in Table 3; 30.9 in Section 4.3.3 (unresolved). Final EPDMS ↑ | GuideFlow: 27.1 in Table 3. The table supports a 3.4-point advantage. Stage-II degradation remains substantial; the multimodal Stage-I results use a different sensor configuration. e-navharde-robustnesse-protocol |
| NAVSIM-v1 Navsafe robustness Safety-critical subset; Table 4 baselines evaluated using official weights. | 67.5 PDMS ↑ | DiffusionDrive: 66.1; WoTE: 65.3. Higher aggregate score, but WoTE has higher NC and TTC; this is not a safety guarantee. e-robustness |
| NAVSIM-v2 Navhard multimodal robustness Camera+LiDAR, Stage I without synthetic scenarios; Table 5. | 72.7 Stage-I EPDMS ↑ | WoTE: 66.7. Separate protocol from Table 3; the 6.0-point difference cannot be compared directly with final EPDMS. e-robustness |
| FaTG/VLoE component ablation NAVSIM-v1 Navtest, Table 6. | Neither: 84.1; FaTG only: 86.6; VLoE only: 87.7; both: 88.9. PDMS ↑ | Combined system gains 4.8 points over neither module. Both components help within this setup; model capacity and optimization are not isolated. e-components |
| WAM architecture and horizon ablations NAVSIM-v1 Navtest, Tables 7–8; separate architecture and temporal comparisons. | Mamba-only: 85.6; hybrid variants: 88.1/88.5/88.9. Direct 0→4 s: 87.5; recurrent 0→2→4 s: 88.9. PDMS ↑ | The best hybrid uses Transformer–2×Mamba–Transformer. Supports this planning configuration; does not directly measure reduced future-prediction error. e-wam-ablation |
| VLM ablation on Navsafe NAVSIM-v1 Navsafe, Table 9. | With VLM: 67.5 PDMS ↑ | Without VLM: 64.5. A 3.0-point gain; the experiment does not isolate language semantics from added capacity. e-vlm-ablation |
| VLM ablation on Navhard NAVSIM-v2 Navhard Stage I, Table 10; no synthetic scenarios. | With VLM: 72.7 Stage-I EPDMS ↑ | Without VLM: 68.2. The 4.5-point aggregate gain is reported, but EP/EC differ from Table 5's MindDrive row. e-vlm-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 6. Generation and evaluation each improve this ablation baseline. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the two checkmark columns as a small factorial comparison. The first row disables both FaTG and VLoE. The second adds only future-aware generation; the third adds only the VLM-oriented evaluator; the last includes both. All rows use Navtest under NAVSIM-v1 metrics. Read PDMS vertically, then compare DAC, EP and TTC to see that gains span several components. To assess FaTG within the complete system, compare the third and fourth rows; to assess VLoE within that system, compare the second and fourth. This avoids treating each isolated gain over the first row as automatically additive. e-components
What it supports. PDMS rises from 84.1 to 86.6 with FaTG alone and to 87.7 with VLoE alone; both reach 88.9. The combined increase is 4.8 points. Each component improves the other component's configuration, supporting their usefulness together within the reported setup.
Where the evidence stops. The combined gain is smaller than the sum of the two isolated gains. No matched-capacity baseline, seed variability, candidate oracle score or ranking-error analysis is provided, so the table does not isolate a unique causal mechanism.
7. Analysis & limitations
7.1 What the evidence leaves open
Reporting is internally inconsistent: Navhard final EPDMS differs between table and prose; prose also swaps Stage-I TTC/HC values. Tables 5 and 10 disagree on EP/EC for the 72.7 configuration. These discrepancies remain unresolved. e-navharde-vlm-ablation
Non-reactive and synthetic-observation benchmarks do not establish interactive road safety. Figure 5 supplies two qualitative comparisons, without a failure distribution. No latency, seed uncertainty, world-prediction accuracy or human-alignment study is reported; reinforcement learning is future work. e-protocole-qualitativee-componentse-wam-ablation
Diagram and notation ambiguities matter: Figure 2's first scene block says Decoder, whereas Eq. (5)/Figure 3 specify Encoder; Figure 3's shortcut endpoints do not clearly reproduce Eq. (6). N/K token-count notation also shifts in LaST-Former. e-overviewe-wame-alignment
7.2 Questions for discussion
- Do gains come from better candidate coverage, better ranking, or both?
- Does the critic require meaningful language and action-conditioned futures when candidate sets and compute are controlled?
8. Reproducibility audit
8.1 Requirements and known gaps
Use the specified three-view 256×1024 image composite, 64 m×64 m LiDAR region, ResNet-34 fusion, training-derived anchors and Tiny-LLaVA-1B with LoRA. Keep Navtest, Navsafe and each Navhard sensor/stage protocol separate. e-anchorse-implementatione-protocole-robustness
The paper leaves anchor count, BEV resolution, losses and scoring labels, full prompt, metric weights, mask layout, optimizer, training duration, LoRA configuration, dataset counts, seeds and hardware unspecified. These must be resolved before claiming faithful reproduction. e-anchorse-wame-alignmente-critice-implementatione-protocol
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does action-conditioned forecasting improve candidate quality?
Reader-proposed check: fix training scenes, anchors, candidate count, perception initialization and evaluator. Compare the normal WAM with an equal-budget model whose action tokens are shuffled among anchors during both training and evaluation, preserving token statistics. Also compare direct 0→4 s with recurrent 0→2→4 s forecasts at matched compute. Measure future BEV prediction error, the best simulator-scored candidate, and the selected candidate's PDMS on Navtest v1. If correct conditioning improves neither prediction accuracy nor candidate quality across repeated seeds, the claimed what-if mechanism is weakened even if total PDMS changes. e-anchorse-wame-decodere-wam-ablation
Check 2: Does meaningful language improve ranking on identical candidates?
Reader-proposed check: cache identical FaTG candidates and scene features, then train scorers with the meaningful evaluation prompt, a token-length-matched neutral prompt, and a capacity/compute-matched nonlanguage scoring control. Keep labels, data and optimization budget fixed; disclose the missing score-target and aggregation recipe before testing. On Navsafe, compare selected PDMS, candidate-pair ranking accuracy and each component's calibration, holding the oracle candidate score constant. A reproducible advantage for meaningful prompts would support a language-specific contribution; no advantage would favor capacity or feature-processing explanations. These checks are proposals, not executed reproductions. e-alignmente-critice-vlm-ablatione-components
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page; Figure 1; all method pages and Figures 2–4; all ten quantitative tables, their captions and surrounding result explanations; and Figure 5 with the conclusion. All six final original crops were individually viewed. Diagram labels, residual shortcuts, sentinel flow and score-token readout were cross-checked with the text and equations. Pages 11–13 are included for uncropped robustness results, ablations, discrepancies and proposed-check premises; pages 9–10 cover implementation details. All six text chunks, including references through page 16, were read. Page 3 and reference-only pages 14–16 were not visually inspected; no retained scientific claim relies on those page images. No separate appendix or supplement was supplied.
PDF pages inspected for this edition: 1, 2, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Work (2.1–2.3)
- 3. Methodology (3.1–3.3.2; Eqs. 1–15)
- 4. Experiments (4.1–4.6; Tables 1–10)
- 5. Conclusion
- Acknowledgments
- References (pp. 13–16)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied edition is arXiv:2512.04441v2, dated 8 December 2025. Its title and all eleven authors match the catalog; the catalog submission date is 4 December 2025. The earlier edition was not supplied or compared.
- Text extraction does not reconstruct figure images; this limitation was addressed by inspecting the original PDF figures and tables.
- Separate supplemental material availability has not been fully verified. No separate supplement was supplied; no appendix occurs in this PDF.
- Code was not inspected and experiments were not reproduced. References were read as part of the paper, without inspecting their external sources.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin
The title and eleven authors match the catalog. The supplied PDF identifies arXiv:2512.04441v2, 8 December 2025; seven affiliations are printed.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Introduction; Figure 1
The authors motivate balancing future-aware candidate generation with multi-objective candidate selection.
Go to primary source ↓e-overviewPDF pp. 4–5, Section 3.1 and Figure 2
Camera/LiDAR perception and ego intent feed FaTG; scene, trajectory and language tokens feed VLoE. Figure 2 labels the pre-transition block Scene Decoder and shows an unexplained perception snowflake.
Go to primary source ↓e-anchorsPDF pp. 5–6, Section 3.2.1, Eqs. (1)–(4)
K-Means over training expert trajectories yields N anchors. Encoded anchors and ego status produce action tokens, bilinearly injected into BEV features to form scene variants.
Go to primary source ↓e-wamPDF p. 6, Figure 3/caption and Eqs. (5)–(7)
A Transformer encoder, two pre-normalized residual Mamba blocks and Transformer decoder predict future scene features. Simulator-generated current/future BEV semantic map features supervise training. Eq. (6) specifies one residual update per Mamba block; drawn shortcuts are ambiguous.
Go to primary source ↓e-decoderPDF p. 7, Section 3.2.2, Eqs. (8)–(9)
Each action token queries concatenated current/future scene features through cross-attention; the offset head refines its trajectory anchor.
Go to primary source ↓e-alignmentPDF pp. 7–8, Section 3.3.1, Eqs. (10)–(13) and Figure 4
Scene features undergo Conv2D, projection and cross-attention; projected trajectories are attended by metric queries. Scene/trajectory contexts replace sentinel placeholders in the language sequence. N and K are used inconsistently.
Go to primary source ↓e-criticPDF p. 8, Section 3.3.2, Eqs. (14)–(15) and Figure 4
Causal VLM processing produces a score-token hidden state, passed to a numerical score head. Exact mask entries, complete prompt, score supervision and objective aggregation are not specified.
Go to primary source ↓e-implementationPDF pp. 9–10, Section 4.2
Three front camera views form a 256×1024 composite; LiDAR covers 64 m×64 m. ResNet-34 features fuse through BEV cross-attention. Tiny-LLaVA-1B is adapted with LoRA; no complete optimization or compute recipe is provided.
Go to primary source ↓e-protocolPDF pp. 8–9, Section 4.1; p. 11, Section 4.3.2
NAVSIM uses OpenScene/nuPlan data. V1 uses non-reactive PDMS; v2 adds compliance/comfort metrics. Navtest reports Stage I only. Navhard uses real observations then synthetic future observations with Gaussian-weighted aggregation.
Go to primary source ↓e-navtestPDF p. 9, Table 1, MindDrive/WoTE/DIVER/DiffusionDrive/Human rows; p. 11, Section 4.3.1
PDMS is 88.9 for MindDrive, 88.3 for WoTE/DIVER, 88.1 for DiffusionDrive and 94.8 for Human. MindDrive NC 98.4 and comfort 99.9 do not lead all rows, contrary to the prose's blanket claim.
Go to primary source ↓e-navtest-v2PDF p. 10, Table 2, MindDrive and ARTEMIS rows and caption
Navtest Stage-I EPDMS is 84.2 versus ARTEMIS 83.1. MindDrive DDC/LK/EC are 99.3/96.5/96.8. The PDMS column repeats Table 1.
Go to primary source ↓e-navhardPDF p. 10, Table 3 and caption; p. 11, Section 4.3.3
Table 3 gives final EPDMS 30.5 versus GuideFlow 27.1; prose says 30.9. MindDrive Stage-I TTC/HC are 95.6/97.6 in the table, versus 99.3/94.4 in prose. Stage-II NC/LK are 82.6/49.2.
Go to primary source ↓e-robustnessPDF p. 11, Tables 4–5 and Section 4.4; p. 12, opening continuation
Navsafe PDMS is 67.5 versus DiffusionDrive 66.1. Multimodal Navhard Stage-I EPDMS is 72.7 versus WoTE 66.7. Table 4 baselines use official weights. The text separates the sensor configuration from Table 3 and states Stage II uses camera-only inputs.
Go to primary source ↓e-componentsPDF p. 12, Section 4.5.1 and Table 6, four FaTG/VLoE rows
Navtest v1 PDMS: neither module 84.1; FaTG only 86.6; VLoE only 87.7; both 88.9. No seed variability or capacity-matched controls are reported.
Go to primary source ↓e-wam-ablationPDF p. 12, Section 4.5.2 and Tables 7–8
Mamba-only PDMS is 85.6; hybrid variants score 88.1, 88.5 and 88.9. Direct 0→4 s prediction scores 87.5 versus 88.9 for 0→2→4 s recurrence. These are planning metrics, not measured rollout errors.
Go to primary source ↓e-vlm-ablationPDF p. 12, Table 9; p. 13, Table 10 and Section 4.5.3; p. 11, Table 5
Adding the VLM changes Navsafe PDMS 64.5→67.5 and Navhard Stage-I EPDMS 68.2→72.7. Table 10's With VLM EP/EC are 80.7/71.0, whereas Table 5's MindDrive row gives 83.7/71.1.
Go to primary source ↓e-qualitativePDF p. 13, Section 4.6, Figure 5 and Section 5
Two Navsafe examples compare intersection and dense-traffic plans with TransFuser. Reinforcement learning in the world–VLM loop is proposed as future work.
Go to primary source ↓8.5 Primary sources
MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Driving ↗
PDF · 9,584 extracted words
Source fingerprint
8e88ab1750dace5965a6bb683fad8ac5e020c0f21fe48ba0b5569bf1212c1bee