DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
1. Paper overview
In one sentence: Shared vision-language features support an action-conditioned BEV world model whose rewards refine a driving planner, with benchmark gains that still depend on staged training and incompletely specified inference details. e-motivatione-pipelinee-stage1e-stage2e-stage3e-navsim1e-nuscenes
| At a glance | What to know |
|---|---|
| Research problem | Reader analysis A planner can share features with a world model without evaluating what its proposed actions would cause. The authors target this gap using action-conditioned BEV imagination, while avoiding pixel-level video rollouts. Their claims about internalizing physical laws are a motivation, rather than a separately validated capability. e-motivatione-stage1e-stage2 |
| Core mechanism | Source description A common InternVL3-2B representation feeds a trajectory decoder and a BEV denoiser, linking planning supervision with future semantic-map supervision. e-pipelinee-stage1 |
| A key reported result | NAVSIMv1 trajectory planning: 91.3 PDMS ↑. Paper's benchmark comparison, three-camera model; exact split identifier unspecified. The protocol is non-reactive simulation, although tables call it closed-loop. DriveVLA-W0 90.2; WoTE 88.3; human reference 94.8. Highest listed learned-method score, a 1.1-point gain over DriveVLA-W0. Sensors differ across rows; this is not a controlled architecture-only comparison. e-navsim1e-protocol |
| Reading caution | Reader analysis The reported evidence is short-horizon benchmark evaluation. NAVSIMv1 is described as non-reactive and NAVSIMv2 as pseudo-simulation. Selected trajectory visualizations do not establish reactive-agent behavior, real-vehicle safety, or general physical-law understanding. e-protocole-qualitative |
Core contributions
- Source description
A common InternVL3-2B representation feeds a trajectory decoder and a BEV denoiser, linking planning supervision with future semantic-map supervision. e-pipelinee-stage1
Figure 2. Shared features connect separate planning and imagination modules through a progressive training schedule. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the top with four input streams, then read each stage downward. Flames and red borders mark trainable blocks; snowflakes and blue borders mark frozen ones. Stage 1 updates InternVL, the denoiser, and the action head using future segmentation and action targets. Stage 2 freezes the VLM and action head while future ground-truth actions enter the controllable denoising branch. Stage 3 freezes the VLM and denoiser, with reward feedback directed toward the action head. The red feedback arrow therefore denotes learned action refinement, not a physical steering command. Equations (10)–(12) explain that refinement through reward-weighted imitation. e-pipelinee-stage1e-stage2e-stage3
What it supports. The model shares a representation without collapsing every function into one decoder. Its world-model branch supplies future features and reward information that can affect action learning. The main architectural distinction is the staged introduction of action conditioning and feedback, beyond simply predicting actions and scenes from common features.
Where the evidence stops. Ground-truth future actions in the middle panel are training inputs. The diagram does not provide a complete test-time candidate-selection algorithm. Its separate heads also matter when interpreting the catalog's Dual-system label.
2. Motivation
2.1 The problem and the proposed response
A planner can share features with a world model without evaluating what its proposed actions would cause. The authors target this gap using action-conditioned BEV imagination, while avoiding pixel-level video rollouts. Their claims about internalizing physical laws are a motivation, rather than a separately validated capability. e-motivatione-stage1e-stage2
2.2 What this reading follows
A useful driving world model must do more than predict a plausible scene: it must help distinguish the consequences of the planner's actions. DriveWorld-VLA approaches this by connecting a trajectory head and BEV prediction branches through shared vision-language features. Its three stages first align prediction tasks, then introduce action-conditioned latent generation, and finally weight planner training by a learned reward. Read the architecture before the score tables: the central contribution is the interaction between modules and training stages. The strongest evidence is improved benchmark planning, while the paper's broader language about causal reasoning and physical understanding exceeds what the reported tests directly measure. e-motivatione-pipelinee-stage1e-stage2e-stage3e-navsim1e-nuscenes
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The architecture supports WAM and a functional Dual-system reading: a distinct DiT world-model branch and action decoder interact through shared VLM features and reward feedback. Dual-system must not imply independent representations here. Forward action-conditioned prediction plus reward-weighted imitation supports Other mechanisms/Outside quadrants; no inverse-dynamics action extraction is specified. The broad JEPA and WM-RL subcategory labels should not be read as verified JEPA or policy-gradient objectives. e-pipelinee-stage1e-stage2e-stage3
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 First make planning and prediction speak through the same representation
Begin with the information available at the current driving step: camera views, BEV features, instructions, and historical actions. InternVL fuses their tokens, and the final hidden states provide a common representation. Appendix A explains how a variable-length vision-language sequence is compressed: projection reduces the hidden dimension from 1536 to 256, and 700 learned queries aggregate the result. A BEV prediction branch and an action decoder then consume related features but retain distinct jobs. Stage 1 uses semantic-map decoding to teach predictive scene structure while imitation supervises the trajectory. Reader interpretation: sharing can make scene supervision relevant to planning, but it does not by itself test whether a proposed action changes the imagined future. That missing dependency is why the second stage matters. e-stage1e-pipelinee-tokens
Figure 3. The second denoising branch makes future BEV generation depend on the proposed action. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the horizontal arrows from the sampling and encoder blocks into convolutional fusion, cross-attention, and the DiT block. The vertical arrows show where BEV, action, and timestep enter. In Stage 2, the future action is ground truth and the future BEV target comes from frozen encoders, as Equations (6)–(8) specify. The figure places a future BEV tensor above Euler Sampling; this training-oriented drawing must not be read as requiring observed future BEV at deployment. Section 3.3 instead describes conditioning on predicted actions. Its 25 Euler sampling steps are distinct from the figure's separate 12 marker on the DiT block. e-stage2e-stage3
What it supports. Action information enters the future-prediction computation itself, rather than only a downstream score. Flow matching operates on encoded BEV targets, allowing an action-conditioned world model without a pixel-level video rollout. Whether the resulting futures change correctly under unfamiliar actions remains an empirical question beyond this schematic.
Where the evidence stops. Notation is compressed: the figure labels current BEV as B_t, while Equation (8) uses B'_t. Equation (9) prints unprimed actions although its prose specifies predictions; x_k interpolation and its relation to the sampled BEV state remain underspecified.
5.2 Then add action dependence without moving the latent target
Stage 2 changes the prediction problem. The model now receives a future expert action sequence as a condition, and it learns to generate the corresponding future BEV representation. Frozen Stage 1 encoders produce that target from future observations, keeping its meaning anchored while the second denoising branch learns flow matching. Equation (8) supervises a displacement from Gaussian noise to the encoded target. At use time, Section 3.3 replaces expert conditioning with the predicted action and uses 25 Euler sampling steps. Reader interpretation: this creates a computational route for comparing action consequences, but it does not prove causal correctness outside the training distribution. The paper leaves the interpolation for x_k and the exact candidate-generation procedure insufficiently specified, so a reproduction must resolve those details explicitly. e-stage2e-stage3
5.3 Finally distinguish reward-weighted learning from an online planner
Stage 3 combines two accounts of the future: the history-conditioned prediction and the action-conditioned rollout. The reward model sees both, together with the proposed trajectory, and is supervised by simulator-derived trajectory evaluations. Its predicted scalar multiplies the action imitation error, while segmentation and reward losses remain part of training. The VLM and denoiser are frozen during this refinement. This is a concrete way for imagined outcomes to influence the learned action head. Reader interpretation: the equations support reward-weighted imitation, but they do not specify policy-gradient reinforcement learning or a complete deployment-time search algorithm. Table 4 supports the progressive pipeline; Table 5's 91.3 versus 83.6 PDMS comparison further supports separating the later stages rather than optimizing them simultaneously after Stage 1. e-stage3e-stage-ablatione-schedule
5.4 Training and inference
During training
Stage 1 combines segmentation and action losses. Stage 2 optimizes only flow matching, with the VLM and action head frozen in Figure 2. Stage 3 freezes the VLM and denoiser, trains reward and action heads, and retains segmentation supervision on fused future latents. e-pipelinee-stage1e-stage2e-stage3
NAVSIM uses three front-facing views stitched to 256×1024, ResNet-34, AdamW at 10^-4, batch 16, and 20 epochs per stage: approximately 120 hours on eight NVIDIA H20 GPUs. nuScenes uses six 640×384 views, pretrained Swin-T with BEV-Planner encoding, AdamW at 7×10^-5, batch 1, and 24 epochs per stage: approximately 93 hours on eight H20s. e-implementation
During inference
Section 3.3 describes predicted-action conditioning and 25 Euler sampling steps before reward scoring. Future ground-truth observations/actions supply Stage 2 training targets, not permissible deployment inputs. The equations specify reward-weighted training but leave candidate count, final trajectory selection, and any repeated online refinement algorithm unspecified. e-stage2e-stage3
5.5 Implementation flow
- Encode a shared scene representation
Images, text, BEV features, and serialized historical actions enter InternVL. Final hidden states form the shared representation. Appendix A tiles images into 448×448 patches, assigns 256 image tokens per patch, projects hidden dimension 1536 to 256, and aggregates with 700 learned queries. Section 3.1 names BEVFormer; implementation backbones differ by benchmark. e-pipelinee-stage1e-tokense-implementation
- Learn history-based prediction
Cross-attention enriches current BEV features with shared hidden states. The first denoising branch forecasts future BEV features from history, while a separate action head predicts the trajectory. Current and future semantic BEV maps supervise the representation; expert actions supply imitation targets. e-stage1
- Learn controllable futures
Frozen Stage 1 encoders process future observations to produce latent targets. The second branch uses a DiT conditioned on current BEV features and ground-truth future actions. Flow matching learns a noise-to-future-latent transformation, rather than a pixel decoder. e-stage2
- Score imagined consequences
The model predicts an action and a history-based future, then generates an action-conditioned future. A reward model consumes both future representations and the predicted action. Its targets come from simulator evaluation of predicted trajectories; its scores weight action imitation. e-stage3
6. Experiments & results
DriveWorld-VLA connects a driving planner to two BEV prediction branches through shared vision-language hidden states. It first learns scene and trajectory prediction, then learns action-conditioned latent futures, and finally refines the action head using learned reward weights. The reported gains concern benchmark planning scores and collision rates; they do not establish physical deployment or general causal understanding.
6.1 Read the original evidence
Table 1. The top learned-method PDMS combines favorable collision, comfort, and progress scores. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the rightmost PDMS column first, then inspect the component columns and sensor configurations. DriveWorld-VLA reaches 91.3 versus 90.2 for DriveVLA-W0 and 88.3 for WoTE. NC is No Collision, DAC is Drivable Area Compliance, TTC is Time-To-Collision, C is Comfort, and EP is Ego Progress. The highlighted row has NC 99.1, C 100.0, and EP 85.9, but it does not lead every component. In the sensor column, C denotes a camera and L denotes LiDAR; this differs from C as the comfort metric. Section 4.1 describes the benchmark as non-reactive simulation. e-navsim1e-protocole-implementation
What it supports. The 1.1-point PDMS advantage over DriveVLA-W0 is an aggregate benchmark result. It is consistent with stronger planning under this evaluator, while the human reference remains higher at 94.8. The component columns reveal a balance of progress and constraint satisfaction, rather than uniform dominance across metrics.
Where the evidence stops. Sensor inputs differ: the compared rows include one, three, or six cameras and sometimes LiDAR. The table provides neither matched training budgets nor uncertainty intervals. Its scores do not demonstrate real-world reactive driving safety.
Table 3. Lowest reported average collision rate coexists with higher trajectory error than several baselines. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Keep the two metric blocks separate. Lower L2 measures closer agreement with recorded trajectories; lower CR measures fewer evaluated collisions. DriveWorld-VLA reports averages of 0.61 m and 0.16%, with three-second values of 0.99 m and 0.38%. HERMES-p has lower average L2 at 0.36 m but higher average CR at 0.32%. FSDrive has 0.53 m average L2 and 0.17% average CR, and its three-second CR is lower than DriveWorld-VLA's at 0.32%. The retained asterisk note refers to LAW checkpoint evaluation performed by the paper's authors; it does not describe a reproduction performed for this report. e-nuscenese-implementatione-prompts
What it supports. The evidence supports a favorable average collision-rate tradeoff, not an across-the-board win. The margin over FSDrive is 0.01 percentage points in average CR. The 0.16% headline is the reported average across the evaluated horizons, whereas the standalone three-second collision rate is 0.38%.
Where the evidence stops. No uncertainty is reported for the small average CR margin. Section 4.1 explicitly disables ego-state information, but the example prompt in Figure S1 mentions position and velocity; this implementation discrepancy remains unresolved.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIMv1 trajectory planning Paper's benchmark comparison, three-camera model; exact split identifier unspecified. The protocol is non-reactive simulation, although tables call it closed-loop. | 91.3 PDMS ↑ | DriveVLA-W0 90.2; WoTE 88.3; human reference 94.8. Highest listed learned-method score, a 1.1-point gain over DriveVLA-W0. Sensors differ across rows; this is not a controlled architecture-only comparison. e-navsim1e-protocol |
| NAVSIMv2 trajectory planning Two-stage pseudo-simulation benchmark; exact split identifier unspecified. | 86.8 EPDMS ↑ | DriveVLA-W0 86.1; DiffusionDrive 84.5. A 0.7-point aggregate advantage over DriveVLA-W0, not superiority on every component metric. e-navsim2e-protocol |
| nuScenes open-loop trajectory planning Validation set, three-second horizon; implementation explicitly says no ego-state information. | 0.61 m; 0.16%. At 3 s: 0.99 m and 0.38%. Average L2 (m) ↓; average CR (%) ↓ | FSDrive: 0.53 m/0.17% averages and 0.32% CR at 3 s. HERMES-p: 0.36 m/0.32% averages. Lowest listed average CR, but worse average L2 than both comparisons and worse 3-second CR than FSDrive. The 0.01-percentage-point average CR margin over FSDrive has no reported uncertainty. e-nuscenese-implementation |
| Progressive training ablations Table 4: sequential stages on NAVSIMv1 and nuScenes validation. Table 5: simultaneous Stage 2/3 after Stage 1 with doubled training length. | Stage 1/1+2/1+2+3: PDMS 87.6/89.5/91.3; CR 0.25/0.19/0.16. Progressive versus simultaneous PDMS: 91.3 versus 83.6. PDMS ↑; average CR ↓ | No-stage baseline: PDMS 87.1 and CR 0.34. Staging helps this setup. The sequential comparison changes training duration and modules, so it does not isolate reward quality alone; Table 5 specifically examines scheduling. e-stage-ablatione-schedule |
| Representation and supervision ablations NAVSIMv1, Tables 6–7 and accompanying supervision procedure. | VLM frozen/no pretraining 87.2; frozen/pretrained 87.6; unfrozen/pretrained 88.3. Task-only versus task+feature rows: 87.9 versus 91.3. PDMS ↑ | The VLM sweep is separate from the full-system result. The feature experiment injects N(0,5) latent noise at inference. Results favor adapted representations and intact latents. The latter comparison cannot cleanly attribute 3.4 points to removing a training loss. e-vlm-ablatione-feature-ablation |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Sequential stages improve both benchmark summaries, with different magnitudes across metrics. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the first three columns as a training sequence, not as independent switches in a factorial experiment. The no-stage row is the baseline; subsequent rows add Stage 1, then Stage 2, then Stage 3. PDMS rises from 87.1 to 87.6, 89.5, and 91.3. The nuScenes average CR entries fall from 0.34 to 0.25, 0.19, and 0.16. Inspect the horizon columns before claiming a large Stage 3 effect: its two- and three-second CR changes are 0.11 to 0.10 and 0.40 to 0.38. The paper uses different baselines and reward models across these benchmarks. e-stage-ablatione-stage2e-stage3e-protocol
What it supports. Action-controllability training adds 1.9 PDMS points after Stage 1, and refinement adds another 1.8. The same ordering improves average nuScenes CR, but the horizon-specific changes are modest. This supports the usefulness of the full training sequence under the reported setups.
Where the evidence stops. Adding stages also changes optimization and training duration. This table does not isolate action conditioning from extra training, or reward quality from other Stage 3 losses. Despite its Closed-Loop label, the NAVSIM protocol remains non-reactive.
Table 7. Feature-related performance changes require reading the intervention behind the table. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the Task and Features checkmarks: task supervision is present in both rows, whereas the feature column changes. The paper identifies segmentation and action losses as task-level signals and denoising supervision as feature-level guidance. The full row reaches 91.3 PDMS versus 87.9 in the row marked without features. Now consult the accompanying paragraph on page 8: the described intervention injects N(0,5) noise into latent variables at inference to weaken the effect of feature supervision. That procedure is essential to interpretation and is not visible in the table's binary labels. e-feature-ablatione-stage1e-stage2
What it supports. The reported 3.4-point gap shows sensitivity to the paper's feature-related intervention. It is compatible with the importance of usable latent features, but it does not establish that removing the feature training objective alone accounts for the full difference. A controlled loss ablation would answer a narrower question.
Where the evidence stops. The table's loss-style labels and the inference-noise procedure do not form a clean training-only contrast. Reproduction should separate loss removal from latent corruption and document how the N(0,5) noise distribution is parameterized.
7. Analysis & limitations
7.1 What the evidence leaves open
The reported evidence is short-horizon benchmark evaluation. NAVSIMv1 is described as non-reactive and NAVSIMv2 as pseudo-simulation. Selected trajectory visualizations do not establish reactive-agent behavior, real-vehicle safety, or general physical-law understanding. e-protocole-qualitative
Figure S1 says eight NAVSIM cameras and nuScenes position/velocity inputs, conflicting with the three-camera implementation and explicit no-ego-state claim. Its common eight-point output also lacks a clarified mapping to the differing benchmark horizons. These are unresolved source inconsistencies. e-promptse-implementation
Figure 3 and Equation (9) abbreviate or reuse BEV/action notation. The prose says predicted actions condition Stage 3, but Equation (9) retains an unprimed action. Reward architecture, calibration, candidate selection, and inference latency are not sufficiently specified for exact reproduction. e-stage2e-stage3
7.2 Questions for discussion
- Would reward scoring distinguish deliberately different actions when the current observation is held fixed?
- Would feature-loss removal still reduce planning quality without any inference-time latent noise?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires benchmark data and evaluation code, the InternVL/BEV encoders, semantic BEV targets, expert trajectories, and simulator-derived reward labels. Appendix A specifies token construction and latent compression, but maximum sequence length, precise reward implementation, and software versions remain unstated. e-pipelinee-stage1e-stage3e-implementatione-tokens
The title-page abstract promises future code/model release. The source supplies hardware and training schedules but no deployment timing; availability or reproducibility cannot be inferred from that promise. e-identitye-implementation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does the imagined future add action-specific information to reward scoring?
Reader-proposed check, not performed: freeze one trained model and use held-out NAVSIM scenes with identical current BEV inputs and noise seeds. Evaluate feasible braking, continuation, and lateral trajectory candidates using the same non-reactive simulator. Compare reward rankings with correctly paired imagined futures against rankings after permuting futures across candidate actions while keeping the action input fixed. Report reward-ranking agreement, collision calibration, and selected-trajectory PDMS. If permuting the futures leaves these measures unchanged, the claim that action-specific imagination materially guides scoring is weakened, even if the complete planner remains strong. e-stage2e-stage3e-protocole-navsim1
Check 2: Separate feature-loss benefits from inference-time corruption
Reader-proposed check, not performed: run a two-by-two NAVSIM comparison with feature supervision present/removed during training and latent noise absent/present during evaluation. Keep initialization, data split, task losses, training budget, and evaluator fixed across paired seeds. Specify the N(0,5) convention and exact injection location before testing. Report PDMS and its component scores with uncertainty. A genuine training-supervision effect should persist between the two clean-inference conditions; a gap appearing only after noise injection would instead identify latent-corruption sensitivity as the main explanation for Table 7. e-feature-ablatione-stage1e-stage2e-implementation
8.3 Reading coverage
Visual audit: Inspected the title/author/version page; all main Figures 1–4 and Tables 1–7; the method equations, evaluation protocols and hardware/training details on pages 3–7; Appendix A tokenization and representation details on page 12; prompt Figure S1 on page 13; and every qualitative appendix figure on pages 14–20. All six final original crops were inspected. Architecture arrows and freeze markers were checked against Sections 3.1–3.3; Figure 3/Equation (9) notation and Figure S1/Section 4.1 input discrepancies are disclosed. Table 7 was interpreted jointly with its page-8 intervention description. Pages 9–11 were read as text and are outside this image pass. No separate supplement or code was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 12, 13, 14, 15, 16, 17, 18, 19, 20. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (p. 1)
- 1. Introduction (p. 2)
- 2. Related Works (pp. 2–3)
- 3. Method, including 3.1–3.3 (pp. 3–5)
- 4. Experiments, including 4.1–4.4 (pp. 5–8)
- 5. Conclusion (p. 8)
- Impact Statement and References (pp. 9–11)
- Appendix A. More Experiment Details, A.1–A.3 and Figure S1 (pp. 12–13)
- Appendix B. More Visualization, Figures S2–S8 (pp. 12, 14–20)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction's missing-image limitation was addressed by inspecting the supplied PDF: all main and appendix figures and Tables 1–7 were visually reviewed. Reference pages 9–11 were read as text but were not rendered.
- Identity/version scope: the title and all seven authors match the catalog; the PDF uses en dashes in Vision–Language–Action and prints Feiyang jia with a lowercase surname initial. The margin identifies arXiv:2602.06521v1, 6 February 2026, while the footer says Preprint, February 9, 2026. This report concerns that supplied artifact; the catalog's ICML 2026 venue and any later proceedings edition were not independently verified.
- No external source, code repository, checkpoint, or separate supplement was inspected. No experiments were run.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block, arXiv margin, preprint footer, Abstract
Title matches the supplied identity, with typographic en dashes. Authors are Feiyang jia, Lin Liu, Ziying Song, Caiyan Jia, Hangjun Ye, Xiaoshuai Hao, and Long Chen. Affiliations identify Beijing Jiaotong University and Xiaomi EV. Margin: arXiv:2602.06521v1, 6 Feb 2026; footer: Preprint, February 9, 2026. Code/models are promised for future release.
Go to primary source ↓e-motivationPDF pp. 1–2, Abstract, Figure 1 and Section 1
The authors distinguish external world-model interaction from feature sharing and propose action-conditioned latent imagination to connect future prediction with planning. Physical-law internalization is asserted rather than independently measured.
Go to primary source ↓e-pipelinePDF p. 3, Figure 2 and Section 3.1
InternVL3-2B connects multimodal tokenization to denoiser and action heads. Flames/snowflakes indicate trainable/frozen blocks across three stages; Stage 2 introduces future ground-truth actions, and Stage 3 introduces reward feedback.
Go to primary source ↓e-stage1PDF pp. 3–4, Section 3.1, Equations (1)–(5)
Images, BEV, historical actions, and text form shared hidden states. Cross-attention enriches BEV; the first denoiser predicts future features and SEG decodes maps. A separate ACT decoder predicts expert-supervised trajectories; Stage 1 sums segmentation and action losses.
Go to primary source ↓e-stage2PDF p. 4, Section 3.2, Figure 3, Equations (6)–(8)
Frozen encoders generate future latent targets. The second denoiser is a DiT conditioned on current BEV and ground-truth future actions, optimized by flow matching against target-minus-Gaussian-noise displacement. Figure 3 shows BEV fusion, action cross-attention, timestep conditioning, and a 12 marker on the DiT block; no x_k interpolation formula is provided.
Go to primary source ↓e-stage3PDF p. 5, Section 3.3, Equations (9)–(12)
Predicted actions condition Euler generation with N=25. Reward consumes both future BEV branches and the predicted action, using simulator-derived targets. Predicted reward weights imitation error; Stage 3 adds segmentation/reward losses while VLM and denoiser are frozen. Equation (9) prints unprimed action despite predicted-action prose.
Go to primary source ↓e-protocolPDF p. 5, Section 4.1, Dataset and Metrics; p. 13, Figure S1 NAVSIM prompt
NAVSIMv1 is described as non-reactive open-loop simulation with PDMS penalties and weighted progress/TTC/comfort. NAVSIMv2 uses two-stage pseudo-simulation and EPDMS. Figure S1 describes a four-second non-reactive NAVSIM evaluation with an LQR controller and recorded background trajectories. nuScenes uses L2 and CR.
Go to primary source ↓e-implementationPDF p. 5, Section 4.1, Implementation
NAVSIM: three views, 256×1024 composite, ResNet-34, AdamW 1e-4, batch 16, 20 epochs/stage, eight H20s, approximately 120 hours. nuScenes: six 640×384 views, pretrained Swin-T/BEV-Planner, AdamW 7e-5, batch 1, 24 epochs/stage, eight H20s, approximately 93 hours; explicitly no ego-state information.
Go to primary source ↓e-navsim1PDF p. 6, Table 1, DriveWorld-VLA, DriveVLA-W0, WoTE and Human rows
DriveWorld-VLA uses 3×C and scores NC 99.1, DAC 98.2, TTC 96.1, comfort 100.0, EP 85.9 and PDMS 91.3. DriveVLA-W0 has PDMS 90.2 with 1×C; WoTE 88.3 with 3×C+LiDAR; Human 94.8.
Go to primary source ↓e-navsim2PDF p. 6, Table 2, DriveWorld-VLA, DriveVLA-W0 and DiffusionDrive rows
EPDMS is 86.8, 86.1 and 84.5 respectively. DriveWorld-VLA has DAC 99.1, DDC 99.6 and LK 97.0, but does not lead every metric.
Go to primary source ↓e-nuscenesPDF p. 7, Table 3, full metric header and DriveWorld-VLA, FSDrive, HERMES-p rows
DriveWorld-VLA L2 at 1/2/3 seconds and reported average: 0.28/0.58/0.99/0.61 m; CR: 0.00/0.10/0.38/0.16%. FSDrive averages 0.53 m and 0.17%, with 3-second CR 0.32%; HERMES-p averages 0.36 m and 0.32%. The asterisk marks LAW results reproduced by the paper's authors using an official checkpoint.
Go to primary source ↓e-stage-ablationPDF p. 7, Table 4, all training-process rows; p. 6, Section 4.3
No-stage/Stage-1/Stage-1+2/all-stage PDMS: 87.1/87.6/89.5/91.3; average CR: 0.34/0.25/0.19/0.16. Stage 3 changes nuScenes 2/3-second CR from 0.11/0.40 to 0.10/0.38. Text notes different baselines and reward models across benchmarks.
Go to primary source ↓e-schedulePDF p. 7, Table 5 and Ablation on training strategies
Non-progressive jointly trains Stage 2 and Stage 3 after Stage 1, with doubled training length described for fairness. PDMS is 83.6 versus 91.3 for the progressive schedule.
Go to primary source ↓e-vlm-ablationPDF p. 7, Table 6 and Ablation on VLM strategies
Frozen/no-pretraining, frozen/pretrained, and unfrozen/pretrained configurations score 87.2, 87.6, and 88.3 PDMS. Pretraining follows RecogDrive with three SFT epochs. The table is a separate strategy sweep.
Go to primary source ↓e-feature-ablationPDF p. 7, Table 7; p. 8, Ablation on Supervision
Task-only and task+feature rows score 87.9 and 91.3 PDMS. The procedure describes N(0,5) noise injected into latents at inference to weaken feature-level constraints; this is not described as an otherwise identical loss-removal-only comparison.
Go to primary source ↓e-tokensPDF p. 12, Appendix A.1–A.3
Images are adaptively tiled into 448×448 patches with an extra thumbnail when multiple patches are used; each patch has 256 visual placeholder tokens. Hidden dimension 1536 is projected to 256, then 700 learned queries cross-attend to form a compact representation. Maximum sequence length is named but not numerically specified.
Go to primary source ↓e-promptsPDF p. 13, Figure S1, NAVSIM, nuScenes and Common blocks plus caption
NAVSIM example says eight cameras and a four-second forecast; nuScenes says six cameras, ego position/velocity, and a three-second forecast. The common prompt uses four historical timesteps, navigation text and eight future points. These statements do not resolve conflicts with Section 4.1.
Go to primary source ↓e-qualitativePDF p. 8, Figure 4; pp. 14–18, Figures S2–S6; pp. 19–20, Figures S7–S8
NAVSIM panels compare Stage 2, Stage 3 and ground-truth trajectories; Figure 4 highlights selected proximity cases. nuScenes panels pair six camera views with Stage 3 and ground-truth trajectories. These are selected qualitative examples, not quantified world-model accuracy or real-vehicle deployment trials.
Go to primary source ↓8.5 Primary sources
DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving ↗
PDF · 8,457 extracted words
Source fingerprint
047ebabbce3437a5722302de5ac813c3e3fb664c276099f92d444f581ab579c8