The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey
1. Paper overview
In one sentence: The survey separates what a driving world model predicts from how a driving system uses it, revealing why better scene generation alone cannot establish reliable planning. e-identitye-taxonomye-planninge-survey-boundarye-future
| At a glance | What to know |
|---|---|
| Research problem | Author claim Driving research combines expensive, unevenly distributed data with demanding geometric, temporal and safety requirements. The authors argue that grouping all world models together obscures different technical problems: video fidelity, 4D structure, latent grounding and multi-agent behavior require different representations and evaluations. The survey therefore separates prediction modality from application and reviews the ecosystem needed to train and assess each family. e-scopee-taxonomy |
| Core mechanism | Source description The main contribution is a two-axis synthesis: five prediction spaces and four application families, linked to simulators, datasets and metrics. It is an organizing framework for existing research, not a newly trained driving system. e-taxonomy |
| A key reported result | Closed-loop driving on CARLA Bench2Drive: Think2Drive: DS 91.85, SR 85.41%; Raw2Drive: DS 71.36, SR 50.24%. Driving Score and success rate (%), higher is better. Table 8, Bench2Drive rows only; Section 5.4 says Think2Drive uses privileged simulator information. Non-DWM ORION: DS 77.74, SR 54.62%. The strongest listed DWM result has an information-access advantage; Raw2Drive trails ORION on these two metrics. This is simulation evidence, with no uncertainty intervals reported in the compilation. e-closed |
| Reading caution | Author claim The authors warn that long rollouts, viewpoint changes and weather variation can expose hallucinations or physically inconsistent dynamics. Offline prediction tests do not capture the planner–simulator interaction needed to establish reliable closed-loop simulation. e-future |
Core contributions
- Source description
The main contribution is a two-axis synthesis: five prediction spaces and four application families, linked to simulators, datasets and metrics. It is an organizing framework for existing research, not a newly trained driving system. e-taxonomy
Figure 2. Prediction space and application are separate organizing axes. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the two large middle blocks. The left classifies the space being predicted: visual, 4D, multimodal, latent or vectorized. Point clouds and occupancy are subdivisions of 4D space. The right classifies use: simulation, data generation, enhancing driving or pre-training. Read across these blocks as two questions about the same method, not as a sequence of network modules. The green band supplies the datasets, simulators and metrics needed to study those choices. The yellow band lists evaluation tasks, and the orange band collects unresolved problems. Figure 2 and its caption describe the survey's structure; they do not specify an implemented architecture. e-taxonomye-future
What it supports. The map helps prevent a common category error: predicting occupancy does not determine whether the predictor is a simulator, a planner component or a pre-training objective. A useful reading of any listed method therefore needs both its prediction target and its application pathway.
Where the evidence stops. This is the authors' organizing framework, not evidence that one model implements every block. The figure's final orange label reads 'Attach & Defense'; Section 6 names that topic 'Attack and Defense'. The original graphic is preserved.
2. Motivation
2.1 The problem and the proposed response
Driving research combines expensive, unevenly distributed data with demanding geometric, temporal and safety requirements. The authors argue that grouping all world models together obscures different technical problems: video fidelity, 4D structure, latent grounding and multi-agent behavior require different representations and evaluations. The survey therefore separates prediction modality from application and reviews the ecosystem needed to train and assess each family. e-scopee-taxonomy
2.2 What this reading follows
A driving world model might generate camera frames, forecast occupied space, predict hidden features or simulate traffic agents. Those choices do not tell us whether the model selects actions at runtime, supplies training supervision or produces synthetic data. This survey makes the two questions separable. Read its modality taxonomy alongside its four planner-integration pathways, then use the benchmark tables to test how far each kind of evidence reaches. The supplied February 2026 revision contains broad literature comparisons, not a new model or a common experimental rerun. Its strongest lessons concern representation choices, evaluation contracts and the remaining gap between plausible futures and dependable closed-loop behavior. e-identitye-taxonomye-planninge-survey-boundarye-future
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Related resources. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The recorded survey/resource category and Not applicable architecture, prediction paradigm and quadrant fit this work. Its taxonomy organizes many distinct systems; no single proposed network warrants a One Model or joint future/action classification for the survey itself. Joint training in one reviewed pathway also does not establish inference-time control in every DWM. e-taxonomye-planning
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Choose what the future must preserve
Begin with the information a downstream task needs. The survey's visual-space methods preserve rich appearance but must learn geometry and cross-view consistency through image generation. Point clouds expose measured geometry, although their sparsity and irregular topology complicate forecasting. Occupancy offers a structured spatial description but introduces expensive 4D representations and data requirements. Latent models focus on features useful to a task, while vectorized models abstract traffic participants and signals, often using privileged information. The reader's deduction is that output realism and decision usefulness can diverge: a representation may retain task-relevant behavior while omitting texture, or generate texture without establishing reliable dynamics. The survey's multimodal discussion explores shared representations, intermediate occupancy and interaction between separate generators as ways to combine complementary information. Those are alternative design families, not one mandatory processing pipeline. e-visuale-4de-efficient-occupancye-latente-vectore-multimodal
Figure 5. World predictions can affect selection, learning, joint output or reasoning. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow panel (a) from the current frame through the planner to alternative actions, then into the DWM and its evaluate-and-select output. The second arrow from the current frame conditions the DWM on the scene. Panel (b) instead passes a predicted action to the DWM and compares the predicted future with GT Future, meaning ground-truth future. The double-headed Scene Supervision marker denotes comparison; the caption and Section 4.3 explain training by back-propagation, not an environment feedback loop. Panel (c) produces action and future from a unified DWM Planner. Panel (d) depicts an imagined sequence before the predicted action. These are four conceptual integration patterns. e-planning
What it supports. The position of the prediction module matters as much as its accuracy. Candidate scoring adds computation during planning, whereas scene supervision can affect learned planner weights without added inference overhead. The bottom panels represent joint prediction/planning and reasoning, which require their own architectural and runtime evidence.
Where the evidence stops. Panel (b) is not a complete gradient graph: it does not identify frozen modules or optimization details. The source also cautions that selecting among candidates can mask planner deficiencies and that unified architectures have not yet established a significant advantage.
5.2 Trace prediction through learning and deployment
Use Figure 5 as an information-flow checklist. Candidate evaluation begins with a planner, branches into possible actions and asks the world model to predict consequences before selecting one. Dense scene supervision instead sends a planned action into a predictor during training and compares its future with the observed future; Section 4.3 says this can improve planning without extra inference computation. Unified prediction and planning place both outputs within one framework, and the reasoning pathway considers imagined futures before choosing an action. Separately, Figure 6 uses forecasting to pre-train representations for other tasks. The reader's interpretation is that an auxiliary loss, a transferred representation and an online world-model rollout provide distinct mechanisms for improvement. Evidence of joint training alone cannot determine which mechanism remains active when a vehicle selects its next action. e-planninge-pretraining
Table 3. Read the measured quantity before interpreting the direction arrow. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row across metric, direction, task and description. FID and FVD compare generated and ground-truth feature distributions; CD compares point clouds. L2 and collision rate belong to open-loop planning. The occupancy rows assess geometric and semantic overlap, while the final driving rows concern route completion, infractions, success, efficiency and comfort. This is a menu of different evaluation targets, not a single quality scale. Preserve a visible inconsistency: ADE and FDE are defined as displacement errors but receive upward arrows. Section 2.2 identifies them with command-following evaluation without explaining an inverted score, so their direction cannot be resolved from this table. e-metricse-metric-discrepancy
What it supports. Table 3 supports evaluating both prediction and downstream behavior. Its definition of Driving Score combines route completion with an infraction penalty per sample, making clear why progress alone is insufficient. Distributional similarity, controllability and driving success each supply a different piece of evidence.
Where the evidence stops. The ADE/FDE arrows remain unresolved. Separately, Section 5.1's phrase describing improved FID/FVD as higher scores conflicts with the downward arrows here and in Table 4. This edition follows the FID/FVD table directions and does not silently repair the source graphic.
5.3 Ask what each benchmark actually rules out
Read the performance section in layers. Video scores test distributional similarity, and point cloud or occupancy scores test geometric prediction under specified inputs and horizons. Open-loop planning then evaluates trajectories from recorded observations; closed-loop planning adds repeated simulator interaction. A favorable result at one layer does not automatically settle the next. The tables illustrate the problem: video methods differ in resolution and duration, occupancy reconstruction is separated from future forecasting, and Think2Drive's strong closed-loop result uses privileged information. The reader's deduction is to write an evaluation contract before comparing methods: inputs, conditions, split, horizon, metric, simulator setting and runtime budget. Unknown entries stay unknown. This approach also keeps the authors' calls for reliable simulation and better multisensor alignment as research goals rather than conclusions already established by a leaderboard. e-metricse-videoe-pointe-occupancye-opene-closede-future
5.4 Training and inference
During training
Across the reviewed families, diffusion approaches learn conditional scene generation, while autoregressive approaches serialize scene, action and sometimes language representations for next-token prediction. The survey gives conceptual descriptions rather than a common loss, optimizer, training schedule or frozen-module specification. e-visuale-4de-planning
Training uses include future-scene alignment for dense planner supervision, latent rollouts for learning, and vision-to-4D predictive pre-training with image–4D pairs. The survey identifies limited aligned multimodal data as a scaling constraint; it supplies no single executable training recipe or shared compute budget. e-latente-planninge-pretraining
During inference
In candidate evaluation, a planner proposes trajectories, the DWM predicts their consequences, and rewards or costs select a decision. Repeating prediction for multiple candidates increases runtime; the authors caution that selection can conceal weakness in the original planner. e-planning
In the supervision pathway, prediction improves training without added inference overhead, according to Section 4.3. Unified pathways instead directly combine prediction and planning, sometimes with reasoning. Figure 3(c) adds an environment feedback loop; generated observations alone do not demonstrate action execution. e-planninge-simulation
5.5 Implementation flow
- Choose the prediction space
Visual DWMs prioritize texture, controllability and multi-view consistency. Point clouds preserve measured geometry but are sparse and unordered; occupancy represents voxel structure through time. Multimodal models combine complementary scene descriptions. Latent models predict task-oriented features, while vectorized models abstract agents and traffic signals from privileged state. e-visuale-4de-multimodale-latente-vector
- Compress and couple scene dynamics
The survey describes BEV tokenization, ray-centric and range-image point cloud representations; occupancy methods use BEV, multiple planes, sparse queries or continuous fields. Separating static backgrounds from dynamic objects can reduce computation, but explicit object-based separation depends on semantic labels. Multimodal alternatives include shared representations, hierarchical occupancy-first generation and interaction between separate generators. e-4de-efficient-occupancye-multimodal
- Locate the planner connection
Figure 5 distinguishes scoring candidate actions, supervising a planner through predicted scenes, jointly producing actions and futures, and reasoning through imagined futures. These are different information flows. A future-scene loss used during learning need not imply a world-model rollout during deployment. e-planning
- Separate simulation, synthesis and transfer
Figure 3 separates observation generation, traffic generation and an action-feedback loop connecting traffic to realistic observations. Figure 4 groups environment variation, novel views and multimodal data synthesis. Figure 6 transfers representations learned through future prediction to downstream tasks. e-simulatione-datae-pretraining
- Match evidence to the objective
Generation metrics assess distributional similarity or geometric overlap; controllability tests assess instruction following; open-loop planning uses recorded observations; closed-loop evaluation includes repeated simulator feedback. These evaluation levels answer different questions and should not be collapsed into one ranking. e-metricse-opene-closed
6. Experiments & results
This survey organizes driving world models by what they predict and how those predictions enter autonomous driving. Its useful distinction is between reconstructing future sensory scenes, learning task-oriented latent dynamics, and simulating traffic behavior. These representations can support simulation, synthetic data, planning or pre-training, but a good generation score does not by itself establish reliable closed-loop driving. The report reads the February 2026 revision and treats its benchmark tables as literature compilations.
The source is a survey with conceptual application diagrams and literature comparison tables. It proposes no new network with a controlled ablation, shared training recipe or common rerun. Figure 5 therefore supplies the method-integration visual, and Tables 4, 6 and 8 supply quantitative comparisons; no visual is presented as a survey-run ablation. The edition includes six original crops, but cannot provide a genuine mechanism-isolating ablation or measured training-compute figure absent from this source. e-survey-boundarye-planninge-videoe-occupancye-closed
6.1 Read the original evidence
Table 4. Generation quality is reported alongside capabilities that change the difficulty. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Before reading the score columns, inspect MV, Resolution and Core Arc. MV marks multi-view forecasting support, and Core Arc. names the core architecture; Auto. Tran. abbreviates autoregressive Transformer. MD is maximum generation duration in seconds, not a statement of the clip length used to calculate every FVD value. Compare CVD-STORM near the bottom with MiLA: both support multi-view, but their reported durations differ and MiLA's resolution is missing. Keep dashes and blank cells as missing information. Section 5.1 further warns that structured scene conditions carry spatial priors absent from command-only conditions, so the table does not establish a matched experiment. e-videoe-metric-discrepancy
What it supports. CVD-STORM reports FID 3.8 and FVD 14.0 with 256×448 output and 20 s maximum duration. MiLA reports lower FID, 3.0, but higher FVD, 18.2, and 10 s maximum duration. Even the numerical leaders depend on which metric is being read.
Where the evidence stops. These are survey-reported results from different methods and settings. The paper explicitly calls this comparison not strictly fair. The conflicting FID/FVD direction in the surrounding prose is disclosed with Table 3; it does not change the downward arrows retained here.
Table 6. Separate present reconstruction, future prediction and sensor access. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First divide the rows by Input: O means occupancy, C means camera, and the last row uses Layout. Then separate the two metric blocks, mIoU and IoU, both percentages with upward arrows. Within each block, Recon. evaluates the current timestamp, while 1s, 2s and 3s evaluate future states; Avg. covers those three future horizons only. Follow DOME-O and DOME-F within their respective input blocks. The comparison reveals both temporal degradation and the challenge of predicting 4D structure from cameras. It is particularly important not to substitute a high reconstruction score for a forecast score when judging a world model. e-occupancye-metrics
What it supports. DOME-O's mIoU falls from 35.11% at 1 s to 20.29% at 3 s; DOME-F falls from 24.12% to 13.24%. Their average forecast mIoUs are 27.10% and 18.25%. The table makes horizon and input access visible rather than compressing both into one ranking.
Where the evidence stops. The paired names do not make this a controlled sensor ablation: the survey does not establish equal training or all implementation choices. The mIoU–IoU relationship also cannot by itself prove which object sizes or dynamic classes caused an error. A source arithmetic exception remains in the final UniScene row: its printed forecast averages are 31.76% mIoU and 34.84% IoU, whereas the displayed 1/2/3 s values average approximately 30.01% and 33.38%. The caption does not explain this discrepancy. The original values are preserved; the DOME comparison above does not depend on them.
Table 8. Keep benchmark groups and privileged information separate. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the rightmost Benchmark column and stay within one group. Town05 Short, Town05 Long, Town13 and Bench2Drive are not interchangeable settings. RL marks reinforcement learning; Core Arc. identifies the DWM architecture and Pred. Mod. its predicted modality, with O, C and B denoting occupancy, camera and BEV map. A dash records missing information rather than zero performance. In the Bench2Drive block, compare DS and SR first, then read efficiency and comfort as separate attributes. Finally consult Section 5.4: the table does not have a dedicated privilege column, but the text states that Think2Drive uses privileged information from the simulator. e-closede-metrics
What it supports. Think2Drive reports DS 91.85 and SR 85.41%, exceeding ORION's 77.74 and 54.62% in this compilation. Raw2Drive reports 71.36 and 50.24%, below ORION on both metrics. The results therefore support a qualified picture of progress rather than a universal DWM advantage.
Where the evidence stops. Think2Drive's privileged inputs prevent attributing its advantage solely to a world model. These are CARLA simulation results with no uncertainty intervals in the table; they establish neither a matched architectural comparison nor physical-road deployment reliability.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Video generation on nuScenes validation Literature compilation; multi-view support, resolution, conditions and maximum generation duration vary. | CVD-STORM: FID 3.8, FVD 14.0, multi-view, 256×448, maximum duration 20 s. FID and FVD, lower is better | MiLA: FID 3.0, FVD 18.2, multi-view, maximum duration 10 s; resolution is omitted. Different methods lead the two metrics. These are reported values, not a matched comparison; Section 5.1 explicitly calls the table not strictly fair. e-video |
| Camera-input point cloud forecasting on nuScenes validation Table 5 camera-input block; future horizons 1, 2 and 3 s. | DriveX-B: 0.66, 0.86, 1.10; average 0.87. Chamfer Distance (m²), lower is better; reported average across horizons | ViDAR: 1.12, 1.38, 1.73; average 1.41. The input category matches, but training and implementation controls are not established by the survey. Its printed unit m² is retained. e-point |
| Occupancy forecasting on Occ3D-nuScenes validation Table 6; compare DOME-O with occupancy input and DOME-F with camera input. | DOME-O: average mIoU 27.10, average IoU 36.36; 1 s/3 s mIoU 35.11/20.29. Forecast mIoU (%) and IoU (%), higher is better | DOME-F: average mIoU 18.25, average IoU 28.84; 1 s/3 s mIoU 24.12/13.24. Both rows decay with horizon. Their input gap is informative but is not a controlled sensor ablation performed by this survey. e-occupancy |
| Open-loop planning on nuScenes validation Table 7; averages over 1, 2 and 3 s from recorded observations. | PWM: average L2 0.78 m; collision rate 0.07%. L2 trajectory error (m) and collision rate (%), lower is better | UniAD: 1.03 m and 0.31%; PreWorld: 0.31 m and 0.53%. Trajectory error and collision rate need separate reading. Cross-paper results do not isolate a world-model contribution or establish closed-loop safety. e-open |
| Closed-loop driving on CARLA Bench2Drive Table 8, Bench2Drive rows only; Section 5.4 says Think2Drive uses privileged simulator information. | Think2Drive: DS 91.85, SR 85.41%; Raw2Drive: DS 71.36, SR 50.24%. Driving Score and success rate (%), higher is better | Non-DWM ORION: DS 77.74, SR 54.62%. The strongest listed DWM result has an information-access advantage; Raw2Drive trails ORION on these two metrics. This is simulation evidence, with no uncertainty intervals reported in the compilation. e-closed |
6.3 Ablations and diagnostic examples
No ablation or diagnostic visual is included in this edition. The source-specific visual limitations are recorded above.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors warn that long rollouts, viewpoint changes and weather variation can expose hallucinations or physically inconsistent dynamics. Offline prediction tests do not capture the planner–simulator interaction needed to establish reliable closed-loop simulation. e-future
Aligned multisensor data remain expensive. Latent prediction improves efficiency but risks representations becoming detached from physical reality. Scene decoding, larger models and reasoning add latency; unified models have not yet established a significant advantage over traditional methods. e-latente-planninge-future
Metric directions are internally inconsistent: Table 3 labels ADE/FDE as higher-is-better despite defining displacement errors. Section 5.1 describes improvement as higher FID/FVD, conflicting with the downward arrows in Tables 3 and 4 and its later discussion of reductions. This report uses the FID/FVD table directions and leaves ADE/FDE direction unresolved. e-metric-discrepancy
The comparisons are heterogeneous literature summaries, without a shared rerun, controlled ablation or uncertainty analysis. Reconstruction, forecasting and planning scores cannot establish the authors' broader claims about physical understanding by themselves. e-videoe-occupancye-opene-closed
7.2 Questions for discussion
- Which evaluation separates plausible video from action-conditioned physical consistency?
- Would a world model still improve planning after matching candidate count, input privilege and runtime?
- How should latent dynamics be grounded without requiring expensive full-scene decoding?
8. Reproducibility audit
8.1 Requirements and known gaps
A comparison audit should retain input privilege, output modality, evaluation split, horizon, resolution, conditioning and metric implementation. Missing table entries must stay missing; the survey alone cannot establish matched training budgets, hardware, seeds or software versions. e-videoe-pointe-occupancye-closed
Reader-proposed checks: normalize a small comparison to a common evaluation contract, and separately test scene-supervised training against candidate selection at inference. Neither check has been run; both need implementation details beyond the survey's conceptual diagrams. e-planninge-videoe-closed
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Audit and rerun a genuinely matched video comparison
Reader-proposed, not performed: start with the CVD-STORM and MiLA rows of Table 4 and record every known and missing input, conditioning, view, resolution and duration field. A subsequent implementation study should evaluate both on the same nuScenes validation scenes, identical admissible conditions, view count, output resolution, evaluation clip horizon and FID/FVD implementation. Distinguish maximum generation duration from evaluation horizon and report generation time. Use paired scene resampling to assess score uncertainty. A ranking reversal or unstable difference would falsify the claim that the unnormalized table order represents a robust advantage. If matching cannot be established, report the comparison as unresolved rather than filling missing settings. e-videoe-metrics
Check 2: Disentangle scene-supervised learning from candidate selection
Reader-proposed, not performed: hold a planner, training data and sensor access fixed, and compare training with versus without the scene-prediction supervision depicted in Figure 5(b), using the same single-output inference path. In a separate arm, freeze that planner and evaluate the same candidate set with its original selector versus a DWM-based selector from Figure 5(a). Keep candidate count fixed and disclose additional latency. As a negative control, shuffle candidate-to-future associations before DWM scoring. Evaluate collision rate and closed-loop route success in a fixed simulator protocol. An advantage surviving the shuffle would weaken the proposed action-conditioned prediction mechanism; an advantage only in the selection arm would not demonstrate improved intrinsic single-output planning. e-planninge-opene-closede-future
8.3 Reading coverage
Visual audit: Actually inspected the title/author/revision page and every body page through the conclusion; reviewed Figures 1–6 and Tables 1–8. All six final original crops were viewed. Figure 5 arrow roles were checked against its caption and Section 4.3. Metric-arrow discrepancies were checked on pages 5 and 13 and disclosed. Page 16 supplies the privileged-information and reliability cautions; page 17 supplies efficiency and future-direction context. References on pages 18–24 were read in the supplied text but not viewed as images; no claim relies on their visual layout. There is no appendix or controlled ablation page in the supplied paper. Separate supplements and cited works were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations, revision stamp and abstract (p. 1)
- 1. Introduction (pp. 1–2)
- 2. DWM Ecosystem; 2.1 Simulator & Dataset; 2.2 Metric (pp. 2–5)
- 3. DWM Taxonomy; 3.1 Visual Space; 3.2 4D Space; 3.3 Multimodal Space; 3.4 Latent Space; 3.5 Vectorized Space (pp. 4–9)
- 4. DWM Application; 4.1 Simulation; 4.2 Data Generation; 4.3 Enhancing Driving; 4.4 Pre-training (pp. 9–13)
- 5. DWM Performance; 5.1 Video Prediction; 5.2 Point Cloud Prediction; 5.3 Occupancy Prediction; 5.4 Planning (pp. 13–16)
- 6. Limitation and Future Direction (pp. 16–17)
- 7. Conclusion (p. 17)
- References [1]–[178] (pp. 17–24); all 11 supplied text chunks read individually
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction limitation was addressed by visually inspecting PDF body pages 1–17 and all six final crops. Reference pages 18–24 were read as text, not visually inspected.
- Identity notes: the exact title and all seven authors match the catalog. The inspected title page states arXiv:2502.10498v2 [cs.CV], 1 February 2026; the catalog records submission on 14 February 2025. This report covers the supplied v2. The initial edition was not supplied, so its textual differences cannot be established.
- The linked collection, cited studies, code, datasets and separate supplements were not opened. No experiments were reproduced. Reported third-party results were verified against this survey, not against the cited studies.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block and arXiv margin stamp
Exact catalog title and seven authors; Huazhong University of Science and Technology and Baidu Inc.; arXiv:2502.10498v2 [cs.CV], 1 Feb 2026.
Go to primary source ↓e-scopePDF pp. 1–2, Abstract and Section 1, contributions
The work surveys prediction modalities, applications, ecosystem, representative performance and future directions.
Go to primary source ↓e-taxonomyPDF p. 3, Figure 2 and caption; p. 2, Figure 1
Five prediction spaces are organized separately from simulation, data generation, enhancing driving and pre-training; Figure 1 illustrates historical inputs, conditions and possible outputs.
Go to primary source ↓e-visualPDF pp. 4–5, Section 3.1
Video approaches emphasize controllability and spatiotemporal consistency; diffusion and autoregressive pathways have different emphases; scenes and actions may be modeled in a unified sequence.
Go to primary source ↓e-4dPDF pp. 5–6, Section 3.2, Point Cloud and Occupancy
4D means three spatial dimensions plus time. Point cloud representations include BEV, ray-centric forms and range images; occupancy methods use diffusion or autoregressive scene/trajectory/text sequences.
Go to primary source ↓e-efficient-occupancyPDF p. 7, Section 3.2, occupancy decoupling and representation paragraphs
Static/dynamic separation, pose transformations, implicit transitions, BEV, multi-plane, sparse and continuous representations address cost; explicit object separation remains constrained by labels.
Go to primary source ↓e-multimodalPDF pp. 7–8, Section 3.3
The survey describes shared multimodal representations, hierarchical occupancy intermediates and separate generators coupled by cross-modal interaction; most methods handle specific modality combinations.
Go to primary source ↓e-latentPDF p. 8, Section 3.4
Latent DWMs support auxiliary supervision, planner components, parallel training rollouts and reasoning; the authors flag interpretability and physical-grounding risks.
Go to primary source ↓e-vectorPDF pp. 8–9, Section 3.5
Vectorized space uses privileged low-dimensional scene descriptions, emphasizing traffic simulation and multi-agent behavior rather than raw sensory processing.
Go to primary source ↓e-simulationPDF pp. 9–10, Figure 3 and Section 4.1
Observation generation and traffic generation are distinct; the closed-loop schematic connects traffic simulation, realistic observation and agent-action feedback.
Go to primary source ↓e-dataPDF pp. 10–12, Figure 4 and Section 4.2
Data generation includes environment variation, novel views and multimodal generation; the source states synthetic data cannot yet fully replace real data.
Go to primary source ↓e-planningPDF p. 11, Figure 5 and caption; p. 12, Section 4.3
Candidate evaluation uses predicted futures and rewards/costs; dense scene supervision acts through training; other pathways unify action/future prediction or add reasoning. Candidate selection incurs runtime cost; unified architectures lack an established significant advantage.
Go to primary source ↓e-pretrainingPDF p. 11, Figure 6; pp. 12–13, Section 4.4
Future prediction supplies transferable representations, including vision-to-4D learning from image–4D pairs; aligned multimodal data limit scale.
Go to primary source ↓e-metricsPDF pp. 3–4, Section 2.2; p. 5, Table 3, generation, planning and DS rows
FID/FVD compare feature distributions; CD compares point sets; IoU/mIoU measure occupancy overlap. DS is RC times IS per sample; IS starts at 1.0 and is reduced by infraction penalties.
Go to primary source ↓e-metric-discrepancyPDF p. 5, Table 3, ADE/FDE and FID/FVD rows; p. 13, Table 4 headers and Section 5.1 second/third paragraphs
ADE/FDE are described as errors but marked upward. FID/FVD are marked downward, while one prose passage describes improvement as higher scores and the next discusses reductions.
Go to primary source ↓e-videoPDF p. 13, Table 4, CVD-STORM and MiLA rows; Section 5.1
CVD-STORM reports 3.8 FID, 14.0 FVD, 256×448 and 20 s maximum duration; MiLA reports 3.0, 18.2 and 10 s with resolution omitted. Both support multi-view. Conditions, resolution and horizon prevent strictly fair comparison.
Go to primary source ↓e-pointPDF p. 13, Table 5, camera-input ViDAR and DriveX-B rows; p. 14, Section 5.2 continuation
DriveX-B CD is 0.66/0.86/1.10, average 0.87; ViDAR is 1.12/1.38/1.73, average 1.41. Header uses m² and 1/2/3 s; experimental settings affect comparisons.
Go to primary source ↓e-occupancyPDF p. 14, Table 6, DOME-O and DOME-F rows and caption; Section 5.3; final UniScene (Layout input) row for the unresolved forecast-average discrepancy
DOME-O forecast mIoU is 35.11/25.89/20.29, average 27.10; IoU average 36.36. DOME-F mIoU is 24.12/17.41/13.24, average 18.25; IoU average 28.84. Recon. is current-time reconstruction, excluded from forecast averages. The UniScene row prints mIoU 35.37/29.59/25.08 with Avg. 31.76 and IoU 38.34/32.70/29.09 with Avg. 34.84. Arithmetic means of those displayed triples are about 30.01 and 33.38, unlike the caption-defined averages; the source does not explain the discrepancy.
Go to primary source ↓e-openPDF p. 15, Table 7, UniAD, PWM and PreWorld rows; pp. 15–16, Section 5.4 Open-Loop Planning
On nuScenes validation, PWM average L2/Col. is 0.78 m/0.07%, UniAD 1.03 m/0.31%, and PreWorld 0.31 m/0.53%. Averages cover 1/2/3 s; input resolution and foundation models affect results.
Go to primary source ↓e-closedPDF p. 15, Table 8, Bench2Drive rows; p. 16, Section 5.4 Closed-Loop Planning
Think2Drive DS/SR is 91.85/85.41%, Raw2Drive 71.36/50.24%, ORION 77.74/54.62%. The text identifies Think2Drive's privileged simulator information. Table separates benchmark groups and RL use, without uncertainty intervals.
Go to primary source ↓e-futurePDF pp. 16–17, Section 6; p. 17, Section 7
Future directions cover data scarcity, reliable simulation, task unification, multisensor modeling, efficiency, and attack/defense; risks include hallucinations, costly aligned data and inference latency.
Go to primary source ↓e-survey-boundaryPDF p. 3, Figure 2; pp. 13–16, Section 5 and Tables 4–8; p. 17, Section 7
The survey compiles representative prior methods rather than presenting a new model with controlled experiments. Its figures are conceptual summaries and application illustrations; no controlled ablation is supplied.
Go to primary source ↓8.5 Primary sources
The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey ↗
PDF · 18,533 extracted words
Source fingerprint
66efc59d5acefa1f842333932b06c347d1a7aee1408a898be37818a4059014b0