World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model
1. Paper overview
In one sentence: World4Drive ranks candidate driving trajectories through predicted latent futures, improving reported collision metrics while learning from future resemblance rather than an explicit safety objective. e-identitye-contexte-dreamere-selectore-nuscenese-navsime-conditions
| At a glance | What to know |
|---|---|
| Research problem | Author claim The paper targets camera-based trajectory planning without new manual perception annotations. Its motivating claim is that a single future latent learned from raw images inadequately captures geometry, semantics and alternative driving intentions, slowing learning and weakening planning. e-problem |
| Core mechanism | Source description A planning decoder proposes trajectories before an action-conditioned dreamer predicts their future latents; a separately parameterized selector ranks those futures during inference. e-architecturee-dreamere-selector |
| A key reported result | Open-loop trajectory planning on nuScenes: World4Drive: L2 0.23 / 0.47 / 0.81 at 1 / 2 / 3 seconds, average 0.50; collision 0.02 / 0.12 / 0.33%, average 0.16%. L2 displacement error (m) and collision rate (%), lower is better. Reported nuScenes evaluation; trajectories sampled at 2 Hz over three seconds. Exact train/evaluation scene lists are not specified in this PDF. Perception-free LAW averages 0.61 m and 0.30%; perception-based LAW averages 0.49 m and 0.19%. LAW uses Swin-Tiny, whereas World4Drive uses ResNet-50. The rounded averages imply approximately 18.0% lower L2 and 46.7% lower collision than perception-free LAW (reader calculation). This is a cross-backbone comparison. World4Drive has the lowest average collision in Table 1, but not the lowest average L2. e-protocole-nuscenes |
| Reading caution | Reader analysis Latent resemblance to the logged future is the selector's training target. Calling its preferred trajectory safest requires additional validation: alternative intentions lack observed counterfactual outcomes, and benchmark collision reductions do not establish calibrated safety probabilities. e-selectore-nuscenese-navsim |
Core contributions
- Source description
A planning decoder proposes trajectories before an action-conditioned dreamer predicts their future latents; a separately parameterized selector ranks those futures during inference. e-architecturee-dreamere-selector
- Source description
Metric-depth positional encoding and Grounded-SAM semantic pseudo-supervision enrich temporal scene features. This removes manual perception labels from the reported driving training protocol, not expert driving targets or pretrained-model dependencies. e-contexte-losse-implementation
Figure 2. Trajectories are proposed first, imagined as latent futures, and then selected. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the two left-hand inputs separately. The trajectory vocabulary V and ego query feed the intention encoder; current images feed the context encoder and temporal aggregation. Their outputs meet in the planning decoder to produce candidate trajectories T. The downward arrow from T supplies action conditioning to the world-model dreamer, which also receives the current scene latent. The selector then connects each imagined future to its corresponding trajectory. On the right, actual future images enter a context encoder and produce a latent explicitly marked Detach. This target branch and the red reconstruction and expert-trajectory loss arrows depict training; Section 3.3.2 specifies score-based selection without observed future images at inference. e-architecturee-dreamere-selectore-loss
What it supports. The world model is used to evaluate already proposed actions and remains active when a trajectory is chosen at inference. The drawing therefore supports a modular planner–predictor–selector interpretation, even though the modules are trained end to end. It does not show a single generator jointly emitting actions and future observations.
Where the evidence stops. Detach marks the target latent's gradient boundary; it does not establish that every foundation model or encoder is permanently frozen. The future-image branch supplies observed targets, not evidence of a deployed future-image sensor or a specified pixel-generation objective.
2. Motivation
2.1 The problem and the proposed response
The paper targets camera-based trajectory planning without new manual perception annotations. Its motivating claim is that a single future latent learned from raw images inadequately captures geometry, semantics and alternative driving intentions, slowing learning and weakening planning. e-problem
2.2 What this reading follows
A camera image can support several plausible driving intentions, but a planner must eventually commit to one trajectory. World4Drive addresses this choice by predicting a latent future for each candidate and training a scoring network to recognize the future closest to the recorded observation. Its scene encoder combines metric-depth geometry, semantic pseudo-labels and recent visual history. The central reading question is how much the resulting ranking adds beyond better visual features and more trajectory candidates. The inspected v1 reports lower average collision rates than perception-free LAW on nuScenes and a higher NavSim PDM score, with qualifications about supervision, comparison settings and nighttime displacement error. e-identitye-contexte-dreamere-selectore-nuscenese-navsime-conditions
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
Figure 2 separates trajectory generation from an action-conditioned dreamer and selector, supporting Dual-system as a modular architecture despite end-to-end optimization. Already proposed actions condition future prediction; this is neither inverse-dynamics action extraction nor a single joint future/action generator. That supports Other mechanisms and Outside quadrants. Driving and latent-prediction tags fit; the paper does not identify a named JEPA implementation. e-architecturee-dreamere-selectore-context
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Build a scene representation before asking it to imagine
Begin with the distinction between an image feature and the information the planner needs. World4Drive adds depth-derived 3D positions to visual features so that image locations carry metric spatial context. A semantic head learns from high-confidence Grounded-SAM masks, and temporal cross-attention incorporates the preceding frame. These components shape the latent used by both trajectory generation and future prediction. The future observation also provides a latent target, with a detach marker in Figure 2. This is why the reconstruction terminology should be read at the feature level: the method specifies a latent MSE, rather than a pixel reconstruction decoder. The perception-label saving is therefore compatible with substantial supervision from pretrained teachers and with the expert trajectories used later in training. e-contexte-architecturee-selectore-loss
Figure 3. Geometry enters as a positional embedding; semantics enters through pseudo-label supervision. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the input image. The upper branch estimates metric depth D_t and constructs a 3D position map P_t; the diagram labels this conversion Re-project. The accompanying text describes obtaining ego-coordinate positions from depth and camera geometry. Sinusoidal positional encoding, labeled SPE, contributes to embedding E_t, which is added to backbone feature map F_t. Equation (3) includes a learned MLP after SPE that the schematic does not separately draw. Below the dashed line, a semantic head predicts masks from visual features and is trained against Grounded-SAM pseudo-labels with the semantic loss. Previous-frame aggregation is described separately in Equation (4), outside this crop. e-contexte-implementation
What it supports. The latent representation is trained with two distinct forms of prior information: metric spatial coordinates and semantic categories. This explains what the authors mean by a physical latent encoder. The design supplies cues useful for planning, but the diagram itself does not validate their accuracy or demonstrate a complete physical simulator.
Where the evidence stops. Perception-annotation-free describes the driving protocol's avoidance of manual perception labels. Semantic pseudo-labels and pretrained depth knowledge still supply supervision and assumptions. The source does not state the exact semantic confidence threshold, prompt list, or foundation-model freezing and caching policy.
5.2 Generate intentions, then evaluate their consequences
The trajectory vocabulary defines possible intentions through clustered endpoints, with six intention queries for each of the three driving commands. Those queries attend to the current scene and produce candidate waypoint trajectories. Each trajectory is then encoded into an action token, which conditions the dreamer's prediction of a future latent. The order matters: a trajectory exists before its future is imagined. During training, the candidate nearest the observed future latent provides index j, reconstruction supervision and the selected trajectory for expert imitation. ScoreNet learns to predict that index from imagined futures. During inference, the model repeats proposal and imagination, then chooses by ScoreNet probability. It does not consult the actual future. This is a trajectory-selection mechanism, rather than inverse dynamics that extracts actions from a desired observation. e-intentionse-dreamere-selectore-loss
Figure 4. The nearest observed-future latent supplies both a reconstruction target and a ranking label. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each colored row on the left as one intention-conditioned future prediction. The yellow rows on the right repeat the same actual future latent, so every candidate is compared against one recorded outcome. The blue double-headed arrows denote comparisons, not forward transitions between futures. The red highlighted pair illustrates the minimum-MSE candidate, whose distance becomes reconstruction loss. Its index j follows the curved arrow to supervise the focal-loss branch. Independently, predicted latents flow upward through ScoreNet and Softmax. At inference the observed-future comparisons disappear: the selected trajectory is the one associated with the highest predicted score, as stated below Equation (7). e-selectore-architecturee-loss
What it supports. The selector learns to approximate a training-time nearest-future decision using predicted latents alone. That makes the training target operationally available without future observations at deployment. It also clarifies the objective: the score favors resemblance to the logged future, rather than directly estimating collision probability or a hand-designed driving reward.
Where the evidence stops. One actual future supervises all hypothetical intentions. The diagram does not supply counterfactual observations for unexecuted trajectories. High score therefore should not be interpreted as a calibrated safety guarantee; the benchmark evidence must assess whether this proxy improves planning.
5.3 Ask whether better future matching means better driving
Reader interpretation: the selector's target makes an efficient learning problem, but matching a recorded future and selecting the safest counterfactual are different criteria. Table 3 provides partial empirical support: with physical priors and intentions retained, removing the world-model pathway worsens both average trajectory error and collision. Yet this comparison changes several connected operations, so it does not isolate ranking quality. Another pair, rows 4 and 6, reveals that adding intentions lowers collision while slightly increasing L2. The nighttime subset exhibits the same direction of tradeoff against LAW. Read the main results through those distinctions: lower collision is valuable evidence for the approach, while predictive accuracy, score calibration, counterfactual validity and real vehicle safety remain separate questions requiring dedicated checks. e-selectore-ablatione-conditionse-nuscenese-navsim
5.4 Training and inference
During training
ScoreNet learns the selected index with focal loss. The combined objective weights semantic, reconstruction, scoring and trajectory losses by 0.2, 0.2, 0.5 and 1.0. Only the selected trajectory receives the described L1 imitation loss; the system is end-to-end trainable. e-selectore-loss
nuScenes uses ResNet-50, six 360 × 640 views, 12 epochs, eight NVIDIA 3090 GPUs, batch size 8 and initial learning rate 5 × 10⁻⁵. NavSim uses ResNet-34, three stitched front views resized to 256 × 1024, 60 epochs, eight 3090 GPUs and batch size 64. e-implementation
During inference
Generate candidates and their predicted futures, then select the trajectory with the highest ScoreNet probability. Actual future observations and minimum-distance target construction belong to training. The world predictor remains in the inference path; it is not merely an auxiliary training loss. e-dreamere-selector
The output is a planned trajectory. The reported NavSim protocol interpolates it using an LQR controller; the paper's benchmark results do not document physical vehicle deployment. e-protocole-navsim
5.5 Implementation flow
- Represent intentions
Cluster endpoints of an 8,192-trajectory vocabulary into six intentions per command. Sinusoidal endpoint encodings are added to a learned ego query and processed by self-attention to form planning queries. e-intentions
- Encode physical context
Grounded-SAM supplies high-confidence semantic masks for a cross-entropy head. Metric depth locates pixels in ego coordinates; sinusoidal encoding and an MLP produce positional embeddings added to visual features. Cross-attention aggregates the previous frame into the current latent. e-context
- Propose and imagine
Planning queries cross-attend to scene latents and an MLP emits trajectories. An action MLP encodes each trajectory; future queries attend to concatenated action and scene features to predict intention-specific latents. The default prediction interval is n = 3 timestamps, not a stated three-second horizon. e-dreamere-selector
- Construct the training selection target
Compare each predicted latent with the actual future latent using MSE. The closest candidate defines index j, receives latent reconstruction loss and supplies the trajectory supervised against the expert. Figure 2 marks the actual future latent as detached. e-architecturee-selectore-loss
6. Experiments & results
World4Drive proposes several command-conditioned driving trajectories, predicts a future latent scene for each, and learns to rank them by resemblance to an observed future. Depth-derived positions, semantic pseudo-labels and temporal attention enrich its scene representation. It improves reported average collision metrics over perception-free LAW, while retaining expert-trajectory supervision and exhibiting important accuracy–collision tradeoffs.
6.1 Read the original evidence
Table 1. Average collision improves more strongly than average displacement error. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left metric block as displacement error in meters and the right block as collision rate in percent; both arrows point downward. Each block reports one-, two- and three-second entries plus an average. Red method cells identify approaches without manual perception annotations; blue cells identify approaches requiring them, according to the results text. Start with World4Drive versus perception-free LAW, then compare the perception-based LAW row to understand the remaining displacement gap. Keep the retained backbone footnote in view: LAW uses Swin-Tiny while the other listed methods use ResNet-50. These are reported benchmark comparisons, not a controlled experiment with identical encoders. e-protocole-nuscenes
What it supports. World4Drive reports average L2 of 0.50 m and collision of 0.16%, versus 0.61 m and 0.30% for perception-free LAW. Calculated from those rounded values, the collision reduction is 0.14 percentage points, or about 46.7% relative. Perception-based LAW remains slightly better in average L2 at 0.49 m.
Where the evidence stops. The evaluation is open-loop, with a three-second trajectory horizon. Collision rate here is not a demonstrated on-road failure probability. Different backbones and supervision regimes, together with absent uncertainty estimates, limit causal and statistical interpretation of the ranking.
Table 2. Camera-only World4Drive improves over the authors' LAW reimplementation, while DiffusionDrive remains ahead. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the Input column before comparing scores: C denotes cameras, L lidar and E ego status. Read NC as no at-fault collision, DAC as drivable area compliance, TTC as time-to-collision, Comf. as comfort and EP as ego progress. All reported directions are upward, and the shaded final column is the composite PDM score. Compare World4Drive with camera-only perception-free LAW to see the reported baseline gain, then inspect the camera-and-lidar DiffusionDrive row for the strongest listed result. The benchmark section specifies four-second trajectories sampled at 2 Hz with LQR interpolation; the implementation section says the authors reimplemented LAW for this evaluation. e-protocole-implementatione-navsime-nuscenes
What it supports. World4Drive's PDMS of 85.1 exceeds LAW's 83.8 by 1.3 score points. Its DAC, TTC and progress values also increase, while comfort stays at 100.0. DiffusionDrive reports 88.1 with camera and lidar inputs, so the table supports competitive performance without claiming the highest score across all settings.
Where the evidence stops. The paper calls this protocol closed-loop; this reading verifies its stated benchmark procedure, not simulator internals or physical deployment. The LAW result is an author reimplementation, and the table mixes modalities and supervision regimes without reporting run-to-run uncertainty.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Open-loop trajectory planning on nuScenes Reported nuScenes evaluation; trajectories sampled at 2 Hz over three seconds. Exact train/evaluation scene lists are not specified in this PDF. | World4Drive: L2 0.23 / 0.47 / 0.81 at 1 / 2 / 3 seconds, average 0.50; collision 0.02 / 0.12 / 0.33%, average 0.16%. L2 displacement error (m) and collision rate (%), lower is better | Perception-free LAW averages 0.61 m and 0.30%; perception-based LAW averages 0.49 m and 0.19%. LAW uses Swin-Tiny, whereas World4Drive uses ResNet-50. The rounded averages imply approximately 18.0% lower L2 and 46.7% lower collision than perception-free LAW (reader calculation). This is a cross-backbone comparison. World4Drive has the lowest average collision in Table 1, but not the lowest average L2. e-protocole-nuscenes |
| NavSim trajectory evaluation Paper-described closed-loop protocol on OpenScene: 1,192 training and 136 test scenarios, over 100,000 keyframes; four-second, 2 Hz trajectories with LQR interpolation. | World4Drive 85.1; components NC 97.4, DAC 94.3, TTC 92.8, comfort 100.0, ego progress 79.9. PDM score (PDMS), higher is better | The authors' camera-based LAW reimplementation scores 83.8; camera-and-lidar DiffusionDrive scores 88.1. The gain over the reimplemented LAW is 1.3 score points, not a measured road-deployment success rate. DiffusionDrive remains higher under a different input/supervision configuration. e-protocole-implementatione-navsim |
| Component ablation on nuScenes Table 3; L2 and collision averaged over the three-second prediction horizon. | Full model, row 6: 0.50 m / 0.16%. Average L2 (m) and collision rate (%) | With physical priors and world modeling but no intentions, row 4: 0.49 / 0.26. With physical priors and intentions but no world model, row 5: 0.61 / 0.36. Without semantics, row 3: 0.51 / 0.29. Adding intentions to row 4 lowers collision while slightly worsening L2. Rows 5–6 support the world-model pathway's utility, but removing that pathway changes prediction, supervision and selection together; it does not isolate ScoreNet alone. e-ablation |
| Nighttime planning on nuScenes Official scene-description grouping in Table 4; subset sizes and uncertainty are not supplied. | World4Drive: 0.76 m / 0.08%. Average L2 (m) and collision rate (%) | LAW: 0.67 m / 0.22%. Nighttime collision improves while displacement error worsens. This directly limits a blanket claim of improved performance under every lighting condition. e-conditions |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Read the checkmarks before interpreting the gains: collision and trajectory error need not move together. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The first two checkmark columns belong to the physical latent encoder; the next two indicate world modeling and intention modeling. Row 1 is the single-modal LAW baseline, and row 6 contains all four components. For a focused intention comparison, use rows 4 and 6: both include depth, semantics and world modeling. For the world-model pathway, use rows 5 and 6, where row 5 keeps intentions but lacks WM. Rows 3 and 6 compare the addition of semantic priors with depth, WM and intentions present. Section 4.4 specifies that these are nuScenes L2 and collision averages over a three-second horizon. e-ablation
What it supports. Adding intentions to the physical-prior world model changes L2 from 0.49 to 0.50 m while lowering collision from 0.26% to 0.16%. Adding the world-model pathway to row 5 improves both metrics, from 0.61 m/0.36% to 0.50 m/0.16%. Thus, the strongest supported claim concerns useful interactions and lower collision, not uniform improvement on every metric.
Where the evidence stops. The table is not a complete factorial study and includes no repeated-run uncertainty. Removing WM changes the predictor, its reconstruction supervision and its use in ranking; rows 5–6 cannot attribute the entire gain specifically to the scoring network.
7. Analysis & limitations
7.1 What the evidence leaves open
Latent resemblance to the logged future is the selector's training target. Calling its preferred trajectory safest requires additional validation: alternative intentions lack observed counterfactual outcomes, and benchmark collision reductions do not establish calibrated safety probabilities. e-selectore-nuscenese-navsim
The convergence plot labels LAW but its caption names PerAct. Its 3.75× claim concerns epochs, not measured wall-clock cost including foundation models. L2 improvement is inconsistently stated as 18.1% in the abstract, 18.2% in the introduction and 18.0% in results; Table 1 values are retained here. e-convergencee-probleme-nuscenes
Scaling prose gives incorrect row references and says dimension 125; Table 6 specifies 128. Figure 5 overlays ground-truth perception annotations, not model-predicted detections. Main result tables provide no seed variation or confidence intervals. e-scalinge-qualitativee-nuscenese-navsim
7.2 Questions for discussion
- Does selecting the future most similar to a logged demonstration also select the lowest-risk counterfactual trajectory?
- Can the nighttime collision gain be retained without the observed displacement-error penalty?
8. Reproducibility audit
8.1 Requirements and known gaps
Required assets include driving trajectories, camera calibration, the trajectory vocabulary, Metric3D v2 giant and Grounded-SAM outputs. The PDF does not fully specify vocabulary construction, semantic prompts/confidence threshold, teacher freezing or caching, optimizer/scheduler, software versions, random seeds or exact scene manifests. e-intentionse-contexte-implementation
Prioritize a fixed-candidate ScoreNet comparison against random ranking and privileged future-MSE ranking, followed by a matched semantic-prior ablation stratified by illumination. These reader-proposed checks would separate ranking utility from candidate quality and test the reported collision–accuracy tradeoff. e-selectore-ablatione-conditions
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold candidates fixed and test the selector
Reader-proposed check, not executed: freeze one trained encoder, planner and dreamer, and save the same candidate trajectories and predicted latents for held-out nuScenes scenes. Compare ScoreNet ranking with uniform random selection and minimum observed-future MSE. The last is a privileged diagnostic available only offline, never a deployable baseline. Report L2, collision, ScoreNet agreement with the MSE index and scene-level uncertainty under the same three-second evaluation. The ranking hypothesis weakens if ScoreNet does not improve over random choice. If privileged MSE selection itself has poor collision performance, the more fundamental problem is the future-matching target rather than its learned approximation. e-selectore-ablatione-protocol
Check 2: Separate semantic-prior benefit from lighting-dependent tradeoffs
Reader-proposed check, not executed: reproduce the Table 3 row-3 versus row-6 comparison with the same ResNet-50 initialization, depth features, six intentions per command, data split and 12-epoch budget. Change only semantic pseudo-supervision; document its prompts and confidence cutoff. Use at least three matched seeds as a proposed robustness control, then report scene counts, L2 and collision separately for day and night with uncertainty intervals. Evidence for the semantic mechanism would be a repeatable collision gain at matched candidate capacity. A gain confined to a small lighting subset, or consistently offset by worse nighttime displacement, would narrow the claim of robust physical scene understanding. e-ablatione-conditionse-contexte-implementation
8.3 Reading coverage
Visual audit: All four text chunks and all ten supplied PDF pages were read. Page 1 verifies title, eleven authors, affiliations and the arXiv v1 date, and exposes the Figure 1 LAW/PerAct caption conflict. Page 2 supplies the motivation and alternative stated improvement percentage. Pages 3–5 verify architecture, the detached target, context encoding, Equations (1)–(8), selector training/inference, protocols and hardware configuration. Page 6 verifies both benchmark tables, footnotes and the ablation evaluation setting. Page 7 verifies component checkmarks, lighting/maneuver subsets and scaling prose; page 8 verifies Table 6, qualitative overlays, the ground-truth-perception caption and supplement references. Pages 9–10 contain the references and no appendix. Figures 1–5 and Tables 1–6 were inspected. Six original crops from pages 3–7 were individually viewed after final cropping, retaining loss markers, table headers and relevant footnotes. Figure 3 omits the MLP explicitly present in Equation (3); the guide preserves that distinction. Separate supplements, external code and later editions were not supplied or inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Works (2.1–2.2)
- 3. Method (3.1–3.4, including 3.2.1–3.2.2 and 3.3.1–3.3.2)
- 4. Experiments (4.1–4.5, including 4.4.1–4.4.3)
- 5. Conclusion
- References [1]–[48]
Outside the original text pass
- Identity/version: the title and primary identifier match. The inspected artifact is arXiv:2507.00603v1 [cs.CV], dated 1 July 2025, with eleven authors. The catalog lists ICCV 2025 and additionally credits XianPeng Lang, absent from this title page. The venue edition and revision history were not supplied; no equivalence with that edition is asserted.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The extraction limitation was addressed by visually inspecting all ten PDF pages, Figures 1–5 and Tables 1–6, and every final crop.
- Separate supplemental material availability has not been fully verified.
- No supplement was supplied. The timestamp-interval and alternative-distance-loss ablations cited on page 5, and additional visualizations and failure analyses cited on page 8, were not available for review. No appendix is present in the supplied ten-page PDF.
- Code, external sources and foundation-model training data were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author block and arXiv margin stamp
The exact observed title matches the supplied primary record. The stamp reads arXiv:2507.00603v1 [cs.CV], 1 Jul 2025. Eleven credited authors run from Yupeng Zheng to Dongbin Zhao; Xianpeng Lang is absent. Affiliations are printed as CASIA, Li Auto, PCL, NUS and Tsinghua.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Section 1
The authors motivate perception-annotation-free planning through physical scene priors and multiple driving intentions. The introduction reports LAW averages of 0.61 m / 0.30% and World4Drive 0.50 m / 0.16%, calling the L2 reduction 18.2%.
Go to primary source ↓e-architecturePDF p. 3, Figure 2 and Section 3.1
The diagram separates intention encoder, physical latent encoder, planning decoder, action-conditioned world-model dreamer and selector. Future images feed a target context encoder; the actual future latent is marked Detach. Selected trajectories receive expert loss.
Go to primary source ↓e-intentionsPDF p. 3, Section 3.2.1, Equation (1)
K-means clusters trajectory-vocabulary endpoints by three command types. The defaults are N = 8192 vocabulary trajectories and K = 6 intentions per command. Positional intention queries and ego queries are combined with self-attention.
Go to primary source ↓e-contextPDF p. 4, Figure 3, Section 3.2.2 and Equations (2)–(4)
High-confidence Grounded-SAM semantic masks supervise a semantic head. Metric depth supplies ego-frame 3D positions. Equation (3) applies an MLP after sinusoidal encoding; its embedding is added to image features. Equation (4) cross-attends current and previous spatial-semantic features. Exact prompts, confidence cutoff and teacher freeze/cache policy are not specified.
Go to primary source ↓e-dreamerPDF p. 4, Section 3.3.1, Equations (5)–(6)
Planning queries attend to world latents and yield trajectories through an MLP. An action encoder maps trajectories to tokens. Learnable future queries cross-attend to action tokens concatenated with current world latents to predict one future latent per intention.
Go to primary source ↓e-selectorPDF p. 5, Figure 4 and Section 3.3.2, Equation (7); preceding continuation of Section 3.3.1
The default prediction interval is n = 3 timestamps. Training selects the candidate nearest the actual future latent by MSE, uses its distance as reconstruction loss and its index as focal supervision for ScoreNet. Inference selects the highest-scoring predicted future. Interval and alternative-distance ablations are referred to a supplement.
Go to primary source ↓e-lossPDF p. 5, Section 3.4, Equation (8)
The selected trajectory is trained with expert-trajectory L1 loss. Semantic, reconstruction, scoring and trajectory terms have respective default weights 0.2, 0.2, 0.5 and 1.0.
Go to primary source ↓e-protocolPDF p. 5, Section 4.1
nuScenes contains 1000 driving videos; evaluation uses 2 Hz trajectories over three seconds, L2 and collision rate. The paper describes NavSim as closed-loop, with 1192 training and 136 test OpenScene scenarios, over 100000 keyframes, four-second 2 Hz trajectories and LQR interpolation. PDMS incorporates NC, DAC, TTC, comfort and ego progress.
Go to primary source ↓e-implementationPDF p. 5, Section 4.2
nuScenes configuration: ResNet-50, six 360 × 640 views, six trajectories per command, 12 epochs, eight NVIDIA 3090 GPUs, batch 8 and learning rate 5e-5. NavSim: ResNet-34, three stitched front views resized to 256 × 1024, 60 epochs, eight 3090 GPUs and batch 64. The authors reimplement LAW for NavSim. Teachers are Metric3D v2 giant and Grounded-SAM.
Go to primary source ↓e-nuscenesPDF p. 6, Table 1, World4Drive and LAW rows, backbone footnote; Section 4.3
World4Drive L2 at 1/2/3 s is 0.23/0.47/0.81 m, average 0.50; collisions are 0.02/0.12/0.33%, average 0.16%. Perception-free LAW averages 0.61 m/0.30%; perception-based LAW 0.49 m/0.19%. The footnote identifies LAW's Swin-Tiny versus other methods' ResNet-50. Blue/red rows distinguish manual-perception supervision. Results prose states 18.0% L2 reduction.
Go to primary source ↓e-navsimPDF p. 6, Table 2, World4Drive, LAW and DiffusionDrive rows and modality footnote
Camera-only World4Drive reports NC 97.4, DAC 94.3, TTC 92.8, comfort 100.0, EP 79.9 and PDMS 85.1. Camera-only perception-free LAW scores 83.8 PDMS. Camera-and-lidar DiffusionDrive scores 88.1. The table reports no uncertainty intervals.
Go to primary source ↓e-ablationPDF p. 6, Section 4.4 protocol; PDF p. 7, Table 3 and Section 4.4.1
Rows 1–6 have L2/collision values 0.61/0.30, 0.55/0.25, 0.51/0.29, 0.49/0.26, 0.61/0.36 and 0.50/0.16. Checked columns identify row 3 as depth+WM+intentions; row 4 as depth+semantics+WM; row 5 as depth+semantics+intentions; row 6 contains all four components. Ablations are nuScenes averages over three seconds.
Go to primary source ↓e-conditionsPDF p. 7, Tables 4–5 and Section 4.4.2
Scene descriptions define day/night, sunny/rainy and maneuver subsets. Nighttime World4Drive is 0.76 m/0.08% versus LAW 0.67 m/0.22%; rainy values are 0.49 m/0.05% versus 0.54 m/0.16%. Table 5 reports lower collision rates for World4Drive for left, right and straight maneuvers. Subset counts and uncertainty are not given.
Go to primary source ↓e-scalingPDF p. 7, Section 4.4.3; PDF p. 8, Table 6
Table 6 lists ResNet-34/256, ResNet-50/128, ResNet-50/256, ResNet-50/384 and ResNet-101/256. At dimension 256, L2 values are 0.52, 0.50 and 0.47 m across the three backbones. The prose's row references do not match the table and its stated minimum dimension 125 conflicts with the table's 128.
Go to primary source ↓e-qualitativePDF p. 8, Figure 5 and caption; Section 4.5 continuation
The qualitative display compares trajectory overlays and shows ranked candidate trajectories. The caption explicitly says perception overlays are ground-truth annotations because World4Drive does not predict explicit perception outputs. Additional visualizations and failure analyses are deferred to supplementary material.
Go to primary source ↓e-convergencePDF p. 1, Figure 1, caption and Abstract
The convergence plot legend labels LAW and World4Drive, while its caption names PerAct. The x-axis is epochs and the y-axis is normalized inverse L2 performance, not raw success rate. The graphic and caption claim 3.75× fewer epochs and 1.18× peak performance. The abstract calls the L2 reduction 18.1%.
Go to primary source ↓8.5 Primary sources
World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model ↗
PDF · 6,565 extracted words
Source fingerprint
627cc44ed008f29584d4b8cdf02c2ef6f027c8051b22f8acc1691257ecf6f9db