Enhancing End-to-End Autonomous Driving with Latent World Model
1. Paper overview
In one sentence: LAW improves a driving planner by predicting action-conditioned future features during training, trading explicit future-image generation for an auxiliary latent objective whose gains depend on target horizon and evaluation setting. e02e03e04e05e06e07e09e10e11e12e14
| At a glance | What to know |
|---|---|
| Research problem | Source description Trajectory supervision alone may underconstrain scene representations, while explicit perception tasks require additional annotations. The authors propose temporal self-supervision that accounts for ego motion: predict the representation of a future observation using current features and planned waypoints. Perception-free still requires waypoint supervision. e02e05 |
| Core mechanism | |
| A key reported result | CARLA Town05 Long closed-loop driving: LAW: RC 97.8±0.9, IS 0.72±0.03, DS 70.1±2.6. Route completion (RC), infraction score (IS), driving score (DS); higher is better.. Town05 Long; training uses 189K Roach-teacher frames from CARLA 0.9.10.1. Interfuser DS: 68.3±1.9; TCP: 57.2±1.5. Simulator execution evidence. The PDF does not define the ± statistic or repetition count; significance is unestablished. e07e11 |
| Reading caution |
Core contributions
Figure 2. The planner supplies the action that conditions its auxiliary future-feature predictor. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the upper row from the current images to the encoder, visual latents and waypoint decoder. Both current latents and predicted waypoints enter the action-aware representation, which the world model transforms into predicted latents. The green supervision arrows point from actual future-image latents toward those predictions. Below, the left planner uses waypoint queries Q with visual latents as keys and values K,V. The BEV alternative first extracts agent and map features, which then supply the waypoint decoder. Its blue supervision arrows target agent-motion and map outputs. These directions agree with the caption and Sections 4.1–4.3. e03e04e05e06
What it supports. The same auxiliary forecasting interface can train two different perception representations. The world model consumes an already predicted trajectory; the figure does not show a predicted future being decoded back into an action. This supports an action-conditioned representation-learning interpretation of LAW's architecture.
Where the evidence stops. Neither this diagram nor the method specifies target stop-gradient or frozen weights. Section 4.1 also calls the latent loss MSE while Equation (3) prints an unsquared L2 norm; the drawing cannot resolve that implementation ambiguity.
2. Motivation
2.1 The problem and the proposed response
Trajectory supervision alone may underconstrain scene representations, while explicit perception tasks require additional annotations. The authors propose temporal self-supervision that accounts for ego motion: predict the representation of a future observation using current features and planned waypoints. Perception-free still requires waypoint supervision. e02e05
2.2 What this reading follows
A driving planner can fit expert waypoints without learning every scene detail that later matters. LAW adds a second demand: the current representation and predicted trajectory must help predict features of a future observation. The target comes from encoding an actual future image, making the auxiliary signal available without new perception annotations. This reading follows the information flow through both planner variants, then separates benchmark rankings from controlled evidence about action conditioning. The reported improvements span open-loop datasets and a closed-loop simulator, but their interpretation depends on metric conventions, component tradeoffs and an unresolved mismatch between the written loss and its printed equation. e02e03e04e05e06e07e09e10e11e12e14
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The snapshot is supported at the module level: Figure 2 separates the waypoint-producing planner from an action-conditioned future-latent predictor. Joint training does not make this one joint future/action decoder. Auxiliary forward prediction supports Other mechanisms and Outside quadrants. Dual-system should not imply two deployed control loops; latent prediction is supported without establishing a specific JEPA implementation. e02e03e04e05e06
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Understand what the future prediction is training
Begin with a normal supervised planner: its encoder extracts a scene representation and its decoder predicts ego waypoints. LAW flattens those predictions and concatenates them with each latent feature before an MLP constructs action-aware latents. A transformer predicts the future representation, while an encoder processes the actual future images to supply targets. The additional loss therefore links scene representation, predicted action and temporal consistency during training. The paper describes improvement of both representation and trajectory prediction, but does not fully specify target-branch gradient handling. A reader should therefore keep the computational dependency separate from an assumed implementation of backpropagation. There is also a concrete loss ambiguity: the prose specifies MSE, whereas the printed equation uses an unsquared L2 norm. Neither issue licenses adding an undocumented momentum teacher. e02e03e04e05
5.2 Trace why the same auxiliary task fits two planners
In the perception-free path, each view query retrieves information from its own image feature map. Waypoint queries then attend directly to the resulting compact visual latents. The BEV path instead projects image features into a spatial representation, uses agent and map queries to extract task features, and passes those features to waypoint decoding. Thus the decoder's information route changes, but the auxiliary interface still receives current latents and predicted waypoints and predicts future latents. Perception-free means no extra perception annotations for this planner, not no supervised learning: ground-truth waypoints remain necessary. In the BEV variant, agent-motion and map losses remain alongside waypoint and latent losses. Figure 2's green future-supervision arrows and blue perception-supervision arrows express these distinct training roles. e03e04e05e06
5.3 Separate representation evidence from a driving-safety claim
Reader analysis: the clearest mechanism evidence comes from comparisons that change the auxiliary task within the same planner. Table 4 supports trajectory conditioning, and Table 5 shows gains on NAVSIM and CARLA when latent prediction is enabled. Yet NAVSIM's aggregate improvement can coexist with weaker components than other methods, and CARLA's ablation improves infraction score while slightly reducing route completion. Target timing also matters: Table 6 favors a moderate horizon over a very distant target. Appendix A.1 adds useful temporal evidence through shared-weight autoregressive prediction and historical inputs, but uses a separate front-view setup and extra fine-tuning. Together these findings support a training method whose benefits depend on configuration. They do not establish counterfactual simulator fidelity, physical safety, or that every visual feature relevant to driving has been captured. e07e10e12e13e14e16
5.4 Training and inference
During training
nuScenes perception-free: Swin-T, 800×320 images, AdamW, initial learning rate 5e-5 with cosine annealing, weight decay 0.01, batch size 8, and 12 epochs on eight A6000 GPUs. Perception-based: 48 epochs of encoder/perception-head training, then 12 with waypoint and latent losses; the predictor uses deformable self-attention. e08
NAVSIM uses ResNet-34, 640×320 images, Adam at 1e-4, batch size 32 and 20 epochs. CARLA uses ResNet-34 with a TCP head, 900×256 images, Adam at 1e-4, weight decay 1e-7, batch size 128 and 60 epochs; learning rate halves after epoch 30. e08
During inference
The depicted planner maps current images through encoder and decoder to waypoints; future-image supervision belongs to training. No inference-time candidate search or inverse-dynamics action extraction is specified. CARLA uses the inherited TCP head and simulator feedback; detailed control conversion is not restated. e02e03e05e07e08
5.5 Implementation flow
- Encode a scene
For perception-free planning, a backbone and PETR-style 3D position embeddings produce image features. Each learnable view query cross-attends to its own view, yielding one latent per view. Learnable waypoint queries attend to those latents, and an MLP predicts waypoints. e05
- Condition on the predicted action
Flatten the M two-dimensional predicted waypoints into a vector of length 2M. Concatenate it with every current visual latent and apply an MLP, retaining the original feature dimension. The conditioning input is the planner prediction, distinct from the supervised trajectory target. e03e04
- Predict the future representation
Transformer blocks apply self-attention across action-aware latent vectors and feed-forward updates. Predicted future features are matched to features encoded from actual future images. The paper describes joint optimization of scene features and trajectory prediction through this auxiliary task. e02e04
- Use the BEV alternative
The perception-based encoder projects image features into BEV queries and flattens the map. Agent and map queries cross-attend to BEV latents; waypoint queries attend to the resulting agent/map features. Agent-motion and map-construction supervision remain present. e03e06
6. Experiments & results
LAW trains an end-to-end driving planner to predict future visual features from current features and its predicted ego trajectory. This auxiliary task works with perspective-view or BEV representations. Controlled ablations, open-loop driving metrics and CARLA closed-loop scores support useful representation learning, while leaving counterfactual dynamics accuracy and physical deployment untested.
6.1 Read the original evidence
Table 1. The strongest nuScenes averages come from the perception-based LAW variant; collision conventions require care. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row across two metric groups: trajectory L2 error in metres and collision rate in percent, both lower being better. Within each group, distinguish individual prediction times from the shaded average. The last two rows are separate LAW architectures, with different supervision requirements. Compare the perception-based row with VAD to see the reported average gains. Keep the note above the table in view: BEV-Planner's double-dagger collision entries use another computation, while the remaining collision entries follow the traditional protocol named in the caption. LAW excludes historical ego status information in this comparison. e05e06e07e09
What it supports. Perception-based LAW reports 0.49 m average L2 and 0.19% collision, compared with VAD's 0.72 m and 0.22%. Perception-free LAW reports 0.61 m and 0.30%, so the two LAW variants should not be treated as interchangeable evidence for the same supervision budget.
Where the evidence stops. These are open-loop trajectory metrics, not physical collision frequencies. The supplied PDF does not name the nuScenes split or supply uncertainty here. Cross-method rankings also do not isolate the effect of the latent objective.
Table 2. NAVSIM's aggregate score improves while individual safety-related components reveal tradeoffs. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the shaded PDMS column, the predictive driver model score, then trace the component columns that contribute to it. NC means no at-fault collision; DAC is drivable area compliance; TTC is time-to-collision; Comf. is comfort; EP is ego progress. Every arrow points upward. LAW is evaluated in the perception-free setting on the test set. The human row supplies a separate reference rather than a learned-model competitor. Comparing LAW with TransFuser and PARA-Drive component by component prevents the aggregate ranking from being mistaken for improvement on every aspect of driving. e07e10
What it supports. LAW's PDMS is 84.6, versus 84.0 for TransFuser and PARA-Drive. Its DAC of 95.4 and ego progress of 81.7 accompany lower NC and TTC than those methods. The result supports an aggregate improvement with a specific component tradeoff, rather than uniform dominance.
Where the evidence stops. The table supplies no uncertainty, so the small aggregate gap does not establish statistical significance. The human PDMS of 94.8 remains substantially higher, and the protocol does not demonstrate physical deployment.
Table 3. CARLA tests executed behavior with sensor feedback, while supervision differs across competitors. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the supervision column before comparing the driving scores. Expert denotes imitation from a privileged expert's trajectories; Dep., Seg., Map. and Det. denote depth estimation, semantic segmentation, BEV map segmentation and 3D object detection. LAW adds latent prediction to expert supervision. RC measures route completion, IS rewards fewer infractions, and DS combines completion with infractions. The rightmost shaded column is the primary score. All arrows favor larger values. Preserve the reported plus-minus entries and missing dashes as printed; they should not be silently converted into standard errors or filled in from other papers. e07e08e11e13
What it supports. LAW has the highest listed DS, 70.1±2.6, versus Interfuser's 68.3±1.9. This benchmark evaluates a policy operating in a feedback loop in the simulator. The separate Table 5 ablation is more targeted evidence for the auxiliary task than the cross-method ranking alone.
Where the evidence stops. The PDF does not define the plus-minus statistic or repetition count. Do not infer significance or recompute DS from aggregate RC and IS entries. CARLA evidence also does not establish transfer to physical vehicles.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| nuScenes open-loop trajectory planning Reported nuScenes protocol; split not named in this PDF; no historical ego status. | Perception-based LAW: 0.49 m / 0.19%; perception-free: 0.61 m / 0.30%. Average L2 displacement (m) and collision rate (%) over 1/2/3 seconds; lower is better. | VAD: 0.72 m / 0.22%. Perception-based LAW improves both averages. BEV-Planner collision entries use a different computation and are not directly comparable. e07e09 |
| NAVSIM test-set planning NAVSIM test set; LAW is perception-free. | PDMS 84.6; NC 96.4, DAC 95.4, TTC 88.7, comfort 99.9, ego progress 81.7. Predictive driver model score (PDMS); no at-fault collision (NC), drivable area compliance (DAC), time-to-collision (TTC), comfort and ego progress; higher is better. | TransFuser and PARA-Drive: PDMS 84.0; human: 94.8. The aggregate advantage coexists with lower NC and TTC than these learned baselines. e07e10 |
| CARLA Town05 Long closed-loop driving Town05 Long; training uses 189K Roach-teacher frames from CARLA 0.9.10.1. | LAW: RC 97.8±0.9, IS 0.72±0.03, DS 70.1±2.6. Route completion (RC), infraction score (IS), driving score (DS); higher is better. | Interfuser DS: 68.3±1.9; TCP: 57.2±1.5. Simulator execution evidence. The PDF does not define the ± statistic or repetition count; significance is unestablished. e07e11 |
| Action-conditioning ablation nuScenes; comparisons within each planner framework. | Perception-free: 0.71/0.41 without LAW, 0.68/0.37 with visual input only, 0.61/0.30 with visual and trajectory inputs. Average L2 (m) / collision (%). | Perception-based counterparts: 0.54/0.25, 0.52/0.21, 0.49/0.19. Both frameworks benefit from trajectory conditioning; this measures planning, not latent prediction accuracy. e12 |
| Latent-prediction ablation across NAVSIM and CARLA Perception-free; auxiliary latent prediction disabled versus enabled. | PDMS 77.5→84.6; DS 67.9±2.1→70.1±2.6. PDMS; CARLA DS, RC and IS. | CARLA RC 98.6±0.8→97.8±0.9; IS 0.68±0.02→0.72±0.03. The higher CARLA score accompanies improved infraction score but slightly lower completion. e13 |
| Future-target horizon ablation nuScenes perception-free; target offset varied independently of evaluation horizons. | 0.5 s: 0.61/0.30; 1.5 s: 0.58/0.25; 3 s: 0.63/0.27; 10 s: 0.72/0.43. Average L2 (m) / collision (%). | The 1.5-second target is best on both averages. Longer target offsets do not monotonically improve planning. e14 |
| Temporal-input and autoregressive extensions Appendix A.1 front-view temporal experiments; 12 initial epochs and 6 fine-tuning epochs for the history comparison. | Predicting 1.5 s then 3 s: 0.69/0.29 versus 0.73/0.32 for 1.5 s alone. Average L2 (m) / collision (%). | After fine-tuning: current input only 0.68/0.33; current plus −1.5 s input 0.55/0.17. Rollout steps share predictor weights. This appendix configuration is distinct from main-table multi-view results. e16 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Conditioning on the predicted trajectory improves over visual-only future-feature training in both planner variants. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read downward within one framework before moving to the other. Two dashes in the input columns mean no world model, not a world model receiving zero-valued inputs. The next row enables visual-latent input alone; the final row also enables the predicted trajectory. Compare the shaded averages while checking the separate one-, two- and three-second columns for the same trend. The upper group uses perception-free perspective-view latents; the lower group retains supervised BEV perception. This layout tests the contribution of conditioning within each framework more directly than comparing unrelated leaderboard rows. e04e05e06e12
What it supports. In the perception-free group, average L2 falls from 0.71 to 0.68 to 0.61 m, and collision from 0.41% to 0.37% to 0.30%. The perception-based group improves in the same order. These results support including the predicted action in the auxiliary learning task.
Where the evidence stops. The columns measure downstream planning rather than accuracy of latent forecasts. They cannot establish that the predictor responds correctly to counterfactual actions, or distinguish informative conditioning from all possible regularization effects.
Table 6. Choosing a more distant future target helps only up to a point in this experiment. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Separate the leftmost Time Horizon from the times inside the metric groups. The row label specifies how far ahead the auxiliary target features lie; the one-, two- and three-second columns measure the planner's waypoint performance. On nuScenes, the target offsets correspond to the first, third, sixth and twentieth future keyframes because keyframes are spaced by 0.5 seconds. Use the shaded averages to compare target choices. The first row reproduces the default perception-free figures in Table 4, while the later rows test whether the training signal improves as the target becomes more distant. e12e14
What it supports. The 1.5-second target gives the best averages, 0.58 m L2 and 0.25% collision. A 10-second target gives 0.72 m and 0.43%, losing the benefit seen with shorter targets. The experiment makes target timing a consequential training choice rather than a neutral implementation detail.
Where the evidence stops. The authors explain this pattern through insufficient change at short offsets and excessive difficulty at long ones. Those are interpretations; the table does not report latent error, target variance or evidence of a universal optimal horizon.
7. Analysis & limitations
7.1 What the evidence leaves open
Open-loop results do not establish physical driving safety; closed-loop evidence is confined to CARLA. Selected map/motion examples do not establish failure frequencies, especially for occlusion. e07e17
Table 4 does not measure future-latent error or counterfactual action fidelity. Table 7 favors transformers over an MLP but lacks a capacity-matched control, so attributing all gains to cross-latent interaction is not isolated. e12e15
7.2 Questions for discussion
- Would shuffled trajectories retain the gains, suggesting generic regularization rather than meaningful action conditioning? (e04, e12)
- How sensitive are results to resolving Equation (3) and target-encoder gradient handling? (e04)
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction needs synchronized current/future images, waypoint targets, benchmark evaluators, and perception annotations for the BEV variant. Preserve Table 1 collision conventions and separate appendix camera/history settings from main results. e04e06e09e16
Resolve the MSE-versus-norm discrepancy, target-encoder gradient handling, encoder weight sharing, predictor depth/width and checkpoint initialization. The PDF does not fully specify these details or the nuScenes split. No frozen or momentum target encoder is explicitly documented. e03e04e07e08
Proposed checks should isolate trajectory information and target-loss implementation while holding data, architecture and schedule fixed; compare planning and latent error. The code-release statement does not establish a tested implementation. e01e04e08e12
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does trajectory content drive the action-conditioning gain?
Reader-proposed, not run: reproduce the perception-free nuScenes comparison with identical data pairs, target horizon, backbone, predictor dimensions and training schedule. Compare correctly paired predicted trajectories with trajectories shuffled across examples within each training batch, plus the no-latent-loss baseline. Preserve the same concatenation width and gradient policy in both conditioned arms. Across repeated seeds, record held-out latent error and the identical L2/collision evaluation. If correctly paired trajectories consistently improve planning and forecasting relative to shuffled ones, the information content of conditioning receives stronger support. If shuffled conditioning retains the gain, the interpretation of Table 4 as evidence for meaningful action-dependent dynamics weakens. e04e08e09e12e14
Check 2: Test the unresolved latent-loss and target-gradient choices
Reader-proposed, not run: implement a two-by-two comparison of the printed unsquared L2 objective versus the prose-defined MSE objective, with future-target gradients enabled versus stopped. Use a shared encoder as an explicitly chosen reconstruction assumption, hold all other components and the training schedule fixed, and document each loss reduction convention. Track feature variance, branch gradient norms, latent prediction error and planning metrics across seeds. A collapsed representation, unstable training or material ranking changes in one cell would show that the omitted implementation choices affect reproducibility. Consistent performance across cells would reduce that concern, but would not identify which implementation produced the paper's results. e03e04e05e08e12
8.3 Reading coverage
Visual audit: Inspected the title/author/version page; both architecture figures; all method equations and training/evaluation details; Tables 1–9; and qualitative Figures 3–6. Figure 2 arrow directions, Q/K,V labels and supervision targets were checked against its caption and Sections 4.1–4.3. Page 5 verifies the MSE-versus-unsquared-norm discrepancy. Six final original crops were inspected at their native rendered resolution; Table 1 retains its collision-protocol note. Pages 7 and 15 supply training and appendix configuration evidence even though they are not cropped. Reference pages 11–14 were read as text. Separate supplementary video and code were not inspected. Original citation-link rectangles and missing table entries are preserved.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 15, 16, 17, 18. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–3: introduction, related work and preliminary definitions (pp. 1–4)
- Sections 4.1–4.3: latent world model and both planner frameworks (pp. 4–6)
- Sections 5.1–5.5: benchmarks, implementation, comparisons, ablations and visualization (pp. 6–10)
- Sections 6–7: conclusion and acknowledgments; references (pp. 10–14)
- Appendix A.1–A.2: temporal extensions and visualizations (pp. 15–18)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Identity: the title and all seven authors match the catalog. The inspected artifact is arXiv:2406.08481v2, dated 28 February 2025, labeled an ICLR 2025 conference paper. No earlier version was supplied, so changes from v1 cannot be established.
- The supplied extraction does not reconstruct figure images; this limitation was addressed by inspecting original PDF pages and all final crops. Reference pages 11–14 were read as text, without a separate visual pass.
- Separate supplemental material availability has not been fully verified. The CARLA demo mentioned in Appendix A.2 was not supplied or viewed.
- The linked code and external references were not inspected, and no training or driving experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block, arXiv margin and abstract
Observed title matches the supplied title; authors are Yingyan Li, Lue Fan, Jiawei He, Yuqi Wang, Yuntao Chen, Zhaoxiang Zhang and Tieniu Tan. The page labels arXiv:2406.08481v2, 28 February 2025, ICLR 2025, and lists four affiliations. The abstract states code is released.
Go to primary source ↓e02PDF pp. 1–2, Section 1 and Figure 1/caption
Temporal future-feature supervision is conditioned on current scene features and predicted ego trajectories; perception annotations are optional additions. Future features supervise training of the current scene representation and trajectory prediction.
Go to primary source ↓e03PDF pp. 3–4, Section 3; p. 4, Figure 2/caption and Section 4.1
Waypoints are future BEV ego positions. Figure 2 distinguishes encoder/decoder planning, the latent predictor, and the future-image encoder target. Lower panels show waypoint query versus feature key/value flow and supervised map/agent outputs.
Go to primary source ↓e04PDF p. 5, Section 4.1, Equations (1)–(3)
Predicted waypoints are flattened, concatenated with each visual latent, and mapped by an MLP. Transformer blocks predict future latents with self-attention across vectors. Prose specifies MSE, but Equation (3) prints an unsquared L2 norm; stop-gradient and a frozen/EMA target encoder are not specified.
Go to primary source ↓e05PDF pp. 5–6, Section 4.2, Equations (4)–(7)
Perspective-view features receive 3D position embeddings. One learnable query per view cross-attends to its image features; waypoint queries attend to resulting visual latents. The loss adds latent prediction to mean L1 waypoint supervision.
Go to primary source ↓e06PDF p. 6, Section 4.3, Equations (8)–(9)
BEV features support separate map and agent queries; their features inform waypoint decoding. Perception-based training adds VAD-defined motion-prediction and map-construction losses to waypoint and latent losses.
Go to primary source ↓e07PDF pp. 6–7, Section 5.1, benchmark descriptions
nuScenes evaluates trajectory displacement and collisions. NAVSIM resamples OpenScene driving logs and reports five PDMS components. CARLA 0.9.10.1 training uses 189K teacher frames; Town05 Long provides closed-loop evaluation. DS is defined as RC multiplied by IS.
Go to primary source ↓e08PDF p. 7, Section 5.2, all three benchmark implementation paragraphs
The paper specifies nuScenes Swin-T/AdamW training on eight A6000 GPUs, staged BEV training and deformable self-attention; NAVSIM ResNet-34/Adam training; and CARLA ResNet-34/TCP-head training with benchmark-specific image sizes, batches, epochs and learning rates.
Go to primary source ↓e09PDF p. 8, Table 1, caption and VAD/BEV-Planner/LAW rows
LAW perception-based reports average L2 0.49 m and collision 0.19%; perception-free 0.61 m and 0.30%; VAD 0.72 m and 0.22%. Caption excludes historical ego status and distinguishes the double-dagger-marked BEV-Planner collision computation.
Go to primary source ↓e10PDF p. 8, Table 2, NAVSIM test-set caption and all rows
LAW PDMS is 84.6 versus 84.0 for TransFuser/PARA-Drive and 94.8 for human. LAW components are NC 96.4, DAC 95.4, TTC 88.7, comfort 99.9 and EP 81.7. TransFuser NC/TTC are 97.7/92.8; PARA-Drive 97.9/93.0.
Go to primary source ↓e11PDF p. 8, Table 3, Town05 Long caption and LAW/Interfuser/TCP rows
LAW reports RC 97.8±0.9, IS 0.72±0.03 and DS 70.1±2.6; Interfuser DS 68.3±1.9, TCP 57.2±1.5. Supervision types differ by row. The caption does not define repetitions or the ± statistic.
Go to primary source ↓e12PDF p. 9, Table 4, caption and both framework row groups
No input means no world model. Adding visual latents, then predicted trajectories, lowers average L2/collision from 0.71/0.41 through 0.68/0.37 to 0.61/0.30 for perception-free; corresponding BEV results are 0.54/0.25, 0.52/0.21, 0.49/0.19.
Go to primary source ↓e13PDF p. 9, Table 5 and following paragraph
Enabling latent prediction changes NAVSIM PDMS 77.5 to 84.6, DAC 89.4 to 95.4 and EP 75.1 to 81.7. CARLA DS rises 67.9±2.1 to 70.1±2.6, IS 0.68±0.02 to 0.72±0.03, while RC falls 98.6±0.8 to 97.8±0.9.
Go to primary source ↓e14PDF p. 9, Section 5.4, time-horizon paragraph and Table 6
Target offsets 0.5, 1.5, 3 and 10 seconds give average L2/collision 0.61/0.30, 0.58/0.25, 0.63/0.27 and 0.72/0.43. They correspond to keyframe offsets 1, 3, 6 and 20 at 0.5-second spacing. Authors attribute the pattern to target difficulty.
Go to primary source ↓e15PDF pp. 9–10, network-architecture ablation paragraph; p. 10, Table 7
Linear projection, two-layer MLP and transformer blocks give average L2/collision 0.70/0.42, 0.64/0.33 and 0.61/0.30. The authors favor interaction across latent vectors; parameter-matched controls are not provided.
Go to primary source ↓e16PDF p. 15, Appendix A.1, Tables 8–9 and surrounding training description
The front-view experiment autoregressively predicts 1.5-second then 3-second latents with shared weights, improving 0.73/0.32 to 0.69/0.29. A 12-epoch initial stage plus 6-epoch fine-tuning compares current-only inputs (0.68/0.33) with current and −1.5-second inputs (0.55/0.17).
Go to primary source ↓e17PDF p. 10, Figure 3/caption; pp. 16–18, Figures 4–6/captions; p. 15, Appendix A.2
Selected LAW-versus-VAD illustrations highlight map reconstruction, agent motion and an example collision trajectory, including occluded/crowded scenes. Appendix A.2 mentions a CARLA demo in separate supplementary materials; the PDF figures are qualitative examples.
Go to primary source ↓8.5 Primary sources
ENHANCING END-TO-END AUTONOMOUS DRIVING WITH LATENT WORLD MODEL ↗
PDF · 7,985 extracted words
Source fingerprint
33bc426172f6f2b83bb8fd6408e768f95dd085aabeeeb0ef768fe95afa1c0586