Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
1. Paper overview
In one sentence: Geometry distillation and causal future-state prediction train compact driving representations, while deployment uses a direct trajectory decoder with no world-model rollout. e03e04e05e06e08e10e11e12e13
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors target representations that retain planning-relevant geometry and dynamics without carrying pixel-level video-generation costs into deployment. They argue that earlier latent planners remain insufficiently compressed, spatially weak, or temporally shallow. e02 |
| Core mechanism | |
| A key reported result | NAVSIM v2 trajectory planning: 89.3 EPDMS ↑. NAVSIM evaluation; source describes 103k training and 12k evaluation scenarios. DriveVLA-W0: 86.1; perception-annotated Drive-JEPA: 87.8. A 3.2-point gain over the strongest listed perception-free comparator, not uniform dominance: NC is 98.1 versus DriveVLA-W0’s 98.5. EPDMS combines multiplicative compliance terms and weighted progress/safety/comfort. e10e17 |
| Reading caution |
Core contributions
Figure 2. Two training signals shape the encoder used by the deployed planner. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the upper left: green image-patch tokens and purple scene queries pass through the vision encoder. The patch branch aligns with geometric targets; the purple outputs become compact scene representations. In the lower training panel, these scene tokens join yellow ego-status tokens and supply context to the dynamic predictor. Follow the dashed EMA-update arrow to the frozen target encoder and the two supervision branches for latent world status and ego status. Finally, read the right-hand planning panel: current scene–ego tokens and trajectory queries feed the trajectory decoder, with the driving command selecting the intended trajectory. e03e04e05e07e08e09
What it supports. The figure separates representation learning from trajectory generation. Geometry alignment teaches the backbone spatial structure, while latent and ego prediction teach temporal information. Section 3.5 explicitly retains only SCWE and the trajectory decoder for inference, so the model’s planning results do not demonstrate test-time search through imagined futures.
Where the evidence stops. The flame and snowflake distinguish trainable and target encoders; the target still changes through EMA. The schematic token counts are illustrative. The reported implementation uses 16 scene queries per camera, and low-level actuator control is not shown.
2. Motivation
2.1 The problem and the proposed response
The authors target representations that retain planning-relevant geometry and dynamics without carrying pixel-level video-generation costs into deployment. They argue that earlier latent planners remain insufficiently compressed, spatially weak, or temporally shallow. e02
2.2 What this reading follows
Latent-WAM asks how much of a driving scene must survive compression for a planner to make useful decisions. It appends learned scene queries to camera-image tokens, teaches the encoder geometry through a frozen foundation model, and trains a causal predictor to anticipate scene and ego-state changes. The resulting representation feeds a separate trajectory decoder. The crucial distinction is that future prediction shapes training, while inference uses the current representation. Read the architecture and attention mask first, then use the benchmark and ablation tables to assess the gains, their unevenness across driving conditions, and the evidence for geometric supervision. e03e04e05e06e08e10e11e12e13
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Other mechanisms |
| Quadrant | Outside quadrants |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Dual-system / Other mechanisms / Outside quadrants assessment is supported by the separate latent transition predictor and trajectory decoder. Future scene and ego prediction trains the representation; trajectories come from another decoder using current state. This is neither inverse-dynamics action extraction nor a joint future/action generator used at inference. Joint optimization alone would not establish One Model architecture. Dual-system here describes module separation, not two deployed control systems. e03e05e07e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Make geometry survive scene compression
The scene queries are a bottleneck inserted into the vision encoder: they interact with the image patches before the compressed tokens are passed onward. Geometric alignment acts on the patch-feature branch, so the shared backbone learns under both geometric and planning objectives. WorldMirror supplies the geometric features without receiving gradient updates. Its targets can therefore be cached before planner training. The appendix makes that cache concrete: resize and crop each camera image, adjust its intrinsic matrix, invert the camera-to-world transform when needed for projection, and enable camera pose and intrinsic conditioning while disabling depth conditioning. The downstream model uses 16 queries per camera. Reader calculation: three views plus one ego token yield 49 world-status tokens per frame, a compact context for the later predictor. e04e05e09e18e19
Figure 3. Historical frame blocks provide context without exposing the future target frame. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read rows as future queries Q and columns as historical keys K. Purple and yellow cells permit attention; gray cells block it. The first query block is labeled frame 2, while its accessible key block is frame 1. The frame-3 queries can access keys from frames 1 and 2. Continue to the bottom block to see the same pattern for frame T. This index offset matters: the filled block on the apparent diagonal is still historical context. Section 3.3 describes teacher forcing and bidirectional interaction within frame blocks, with temporal causality enforced between them. e05e06e07e08
What it supports. The mask lets training predict multiple future blocks in parallel while conditioning on observed history. The scene and ego parts share that history, tying visual prediction to the vehicle’s evolving status. This supports the paper’s training formulation; it is not an empirical measurement of free-running prediction accuracy.
Where the evidence stops. Teacher forcing does not test accumulated rollout error. Separately, the three 3D-RoPE axes are time, camera index and token index; the name does not imply that attention receives explicit metric x/y/z coordinates.
5.2 2. Use prediction to train a planner that does not roll out futures
The world state combines scene tokens with a learned embedding of driving command, velocity and acceleration. During training, future queries predict subsequent world states from historical context, using the frame-block mask to prevent access to future target frames. An EMA-updated encoder supplies latent targets, while separate heads decode future commands and motion variables for direct supervision. These losses coexist with expert-trajectory imitation and geometric alignment in Equation (6). At inference, Section 3.5 removes the prediction machinery. The trajectory decoder uses the current world state and learned trajectory queries to produce candidates, then selects according to the driving command. Reader interpretation: the experiment tests whether forecasting is useful supervision for representation learning, rather than whether a planner can choose actions by evaluating imagined futures. e05e06e07e08
5.3 3. Separate aggregate improvement from the mechanism it proves
The strongest benchmark statement is the NAVSIM v2 EPDMS result of 89.3, exceeding the best listed perception-free comparator by 3.2 points. That score is a composite, and the individual columns expose safety, comfort and progress tradeoffs. HUGSIM adds a distinct test: the NAVSIM-trained model acts in photorealistic reconstructed environments without fine-tuning. Its average completion leads, while its HD-Score ties UniAD and its hard-scenario result is substantially lower. The ablations offer more targeted evidence for geometry and dynamics, but their point estimates lack uncertainty. In particular, changing the temporal schedule also changes historical context and the number of targets. Reader interpretation: the experiments justify investigating these training signals, while a matched-context study is still needed to isolate the benefit of intermediate prediction. e10e11e12e13e15e17
5.4 Training and inference
During training
Training jointly optimizes trajectory imitation, geometric alignment, future-latent MSE and ego-state supervision. The configuration uses four temporal frames, 100 epochs, batch size 512 and 32 A100 GPUs for approximately two days. AdamW uses learning rate 0.0002, weight decay 0.05, 10% linear warm-up, cosine decay to 0.000001 and BF16. The paper reports 191M training parameters, 104M trainable. e08e09
Teacher forcing supplies observed historical tokens instead of rolling predictions forward. Frame blocks permit bidirectional interaction internally, while cross-frame access is causal; this supports parallel training predictions. e06
During inference
Only SCWE and the trajectory decoder are required. The geometric teacher, EMA target encoder, DLWM and auxiliary heads are training machinery. Trajectories come directly from current state, without test-time imagined rollouts or world-model candidate scoring; low-level actuator control is not specified. e03e08
5.5 Implementation flow
- Compress the visual scene
Append 16 learned queries per camera to image patch tokens in DINOv2-Base. Its 768-dimensional features project to a 256-dimensional latent space. Compact scene tokens continue into world-state aggregation; patch features also receive geometric supervision. e04e09
- Teach spatial structure
Project patch features toward WorldMirror’s 2048-dimensional targets and align their LayerNorm-normalized features by cosine similarity. WorldMirror is frozen; targets are cached offline using camera-pose and intrinsic priors, with depth conditioning disabled. e04e19
- Predict scene–ego transitions
Concatenate camera scene tokens with an MLP embedding of command, velocity and acceleration. Learned future queries attend to historical frame blocks. An EMA-updated target encoder supplies latent targets; separate heads supervise future command, velocity and acceleration. 3D-RoPE encodes time, camera and token indices. e05e06e07
- Decode the driving trajectory
Trajectory queries attend to the current world state. An MLP yields K candidate trajectories, and the current command selects one. DLWM and trajectory decoder each have four layers, eight heads, hidden width 256 and FFN width 1024. e08e09
6. Experiments & results
Latent-WAM trains a compact camera-based driving planner using geometric feature distillation and future latent-state prediction. Its deployment choice is to discard the dynamic predictor and decode trajectories from the current scene–ego representation. Reported gains concern NAVSIM planning scores and zero-shot HUGSIM simulation, with an HD-Score tie rather than an outright win (e03, e08, e10, e11).
6.1 Read the original evidence
Table 1. A higher composite planning score coexists with component-level tradeoffs. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the shaded EPDMS column, then compare methods inside the two supervision groups. Latent-WAM scores 89.3, DriveVLA-W0 86.1, and the perception-annotated Drive-JEPA 87.8. The dagger note preserves the annotation distinction. Then scan leftward: NC is no at-fault collision, DAC drivable-area compliance, DDC driving-direction compliance and TLC traffic-light compliance; EP measures ego progress and TTC time to collision. LK, HC and EC cover lane keeping and comfort. Appendix A defines EPDMS through products and a weighted average, so the final column is not a simple arithmetic mean of the printed component columns. e09e10e17
What it supports. The 3.2-point EPDMS improvement over DriveVLA-W0 is directly supported. Yet Latent-WAM’s NC is 98.1 versus 98.5, and its ego progress is 87.7 versus Epona’s 88.6. The table therefore supports stronger aggregate benchmark performance with tradeoffs across individual measures, rather than superiority on every driving criterion.
Where the evidence stops. The entries are reported benchmark scores without uncertainty intervals. Training scale and supervision differ across methods. As a reader check, averaging the nine printed component columns cannot reconstruct EPDMS; consult Appendix A’s formula and the evaluation aggregation.
Table 2. Zero-shot simulation yields the highest average completion and a tied driving score. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read this table in two halves. Road completion, RC, describes how much of the route is completed; HD-Score also incorporates driving quality through the appendix’s collision, drivable-area, time-to-collision and comfort terms. Each half lists easy, medium, hard and extreme scenarios before its reported average. The model is transferred from NAVSIM without HUGSIM fine-tuning and evaluated on 436 pre-challenge scenarios. Compare the average columns first, then inspect difficulty-specific values: the full-table view prevents strong easy-scenario performance from hiding weak performance in harder cases. Keep the printed difficulty legend when reading either half. e11e17e21
What it supports. Latent-WAM’s average RC of 45.9 exceeds LTF’s 41.4 and UniAD’s 40.6. Its 28.9 HD-Score matches UniAD. On hard scenarios, however, its HD-Score is 12.2 against UniAD’s 27.3. Zero-shot transfer is supported, but the gains are uneven across difficulty levels.
Where the evidence stops. The paper does not give the difficulty counts or averaging weights needed to reconstruct the Avg. columns. These reconstructed-environment tests and the appendix’s image sequences do not establish physical-road safety.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM v2 trajectory planning NAVSIM evaluation; source describes 103k training and 12k evaluation scenarios. | 89.3 EPDMS ↑ | DriveVLA-W0: 86.1; perception-annotated Drive-JEPA: 87.8. A 3.2-point gain over the strongest listed perception-free comparator, not uniform dominance: NC is 98.1 versus DriveVLA-W0’s 98.5. EPDMS combines multiplicative compliance terms and weighted progress/safety/comfort. e10e17 |
| Zero-shot HUGSIM closed-loop driving NAVSIM-trained model, no HUGSIM fine-tuning; pre-challenge test set, 436 scenarios, four difficulty levels. | 45.9 / 28.9 Average RC / HD-Score ↑ | LTF: 41.4 / 24.8; UniAD: 40.6 / 28.9. Highest reported average road completion; HD-Score ties UniAD. Hard HD-Score is 12.2 versus UniAD’s 27.3. These are simulated driving results. e11 |
| Component ablation NAVSIM v2 progressive component study. | 87.9 baseline; 87.7 compression; 88.0 with world model; 88.3 with ego supervision; 89.3 full. EPDMS ↑ | Compression plus geometry without dynamics: 88.6; adding world modeling: 89.0. Geometry and dynamics improve this configuration; the full gain over the uncompressed baseline is 1.4 points. Colored deltas refer to that baseline. e12 |
| Geometry injection ablation Compressed model with world modeling and ego supervision. | 89.3 distillation EPDMS ↑ | 88.3 without geometry; 88.0 concatenation. Distillation improves by 1.0 point over no geometry and 1.3 over concatenation. The proposed feature-misalignment explanation is an author hypothesis. e13 |
| Backbone and adaptation ablation NAVSIM v2 geometric-distillation study. | Base: 89.3; Small: 86.3 EPDMS ↑ | Base-LoRA: 68.5; Small-LoRA: 84.7. Full fine-tuning wins in these settings; unspecified LoRA configuration prevents a general conclusion about low-rank adaptation. e14 |
| Temporal prediction schedule ablation NAVSIM v2; all schedules reach frame 8. | 89.3 for −3 → 0 → 4 → 8 EPDMS ↑ | 88.4 for 0 → 8; 89.1 for −3 → −2 → −1 → 0 → 2 → 4 → 6 → 8. More history and intermediate targets help here; the comparison changes both, and does not isolate either effect. e15 |
| Inference latency Single A100, one batch; three warm-up iterations, average of ten forward passes. | 107 ms / 1.1 GB / 104M Latency / memory / reported parameters | World encoder: 100 ms, 86.6M; trajectory decoder: 6 ms, 8.4M. Reported measurement, not a deployment guarantee. Listed component parameters sum to 95M, leaving the 104M total unreconciled. e20 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 3. Compression creates the representation bottleneck; geometry and dynamics recover and improve planning. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each checkmark as enabling the named component, and compare rows that differ in one column. The all-cross baseline sends image patch tokens directly to the trajectory decoder. Compression alone changes 87.9 to 87.7. With compression retained, adding the world model raises the score to 88.0, and ego supervision raises it to 88.3. A second branch adds geometry to compression and reaches 88.6; world modeling and ego supervision then reach 89.0 and 89.3. The colored suffixes use 87.9 as their reference, not the immediately preceding row, as specified in the original caption. e08e12
What it supports. The full system improves by 1.4 EPDMS points over the uncompressed baseline. Geometry’s gain is visible even without the dynamic predictor, while the world-model and ego-supervision rows improve the geometry-equipped configuration further. These comparisons support contributions from both spatial and temporal training signals in this setup.
Where the evidence stops. The table is a progressive study, not a complete factorial experiment. There are no repeated-run uncertainty estimates, so small differences cannot establish statistical reliability or quantify a general interaction between geometry and dynamics.
Table 4. How geometric information enters the encoder matters in the reported comparison. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Hold the surrounding model in mind: the no-geometry row already includes compression, world modeling and ego-state supervision. The concatenation row supplies frozen geometric features as key/value inputs, whereas distillation trains the vision backbone to align projected image-patch features with teacher targets. Read the three scores as a comparison of injection methods within that context. Equation (2) explains the distillation route through LayerNorm and cosine similarity; Appendix B.4 identifies the frozen WorldMirror targets and their camera priors. The best row therefore reflects a particular way of teaching the representation, not merely the presence of additional geometric features. e04e12e13e19
What it supports. Distillation reaches 89.3 EPDMS, 1.0 point above the 88.3 no-geometry model and 1.3 above concatenation. The fact that concatenation scores 88.0 suggests that simply exposing the planner to teacher features is insufficient in this experiment; learning how the encoder represents them is consequential.
Where the evidence stops. The authors’ explanation that concatenation creates conflicting or misaligned signals remains a hypothesis. This table does not isolate geometric correctness from feature regularization, extra supervision or optimization differences, motivating a shuffled-target control.
7. Analysis & limitations
7.1 What the evidence leaves open
Tables report point estimates without seeds, run variation or confidence intervals. Cross-method training data and supervision differ, so the data-efficiency claim is not a controlled scaling experiment. HUGSIM gains also vary sharply by difficulty. e02e10e11e12e13e14e15
Attention examples suggest more concentrated spatial focus after distillation, but do not establish causal understanding. Trajectory and HUGSIM image sequences are selected qualitative examples, not additional aggregate safety evidence or physical-road trials. e16e21e11
7.2 Questions for discussion
- Does correctly paired geometric supervision outperform equally costly shuffled teacher targets? (e04, e13)
- With identical historical context and optimization budget, do intermediate future targets improve planning beyond final-frame prediction? (e06, e15)
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs NAVSIM data, DINOv2-Base, WorldMirror targets and HUGSIM evaluation. Follow the appendix’s width-by-height preprocessing: 1920×1080 → 455×256 → 448×224. Scale focal lengths, shift principal points for cropping, invert camera-to-world extrinsics, and cache WorldMirror features with condition flags [1,0,1]. e09e18e19
Unspecified details include EMA decay, exact K and pose count, command-class mapping, LoRA rank/target layers, software versions and trajectory-to-control conversion. Verify parameter accounting and metric aggregation: Eq. (9) writes a sum from 0 through T divided by T despite describing a temporal average. e05e07e08e09e14e17e20
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does correctly paired geometry matter beyond an auxiliary feature loss?
Reader-proposed check, not run: train the same compressed model with world modeling and ego supervision under three conditions—no geometric alignment, correctly paired cached WorldMirror targets, and targets shuffled across training samples within the same camera view. Preserve the source’s preprocessing and camera conditioning; hold initialization, optimization budget and loss weight fixed, and repeat with at least three seeds. Compare paired NAVSIM EPDMS and its safety components, reporting uncertainty across runs and scenarios. If correctly paired targets do not reliably outperform shuffled ones, the explanation that spatially correct teacher information drives the gain is weakened, even if auxiliary training itself helps. e04e09e12e13e18e19
Check 2: Do intermediate predictions help when history is held fixed?
Reader-proposed check, not run: provide identical observed frames −3 and 0, then compare supervision only at frame 8 with supervision at frames 4 and 8. In both conditions, block ground-truth frame-4 tokens from frame-8 context; this deliberately controls the information supplied by teacher forcing. Keep the frame-8 horizon, normalize losses per target, and hold optimizer updates, batch size and seeds fixed. Add a compute-matched comparison if target density changes training cost. Measure NAVSIM EPDMS, per-horizon latent error and training time across repeated runs. A reliable planning gain would support intermediate supervision; its disappearance would implicate added context or optimization differences in the published schedule comparison. e05e06e08e09e15
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page, all original Figures 1–15 and Tables 1–7, main method/equation/training pages, and supplementary metric, preprocessing, geometric-target and latency pages. All six final original crops were viewed; the causal-mask crop was widened and re-inspected to preserve its row-axis label. Figure 2 arrows, EMA/frozen markers and separate planning branch were cross-checked against Sections 3.1–3.5; Figure 3 query/key indices and mask colors were checked against Section 3.3 and Equation (3), with no claim-relevant conflict found. Reference-only pages 16–18 were read in the complete text but not rendered. No external supplement, code or reproduction was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations, abstract and arXiv v1 stamp (p. 1)
- 1 Introduction (pp. 1–3)
- 2 Related Works, 2.1–2.2 (pp. 3–4)
- 3 Method, 3.1–3.5 (pp. 4–8)
- 4 Experiment, 4.1–4.3 (pp. 9–14)
- 5 Conclusion (p. 15)
- References (pp. 15–18)
- Supplementary A. Metrics, A.1–A.2 (p. 19)
- B. Data Processing Pipeline, B.1–B.4 (pp. 20–21)
- C. Inference Latency (p. 21)
- D. More Visulization, D.1–D.2 (pp. 22–24)
- Source-labeled C.3. HUGSIM (pp. 25–29)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The reviewed artifact is arXiv:2603.24581v1 [cs.CV], dated 25 March 2026. Its title and all sixteen authors match the catalog identity; the title page spells Junyu Han with a capital H, whereas the catalog has Junyu han. No other revision or edition was supplied or compared.
- The supplied extraction does not reconstruct figure images; this gap was addressed by inspecting the original PDF figures, tables and equation layouts. Reference-only pages 16–18 were read as text but not rendered.
- Separate supplemental material availability has not been fully verified; only the supplement embedded in this PDF was reviewed.
- No code, external resources or software environments were inspected, and no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author/affiliation block and arXiv margin stamp
Exact title matches the supplied observed title. Sixteen authors, from Linbo Wang to Dongbin Zhao, and five affiliations are printed. The stamp is arXiv:2603.24581v1 [cs.CV], 25 Mar 2026; Junyu Han is capitalized.
Go to primary source ↓e02PDF pp. 2–3, Section 1 and Figure 1/caption
Motivation contrasts expensive video generation with undercompressed latent planners lacking spatial or historical information. Figure 1 compares performance against differing training-data scales and model sizes; World4Drive has an additional depth estimator.
Go to primary source ↓e03PDF p. 4, Figure 2 and Section 3.1
Architecture separates SCWE, training-time DLWM with EMA target supervision, and trajectory planning. Geometric and temporal objectives optimize the backbone; the planning horizon is four seconds.
Go to primary source ↓e04PDF p. 5, Section 3.2, Equations (1)–(2)
Scene queries concatenate with patch tokens in the DINO encoder. Projected patch features and WorldMirror geometric features undergo LayerNorm and cosine-similarity alignment.
Go to primary source ↓e05PDF p. 6, Section 3.3, world latent aggregation and Equation (3)
Camera scene tokens plus one ego embedding form M×N+1 tokens per frame. Future queries use historical keys/values; an EMA-updated frozen SCWE provides stable targets.
Go to primary source ↓e06PDF p. 7, Section 3.3, Figure 3 and 3D-RoPE paragraphs
Teacher-forced blocks use historical observed context. Figure 3 shows frame-2 queries accessing frame-1 keys, and frame-3 queries accessing frames 1–2. Positional axes are time, camera and token, with stated frequencies 50, 10 and 100.
Go to primary source ↓e07PDF pp. 7–8, Section 3.3 ego status supervision and Equation (4)
Future ego embeddings feed command, velocity and acceleration heads. Command logits have dimension four; velocity and acceleration each have dimension two.
Go to primary source ↓e08PDF p. 8, Sections 3.4–3.5, Equations (5)–(6) and Inference paragraph
Current world status and trajectory queries yield K candidates, selected by command, with ego-local x/y/heading poses. Training combines trajectory L1, alignment, latent MSE and ego losses with weights 0.1/0.2/0.1. Only SCWE and trajectory decoder are required at inference.
Go to primary source ↓e09PDF p. 9, Section 4.1, Architecture and Training Configuration
DINOv2-Base, 16 queries, dimensions 768/256, four frames, three cameras, 2048-dimensional geometry targets and four-layer/eight-head decoders are specified. Training uses 32 A100s, 100 epochs, batch 512, AdamW, BF16 and the stated learning schedule; parameter totals are 104M inference and 191M training, 104M trainable.
Go to primary source ↓e10PDF pp. 9–10, Section 4.2 NAVSIM; p. 10, Table 1, Ours and comparator rows
Source describes 103k training/12k evaluation scenarios. Ours reports EPDMS 89.3 against DriveVLA-W0 86.1 and Drive-JEPA 87.8. Ours NC/DAC/EP/EC are 98.1/97.3/87.7/87.3; DriveVLA-W0 NC is 98.5; Epona EP is 88.6. Table groups perception-based and perception-free methods.
Go to primary source ↓e11PDF pp. 10–11, Section 4.2 HUGSIM; p. 10, Table 2 and difficulty footnote
Zero-shot evaluation uses the NAVSIM-trained model without fine-tuning on 436 pre-challenge scenarios. Ours average RC/HD-Score are 45.9/28.9, UniAD 40.6/28.9 and LTF 41.4/24.8. Ours HD-Score by easy/medium/hard/extreme is 72.5/24.0/12.2/18.1; UniAD hard is 27.3. Environments are photorealistic reconstructions with changing camera viewpoints.
Go to primary source ↓e12PDF p. 11, Table 3, all rows; Section 4.3 Effectiveness of Each Component
EPDMS sequence is 87.9 baseline, 87.7 compression, 88.0 compression+world model, 88.3 with ego status, 88.6 compression+geometry, 89.0 with world model, and 89.3 all components. Caption defines deltas relative to baseline.
Go to primary source ↓e13PDF pp. 11–12, Impact of Geometric Information; p. 12, Table 4
No geometry, concatenation and distillation score 88.3, 88.0 and 89.3. Authors hypothesize frozen-feature misalignment explains concatenation degradation.
Go to primary source ↓e14PDF p. 12, Table 5 and Vision Backbone for Geometric Distillation
Small/Base score 86.3/89.3 and Small-LoRA/Base-LoRA 84.7/68.5. Authors attribute LoRA degradation to low-rank constraints, but do not specify rank or target layers here.
Go to primary source ↓e15PDF pp. 12–13, World Model Prediction Temporal Stride; p. 13, Table 6
Schedules 0→8, −3→0→4→8 and −3→−2→−1→0→2→4→6→8 score 88.4, 89.3 and 89.1. They retain the same final horizon while varying historical context and prediction density.
Go to primary source ↓e16PDF pp. 13–14, Figures 4–5/captions and Qualitative Analysis
Trajectory comparisons use green human and yellow predicted paths. Paired attention maps compare models with/without geometry, across straight/right/left intentions; authors interpret the patterns as more spatially focused after distillation.
Go to primary source ↓e17PDF p. 19, Appendix A.1–A.2, Equations (7)–(9)
EPDMS multiplies NC, DAC, DDC and TLC by a weighted average of EP, TTC, LK, HC and EC. HUGSIM combines NC, DAC, TTC and comfort, then scales a temporal average by route completion. Equation (9) prints t=0 through T with denominator T.
Go to primary source ↓e18PDF p. 20, Appendix B.1–B.3, Equations (10)–(16)
Images resize from 1920×1080 to 455×256 then center-crop to 448×224. Intrinsics are rescaled and crop-adjusted; camera-to-world transforms are inverted for world-to-camera projection.
Go to primary source ↓e19PDF p. 21, Appendix B.4, Equations (17)–(18)
Frozen WorldMirror uses camera pose and intrinsics but excludes depth conditioning via flags [1,0,1]. Its 2048-dimensional patch features are precomputed and cached for all training samples.
Go to primary source ↓e20PDF p. 21, Appendix C and Table 7, all rows
Single-A100 inference is averaged over ten forward passes after three warm-ups for a single batch. Table reports encoder 86.6M/100ms, decoder 8.4M/6ms and all modules 104M/1.1GB/107ms; per-module memory is omitted.
Go to primary source ↓e21PDF pp. 22–29, Appendix D.1–D.2 and source-labeled C.3, Figures 6–15/captions
Additional NAVSIM trajectory and attention examples include forward, lane-change, left-turn and right-turn scenes. HUGSIM trajectory images depict nuScenes, KITTI-360, Waymo and PandaSet reconstructions; no additional aggregate measurements are attached.
Go to primary source ↓8.5 Primary sources
Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving ↗
PDF · 7,390 extracted words
Source fingerprint
b02d50f25f82296cc364859aa883aadd5edce0156a175f168b26112a18253429