Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
1. Paper overview
In one sentence: A cached forecast of motion latents guides a per-step driving policy, with reported NAVSIM gains offset by unresolved result discrepancies and incomplete implementation details. e02e03e04e06e09e11e19
| At a glance | What to know |
|---|---|
| Research problem | Author claim Long-horizon scene prediction and immediate action generation have different temporal requirements. The authors argue that a common rollout schedule either sacrifices anticipation or accumulates action errors; their hierarchy preserves future context while accepting new observations at every action step. e02e03 |
| Core mechanism | |
| A key reported result | NAVSIM v1 driving: 93.8, table-reported PDMS ↑. Single front camera; Table I comparison. Exact evaluation split and sample count are unspecified. DriveVLA-W0‡: 93.0; human reference: 94.8. Reader-computed improvement over the baseline: 0.8 points. Best listed learned-policy aggregate, below the human reference. Section IV.B instead states 93.3. The ‡ baseline uses best-of-6; equivalent selection for Drive-HWM is not documented. e09e11e08 |
| Reading caution | Source description The authors acknowledge computational/memory overhead and missing explicit multimodal-future or uncertainty modeling. Their Y-intersection example shows insufficient yielding and a potential collision in generated observations conditioned on a predicted trajectory, not an observed physical crash. e19e21 |
Core contributions
Figure 2. The hierarchy passes predicted motion representations into a policy that receives new visual evidence every step. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel from bottom to top, then follow the gold arrow between them. The left panel turns observed context into a sequence of Dynamic-Aware Latents, with optical flow supplying motion supervision. The arrow carries latent features into FiLM on the right; it does not carry an explicit driving command. The red schedule labels distinguish a slow update every N steps from fast prediction each step. Equations (11) and (17)–(18) specify the operative details: select the latent aligned with the next transition, then modulate transformer hidden states. Although two outputs are drawn, Section III.C explicitly allows next-image generation to be omitted at inference. e03e04e05e06e14
What it supports. The architectural contribution is an inference-time path from predicted future dynamics to immediate actions. Its fast branch also learns an action-conditioned visual transition during training. This separates the continuing use of a learned future representation from the optional cost of decoding a future image.
Where the evidence stops. The graphic labels actions A_t→A_{t+1}; Eqs. (15), (20)–(24) use prior actions→A_t. It places FiLM beside the expert, while Eq. (18) specifies transformer hidden-state modulation. Exact layers remain unspecified. VL-JEPA here is named V-JEPA in Table V.
2. Motivation
2.1 The problem and the proposed response
Long-horizon scene prediction and immediate action generation have different temporal requirements. The authors argue that a common rollout schedule either sacrifices anticipation or accumulates action errors; their hierarchy preserves future context while accepting new observations at every action step. e02e03
2.2 What this reading follows
Drive-HWM asks how a driving model can anticipate scene changes without committing to a long sequence of actions before seeing new evidence. Its answer is a slow predictor that supplies a cache of future motion representations and a fast policy that revisits the camera observation at every step. The choice of supervision matters: optical flow trains the slow representation, while next-frame RGB prediction trains the fast expert. Follow the architecture into the component and target ablations, then examine the failure case. The reported benchmark gains are promising, but the tables disagree with several prose summaries, and generated future images do not establish safe physical execution. e02e03e04e06e09e11e19
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The catalog is entirely unassigned. Architecture evidence suggests a hierarchical system with two predictive components, not a single undifferentiated world-action model. The fast component jointly learns actions and action-conditioned future visuals; the slow predictor supplies motion latents. This is neither inverse dynamics nor proof that image generation is required for control. No quadrant is treated as an existing assignment. e03e04e05e06
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Build a future cache without rolling out actions
At a slow update, Drive-HWM reads the observed interaction history and predicts several future DALs. Each latent is trained to decode the optical flow between two consecutive future observations. All offsets are predicted from the same history, so a mistaken early flow is not recursively fed into the next prediction. The fast policy then selects the latent aligned with its next transition. With the reported N=K=8 schedule, the cache spans the full eight-step update interval. A direct consequence of Equation (11) is that successive fast steps use successive offsets from that cache, while receiving fresh observations of their own. This is a way to reuse predicted context; it does not remove uncertainty in later offsets or update the slow prediction when an unexpected event occurs between scheduled refreshes. e03e04e07e08
5.2 Locate the world prediction that survives deployment
Two different predictive pathways need to be kept separate. The slow branch's DAL is consumed by the fast model at inference. A FiLM projection converts it to scales and shifts of normalized hidden channels, after which the autoregressive expert predicts action tokens. The fast branch's next-frame prediction is instead an auxiliary training task, conditioned on the action as well as the observed context and DAL. The source explicitly permits skipping image decoding at inference. Reader interpretation: this makes the system a policy guided by a learned future representation, without requiring a rendered future image in every control cycle. It also explains why next-frame accuracy alone would be an incomplete test: the decisive questions are whether the latent changes actions usefully and whether those actions perform well under the evaluation protocol. e04e06e07e08
5.3 Test the target asymmetry without overstating the benchmark
The slow-target and fast-target ablations favor different signals. Flow wins when supervising the slow predictor; RGB wins as the fast policy's auxiliary target. The authors explain this through motion-focused anticipation versus preservation of immediate scene semantics. Reader interpretation: the evidence motivates a division of labor, but the separate ablations do not measure the interaction between those choices. The efficiency table adds another constraint: on one H200 at batch one, the reported 84.8 ms average includes amortized slow computation, while an update step takes 107.2 ms. These are hardware-specific latency results, not a documented deployed control frequency. Read them alongside the unresolved table–prose score conflicts and the yielding failure, rather than treating better averages as proof of robust interactive driving. e11e12e15e18e19
5.4 Training and inference
During training
Jointly pretrain backbone, slow model and fast model on nuPlan for 10K steps with the full objective; fine-tune the complete model on NAVSIM for 6K steps using action loss. Settings: 256×144 images, eight NVIDIA A100 GPUs, batch size 48, AdamW, initial learning rate 2×10⁻⁴ and cosine scheduling. e08
The detailed formulation combines weighted multi-offset flow regression with action-token and next-image-token negative log-likelihoods. Fast steps receive predicted slow latents during training. Section III.D switches to generic latent-distance notation without fully specifying its relationship to these concrete losses or numerical loss weights. e04e06e07
During inference
Refresh the slow prediction every eight steps and temporally index its cache between updates. The fast model incorporates current observations each step and decodes action tokens. Future image generation may be omitted: it is auxiliary supervision, whereas the predicted DAL remains an inference-time input. e06e07e08
5.5 Implementation flow
- Encode the observed history
Observation and previous-action history form the context. Figure 2 additionally depicts an instruction. The fast branch tokenizes images with the Emu3 tokenizer and uses Emu3-8B contextual hidden states. e03e05e06
- Predict motion at several future offsets
At each slow update, all future flow fields are predicted from the same observed history, without recursively feeding predicted flow back. Targets are flows between consecutive future observations. The hidden representation before flow decoding becomes the DAL; raw flow is not the policy input. e04
- Align and modulate
Select the DAL for the next transition from the most recent cached prediction. FiLM projects that latent into channel scales and shifts applied to normalized transformer hidden states, without appending extra input tokens. e04e06
- Generate actions and supervise the transition
A driving-specific autoregressive expert predicts ordered action tokens. During training it also predicts next-frame visual tokens conditioned on those actions, current context and the DAL. No inverse-dynamics action recovery or candidate-search procedure is specified. e06
6. Experiments & results
Drive-HWM couples a periodically refreshed predictor of future motion latents with an observation-grounded autoregressive driving policy. Optical flow supervises the slow branch; next-frame RGB tokens supervise the fast branch during training. NAVSIM tables report stronger aggregate driving scores, but conflicting prose and incomplete implementation details limit precise reproduction.
6.1 Read the original evidence
Table I. Table I reports a single-camera PDMS advantage over the listed learned baselines, subject to protocol and prose discrepancies. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with Sensors and the retained footnotes before comparing scores. Some rows use multiple cameras or LiDAR, while Drive-HWM and the marked DriveVLA-W0 variants use one front camera. Follow the highlighted row across NC, DAC, TTC, comfort and ego progress before reading PDMS at the far right. The strongest listed DriveVLA-W0 aggregate is the double-dagger row, whose footnote specifies autoregressive action prediction with best-of-6 selection. Drive-HWM has no corresponding marker, so the table does not establish identical selection settings. Keep the human row separate from learned methods when describing the leading score. e08e09e11
What it supports. The displayed Drive-HWM aggregate is 93.8 versus 93.0 for DriveVLA-W0‡, a reader-computed 0.8-point gain. Its NC, TTC, comfort and progress values are 99.6, 98.5, 100.0 and 89.0. The human reference remains higher on aggregate at 94.8; Drive-HWM does not lead every individual column.
Where the evidence stops. Section IV.B instead reports PDMS 93.3 and DAC 98.4; this crop shows 93.8 and 99.0. The conflict is unresolved. Exact evaluation splits, sampling uncertainty and equal decoding budgets are not established by this table.
Figure 6. The authors' failure case exposes insufficient yielding in a trajectory-conditioned generated future. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the row labels before interpreting the apparent collision. The top row contains observed inputs; the next row contains ground-truth future observations. The last two rows are generated futures conditioned on different trajectories. Compare the approaching vehicle across the bottom row from the first to the fourth future frame, then inspect the bird's-eye trajectory overlay and its green/red legend. The figure's collision annotation belongs to the predicted-trajectory-conditioned sequence. The authors describe the event as a potential collision caused by inadequate yielding at an unsignalized Y-intersection; it is not a direct recording of physical deployment. e19e21
What it supports. The example shows that plausible future generation and aggregate benchmark gains can coexist with a serious interaction failure. It motivates the authors' stated need to represent multimodal futures and predictive uncertainty, especially when another vehicle's behavior determines whether an ego maneuver is safe.
Where the evidence stops. One selected generated rollout cannot establish a failure rate or a verified physical crash. The safe-looking ground-truth-conditioned row also does not show that Drive-HWM can select that trajectory autonomously. The figure is qualitative evidence of a failure mode.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM v1 driving Single front camera; Table I comparison. Exact evaluation split and sample count are unspecified. | 93.8, table-reported PDMS ↑ | DriveVLA-W0‡: 93.0; human reference: 94.8. Reader-computed improvement over the baseline: 0.8 points. Best listed learned-policy aggregate, below the human reference. Section IV.B instead states 93.3. The ‡ baseline uses best-of-6; equivalent selection for Drive-HWM is not documented. e09e11e08 |
| NAVSIM v2 driving Extended NAVSIM evaluation; exact split unspecified. | 86.4, table-reported EPDMS ↑ | DriveVLA-W0: 86.1; DiffusionDrive: 84.5. The table supports a 0.3-point advantage over DriveVLA-W0. Prose instead reports 86.2. These results do not establish physical vehicle deployment. e10e11e08 |
| Inference efficiency Single NVIDIA H200, batch size 1, slow interval 8. | Fast 81.6; slow 25.6; average 84.8; update-step peak 107.2. Latency in milliseconds ↓ | DriveVLA-W0: 117.8; fast-only: 81.6. Average latency is fast latency plus slow latency divided by the interval. Savings versus the baseline are about 28%; peak latency and extra memory remain relevant. e12e21 |
| Hierarchy and prediction horizon Table IV component/horizon ablation. | Full model K=8: 93.8; K=4: 93.0; K=12: 93.2. PDMS ↑ | Fast-only: 93.0; slow-only: 90.2. Eight steps perform best among tested horizons. The altered update schedule and slow-only action interface are unspecified; prose uses a conflicting full-model score of 93.3. e13e11 |
| Slow prediction target Figure 4; authors hold other configurations fixed. | Optical flow: 93.8 PDMS ↑ | BEV: 92.8; depth: 93.0; RGB: 92.2. Supports flow supervision in this setup; no uncertainty or repeated-run statistics establish the reliability of small differences. e15 |
| Fast auxiliary target Table VIII; all variants retain action supervision. | Next-frame RGB: 93.8 PDMS ↑ | No auxiliary target: 93.1; next-flow: 93.5; next-depth: 93.6. The preferred fast target differs from the slow target. Auxiliary training gains do not require image decoding during driving. e18e06 |
| Motion representation probing Table VI; frozen slow models with lightweight linear probes. Probe label definitions and splits are unspecified. | Optical-flow latent: 83.7 / 76.1 Future Ego Motion / Motion Consistency probing accuracy ↑ | RGB latent: 76.8 / 69.4. Supports more linearly accessible motion information in the flow latent; this is not a driving score or a calibrated forecast-error measure. e16 |
| Slow-to-fast conditioning Table VII; authors vary conditioning while keeping other model configurations unchanged. | FiLM: 93.8, table-reported PDMS ↑ | Concatenation: 92.5; cross-attention: 93.0; gated cross-attention: 93.1; AdaLN: 93.3. FiLM leads this ablation, but its prose instead reports 93.3. No repeated-run statistics establish the reliability of the margin. e17e11 |
| Human preference 20 raters, 50 paired scenarios, Drive-HWM versus DriveVLA-W0. | 68% Drive-HWM; 21% baseline; 11% no preference. Preference share | Ratings use a 1–5 Likert scale across four criteria. Perceived behavior quality is not an objective collision rate; scenario selection and statistical uncertainty are not detailed. e20 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table IV. The tested eight-step hierarchy outperforms branch removal and the shorter or longer tested prediction horizons. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the Slow and Fast checkmarks to separate component removal from horizon changes. The first two rows remove a branch; the last three retain both branches and vary K, the slow prediction horizon. The highlighted default is K=8, with NC 99.6, DAC 99.0 and PDMS 93.8. This is not an independent sweep of update frequency: the table has no N column. Elsewhere the default sets N=K=8. Because the aligned latent must exist throughout the reuse interval, reproducing the K=4 row requires knowing whether N changed or how missing offsets were handled; that detail is absent. e04e08e11e13
What it supports. Using the table values, the full default gains 0.8 PDMS points over fast-only and 3.6 over slow-only. Longer prediction is not monotonically better: K=12 reaches 93.2, below K=8. These observations support the chosen configuration among the reported alternatives without establishing a universal optimal horizon.
Where the evidence stops. The slow-only action interface and nondefault update schedules are unspecified. Section IV.D(a) instead uses a 93.3 full-model PDMS and a 3.1-point slow-only drop. The table–prose discrepancy prevents treating either delta as an independently verified correction.
Figure 4. Future optical-flow supervision yields the strongest reported slow-target driving scores. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the same bar color across groups: red measures No At-fault Collision, purple Drivable Area Compliance, and gray the aggregate PDMS. The horizontal categories identify what the slow branch learns to predict, not what the fast branch outputs. Optical flow leads all three plotted measures, while RGB has the lowest aggregate score. Notice the axis starts at 90 and the figure explicitly labels its focused 90–100 range; bar-height ratios therefore exaggerate relative differences. The authors say other configurations are unchanged. Their neighboring representation images illustrate prediction targets, but do not provide a quantitative flow-error measure. e04e08e15e18
What it supports. The PDMS ordering is optical flow 93.8, depth 93.0, BEV 92.8 and RGB 92.2. This supports learning the slow interface from displacement supervision in the tested system. It does not imply that RGB supervision is generally harmful: the fast branch's auxiliary-target comparison favors RGB.
Where the evidence stops. There are no error bars or repeated-run statistics. The chart measures downstream driving scores, not optical-flow accuracy or uncertainty calibration; it cannot by itself prove that better motion prediction causes the policy improvement.
Table VIII. RGB is the preferred auxiliary target for the fast policy, although the slow predictor favors flow. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. All rows retain action supervision; only the next-frame auxiliary target changes. Start with None to see the action-only training reference, then compare flow, depth and RGB. Here None means no fast visual auxiliary target, not the removal of the slow model in Table IV. That distinction matters because the two controls have different reported scores and test different mechanisms. RGB reaches the highest NC, DAC and PDMS in this table. Section III.C explains how it enters training: the expert predicts next-image tokens conditioned on current context, the aligned DAL and action tokens. Those image tokens need not be decoded during driving. e06e13e15e18
What it supports. Next-frame RGB increases PDMS from 93.1 to 93.8 relative to action-only training, a reader-computed 0.7-point gain. It also exceeds next-depth by 0.2 points and next-flow by 0.3. Together with Figure 4, the results favor assigning different predictive targets to the two temporal roles.
Where the evidence stops. These small point differences have no reported uncertainty estimates. The ablation tests auxiliary supervision, not whether generating images at inference improves control. A crossed slow-target/fast-target experiment would better test whether the two choices interact.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors acknowledge computational/memory overhead and missing explicit multimodal-future or uncertainty modeling. Their Y-intersection example shows insufficient yielding and a potential collision in generated observations conditioned on a predicted trajectory, not an observed physical crash. e19e21
Tables and prose repeatedly disagree: v1 DAC is 99.0 in Table I versus 98.4 in prose; Tables IV/VII list NC 99.6, DAC 99.0 and PDMS 93.8 where prose uses 99.5, 98.4 and 93.3. Table values are preserved, not certified as corrected results. e09e11e13e17
7.2 Questions for discussion
- Does correctly timed DAL information matter beyond generic extra representation capacity?
- How should latent refresh respond to unexpected motion or an uncertain future?
- Would the target asymmetry persist under matched compute and fully specified decoding?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs nuPlan/NAVSIM preprocessing, checkpoints, flow targets, action vocabulary/decoder, context length, timestep duration, FiLM layers, loss weights and evaluation splits. These are not fully specified; tokenizer freezing is also unreported. Table V names V-JEPA; Figure 2 and reference [55] name VL-JEPA. Its Qwen2.5-VL row cites a Qwen3-VL reference. Checkpoint identity needs clarification. e04e05e06e08e14
Reader-proposed priorities are a compute-matched DAL alignment intervention and a factorial slow/fast target comparison. Both should use fixed scenes, documented decoding, multiple seeds and uncertainty estimates before interpreting marginal score differences. e13e15e16e18
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does temporally aligned DAL guidance affect driving?
Reader-proposed check: with fixed weights, observations, decoding and N=K=8, compare the normal cache against within-scene permutations of its future offsets and a cross-scene latent permutation. Keep both branches running so compute is matched. Measure paired NAVSIM score changes, collision-related metrics and action changes; stratify by time since the slow update. Repeat with several seeds and report paired confidence intervals. If correctly aligned latents do not consistently outperform temporal permutations, the claim that future timing supplies useful guidance is weakened. A drop under every corruption would establish dependence on the latent, but would not alone distinguish correct motion reasoning from sensitivity to unfamiliar inputs. e04e06e08e13e16
Check 2: Does the slow-flow/fast-RGB pairing have a reproducible advantage?
Reader-proposed check: train a crossed comparison with slow targets {flow, RGB} and fast auxiliary targets {RGB, none}. Hold checkpoints, training examples, action supervision, update interval, horizon, parameter budgets, decoding and evaluation scenes fixed; document any target-specific decoder costs and match training compute where feasible. Resolve loss weights and target construction first. Use multiple seeds and compare paired PDMS changes across the four cells. If flow helps equally with and without the fast RGB loss, the effects may be additive; a reliably different gain would support an interaction. Failure to recover the target ordering would weaken the claimed division of predictive roles. e04e06e07e08e15e18
8.3 Reading coverage
Visual audit: All 14 PDF pages were rendered and actually viewed, including the title/byline, all method equations, training settings, Figures 1–7, Tables I–VIII, conclusions, references and biographies. The six final original crops were each viewed after extraction; axes, legends and relevant table footnotes are retained. Pages 3, 5, 6 and 7 support the architecture reading; pages 8–11 support numerical cross-checks and discrepancies; pages 12 and 14 support limitations and backbone-reference checks. Figure 2 was checked against Eqs. (11), (15), (17)–(18) and (20)–(24): its action indexing, schematic FiLM placement and backbone naming differences are disclosed. No separate supplement, video or code was inspected; the failure illustration is a static figure containing generated future frames.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; I. Introduction (pp. 1–2)
- II. Related Work (pp. 2–3)
- III.A. Problem Formulation and Framework Overview (pp. 3–4)
- III.B. Slow World Model (pp. 4–5)
- III.C. Fast World Model (pp. 5–6)
- III.D. Training Objective (p. 7)
- IV.A. Implementation Details (pp. 7–8)
- IV.B. Quantitative Comparison (pp. 8–9)
- IV.C. Qualitative Comparison (pp. 9–10)
- IV.D. Ablation Study, including probing, failure analysis and user study (pp. 10–12)
- V. Conclusion; VI. Acknowledgment (p. 12)
- References and author biographies (pp. 12–14)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The supplied artifact is arXiv:2609.03572v1 [cs.CV], dated 3 September 2026. Its title and eight-author byline match the catalog. No other edition was supplied for comparison.
- Text extraction does not reconstruct figure images; this limitation was addressed by visually inspecting all 14 supplied PDF pages and every final crop.
- Separate supplemental material availability has not been fully verified.
- No separate supplement was supplied. Code and external cited works were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, complete byline, affiliation footnote and arXiv margin stamp
Title and all eight authors match the supplied identity; stamp reads arXiv:2609.03572v1 [cs.CV], 3 Sep 2026. Title-page affiliations include Beihang University, Shanghai Jiao Tong University, Dim12 AI, GigaAI, National University of Singapore and TeleAI.
Go to primary source ↓e02PDF pp. 1–3, Abstract, I. Introduction, Figure 1 and II. Related Work
The motivation distinguishes extended scene anticipation from immediate observation-grounded action generation and contrasts the slow–fast hierarchy with action-only and common-rollout models.
Go to primary source ↓e03PDF pp. 3–4, III.A, Eqs. (1)–(4)
Context encodes observations and prior actions. Slow predictions update every N steps over K offsets; the fast model conditions on history and the next-state-aligned prediction.
Go to primary source ↓e04PDF pp. 4–5, III.B, Eqs. (5)–(13)
Future consecutive-frame optical flows supervise parallel multi-offset predictions; a lightweight decoder reads DALs. The aligned latent is passed to the fast model. Weighted robust flow loss can use validity masks, but estimator, weights and penalty settings are not concretely supplied.
Go to primary source ↓e05PDF p. 4, Figure 2 and caption; pp. 5–6, Eqs. (15), (20)–(24)
The diagram labels the slow backbone VL-JEPA and the fast backbone Emu3 8B, with a DAL-to-FiLM arrow and different update schedules. It labels action input A_t/output A_{t+1}, whereas the detailed fast equations use previous actions as input and A_t as output.
Go to primary source ↓e06PDF pp. 5–6, III.C, Eqs. (14)–(26)
Emu3 visual tokens and contextual hidden states feed FiLM modulation and an AR expert. Action-token likelihood is followed by action-conditioned next-frame-token likelihood. Next-frame generation is explicitly optional at inference; exact action encoding and FiLM layer selection are unspecified.
Go to primary source ↓e07PDF p. 7, III.D, Eqs. (27)–(28) and following paragraph
The generic latent-distance formulation supplements action loss with slow and fast predictive terms. Predicted slow latents, shared over N fast steps, are used during training and cached at inference; numerical predictive-loss weights and the precise connection to token losses are not given.
Go to primary source ↓e08PDF pp. 7–8, IV.A, Datasets and evaluation; Training details
nuPlan pretraining and NAVSIM fine-tuning are described; NAVSIM v1/v2 metric formulas and named metrics are provided. Training uses N=K=8, 10K full-objective pretraining steps, 6K action-loss fine-tuning steps, 256×144 input, eight A100s, batch 48, AdamW with initial learning rate 2×10⁻⁴ and cosine schedule. Evaluation splits/sample counts are not specified.
Go to primary source ↓e09PDF p. 7, Table I, Drive-HWM, DriveVLA-W0‡ and Human rows, Sensors and †/‡ footnotes
Drive-HWM uses one front camera and reports NC 99.6, DAC 99.0, TTC 98.5, comfort 100.0, EP 89.0, PDMS 93.8. DriveVLA-W0‡ reports PDMS 93.0; Human reports 94.8. The ‡ footnote specifies best-of-N with N=6 for marked AR variants; Drive-HWM has no such marker.
Go to primary source ↓e10PDF p. 8, Table II, Drive-HWM, DriveVLA-W0 and DiffusionDrive rows
Drive-HWM reports EPDMS 86.4 versus 86.1 and 84.5 respectively. Its component values are NC 98.9, DAC 99.5, DDC 99.4, TLC 99.9, EP 88.5, TTC 98.7, LK 96.5, HC 98.3 and EC 87.5.
Go to primary source ↓e11PDF p. 8, IV.B; p. 9, Computational Efficiency; p. 10, IV.D(a); p. 11, IV.D(e), compared with Tables I–IV and VII
Prose states v1 PDMS 93.3 and DAC 98.4, v2 EPDMS 86.2 and EP 88.1, conflicting with tables. Hierarchy and FiLM prose use 99.5/98.4/93.3 for NC/DAC/PDMS instead of tabulated 99.6/99.0/93.8. The paper does not reconcile these differences.
Go to primary source ↓e12PDF p. 9, Table III and Computational Efficiency paragraph
Single-H200 batch-one latency: Drive-HWM slow 25.6 ms, fast 81.6 ms, peak 107.2 ms, average 84.8 ms with N=8; DriveVLA-W0 117.8 ms; fast-only 81.6 ms. Average is T_f+T_s/N; Table III lists Drive-HWM PDMS 93.8, although prose says 93.3.
Go to primary source ↓e13PDF p. 9, Table IV and footnote; p. 10, IV.D(a)
Fast-only, slow-only and full K=4/8/12 PDMS values are 93.0, 90.2, 93.0/93.8/93.2. The paper does not explain how slow-only produces actions or how the update interval changes with horizon.
Go to primary source ↓e14PDF p. 9, Table V; p. 10, IV.D(b); p. 14, references [55] and [57]
The slow-backbone table holds Emu3 fixed and labels the best row V-JEPA; the fast-backbone table holds V-JEPA fixed and labels the best row Emu3. Reference [55] is titled VL-JEPA. The Qwen2.5-VL row points to [57], titled Qwen3-VL technical report.
Go to primary source ↓e15PDF p. 10, IV.D(c), Figure 4 including axes/legend, and Figure 5
Figure 4 reports slow-target NC/DAC/PDMS: BEV 99.0/98.0/92.8, depth 99.3/98.2/93.0, RGB 98.3/97.3/92.2, optical flow 99.6/99.0/93.8. Other configurations are stated to remain unchanged. The score axis spans 90–100; Figure 5 provides qualitative representations without a calibrated flow-error evaluation.
Go to primary source ↓e16PDF p. 11, Table VI and IV.D(d)
Frozen slow-model latents are evaluated using lightweight linear probes for Future Ego Motion and Motion Consistency. Flow scores 83.7/76.1 and RGB 76.8/69.4; probe targets, splits and detailed accuracy definitions are not specified.
Go to primary source ↓e17PDF p. 11, Table VII and IV.D(e)
Conditioning PDMS: concatenation 92.5, cross-attention 93.0, gated cross-attention 93.1, AdaLN 93.3, FiLM 93.8. The table gives FiLM NC/DAC 99.6/99.0; surrounding prose gives conflicting values.
Go to primary source ↓e18PDF p. 11, Table VIII, caption and IV.D(f)
All variants use action supervision. No auxiliary target, next-flow, next-depth and next-RGB yield PDMS 93.1, 93.5, 93.6 and 93.8. RGB gives NC/DAC 99.6/99.0. The authors attribute the preference to richer local semantic and appearance information.
Go to primary source ↓e19PDF pp. 11–12, IV.D(g), Figure 6 and caption
An unsignalized Y-intersection example compares actual future observations and generated observations conditioned on ground-truth or predicted trajectories. The latter fails to yield sufficiently, with an approaching vehicle entering the path and a potential collision at V_{t+4}.
Go to primary source ↓e20PDF p. 12, IV.D(h), Figure 7 and caption
Twenty raters compare 50 scenario pairs on a 1–5 Likert scale. Preferences are 68% Drive-HWM, 21% DriveVLA-W0, 11% no preference; criteria are safety, comfort, human-likeness and route correctness. Selection/randomization details and statistical uncertainty are not reported.
Go to primary source ↓e21PDF p. 12, V. Conclusion, final paragraph
The authors identify additional computation and memory, propose compression/asynchronous inference as future work, and acknowledge no explicit multimodal-future or predictive-uncertainty modeling.
Go to primary source ↓8.5 Primary sources
Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving ↗
PDF · 11,411 extracted words
Source fingerprint
ff512e1c019e01c4dbff651ef0a06903aca2c16b8b123ed3c44acf0caeceaef9