DriveLaW: Unifying Planning and Video Generation in a Latent Driving World
1. Paper overview
In one sentence: DriveLaW conditions a separate diffusion planner on early video-model features, gaining planning performance while leaving the exact link between visual fidelity and action reliability unmeasured. e02e03e06e10e11e15
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that world models used only as simulators, auxiliary supervisors or parallel video predictors leave their internal driving representations underused. Their proposed remedy is a direct feature pathway from video generation into trajectory planning. The central question is whether a good generative representation also makes a better planning condition. e02e03 |
| Core mechanism | |
| A key reported result | NAVSIM Navtest trajectory planning: 89.1 PDMS; components 99.0/97.1/96.7/100/81.3. PDMS ↑; NC/DAC/TTC/comfort/EP ↑. Image-based DriveLaW; 12k Navtest samples; benchmark closed-loop metrics, without reported RL post-training or learned scorers. Epona 86.2; DriveVLA-W0 87.2; PWM 88.1; WoTE 88.3. Best PDMS among listed methods, but not best progress: DiffusionDrive has EP 82.2. Sensors differ and matched training budgets are not established. e09e11 |
| Reading caution |
Core contributions
- Source description
Selective noise reinjection addresses visual degradation, while progressive video training separates long-duration motion learning from high-resolution refinement before planning fine-tuning. These mechanisms have different evidence: reinjection is illustrated qualitatively, whereas training stages have a quantitative ablation. e05e07e16
Figure 1. The planning connection is the downward feature pathway between two transformers. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the historical frames in the upper left and follow them through the temporal-spatial VAE into the green Video DiT blocks. The text branch supplies a structured description containing ego-motion information. Then follow the black arrows downward: internal video features condition the blue Action DiT blocks. The lower branch also has its own noisy-action input and encoded command/ego status. Its right-hand output is a trajectory, while the upper output passes through a video decoder. Section 3.4 resolves the caption’s broad reference to denoised latents: features are cached from each transformer block during the first video denoising step and reused throughout action denoising. e03e05e06e08
What it supports. The drawing supports a chained architecture with separate video and action transformers, rather than a planner operating on a completed future-image sequence. This makes learned video representations directly available to planning at inference. The reported implementation uses a 2B video model and a 133M action planner.
Where the evidence stops. The schematic does not specify gradient routing or show a stop-gradient operator. Section 4.1 explicitly updates both DiTs during action fine-tuning. Its reinjection loops are schematic; the pixel-derived mask and latent perturbation must be read from Eqs. (5)–(6), not inferred from block placement.
2. Motivation
2.1 The problem and the proposed response
The authors argue that world models used only as simulators, auxiliary supervisors or parallel video predictors leave their internal driving representations underused. Their proposed remedy is a direct feature pathway from video generation into trajectory planning. The central question is whether a good generative representation also makes a better planning condition. e02e03
2.2 What this reading follows
A driving video generator learns features useful for predicting how a scene evolves. DriveLaW asks whether those internal features can also guide the ego vehicle’s trajectory. Its answer is a directed connection between two transformers: the video model supplies features and the action model uses them throughout trajectory denoising. The important timing detail is that planning takes its condition from the first video denoising step. Read the architecture alongside the timing ablation, then separate the video-quality results from the planning results. Together they support useful representation transfer, while leaving causal claims about physical understanding and guaranteed video–trajectory consistency unresolved. e02e03e06e10e11e15
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | IDM |
| Quadrant | Q4 · Dual-system × IDM |
3.1 Evidence-based assessment
Insufficient evidence to decide
The WAM and Dual-system labels fit the explicit video and action transformers and inference-time feature transfer. IDM is less secure: the supplied equations describe a video-feature-conditioned diffusion policy, without an explicit inverse-dynamics map from a current/future state pair to an action. Shared latents and joint fine-tuning do not make this a single-model architecture or establish IDM. The recorded Q4 classification is preserved pending clarification. e03e06e08
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the features, not only the two outputs
The word unified can hide the most important architectural question: what information reaches the planner? In DriveLaW, the answer is visible in the downward arrows of Figure 1. The video model processes encoded observations and text, and internal features from its transformer blocks become conditions for the action model. The planner also has its own action-noise stream and driving context. Section 3.4 makes this connection operational by caching the first-step video features and reusing them at every action step. This supports a two-component system with direct feature transfer at inference. As a reader interpretation, the mechanism is best understood as generative representation reuse. The paper’s equations do not establish an explicit inverse-dynamics mapping from a specified current/future state pair, even though the catalog records IDM. e03e06
5.2 Separate video learning from planning fine-tuning
The training curriculum first gives the video model temporal range and then spatial detail. Stage one uses 740×352×121 clips; stage two uses 1280×704×25 clips. The authors’ rationale is to learn long driving motion before refining markings, vehicles and textures. Table 7 supports the usefulness of both stages for generation: removing either raises FVD, especially removal of the first. Stage three then uses historical frames and future trajectory supervision to train the planning connection. The implementation explicitly updates both transformers, rather than freezing the video model. This matters when reading the introduction’s claim of avoiding gradient interference: the supplied text does not show the routing or losses needed to verify that assertion. It also leaves the first two stages’ stated 30k-iteration allocation ambiguous. e02e07e08e16
5.3 Ask what each improvement actually tests
The results answer several different questions. Table 1 concerns generated-video quality; Table 2 concerns NAVSIM trajectory scores; Table 4 asks whether more driving-video pretraining transfers to planning. None of these alone tests whether a sampled video and a sampled trajectory agree geometrically. Table 6 is especially informative because later denoising features perform worse, despite being further along the video-generation process. My interpretation is that rendering progress and planning usefulness should be evaluated separately. The paper attributes the decline to redundant visual information, but the experiment does not distinguish that from a change in the feature distribution expected by the planner. Even the aggregate safety interpretation needs care: Table 3 improves average collision percentage while worsening its one-second value. e10e11e12e13e15
5.4 Training and inference
During training
The 2B Video DiT starts from LTX-Video weights; the planner has 133M parameters. Video pretraining uses 8 Hz nuScenes/nuPlan clips, first 740×352×121 and then 1280×704×25 (width×height×frames). Reported settings are batch 4, learning rate 1e-5, weight decay 5e-2 and token-wise uniform noise interpolation. The stated 30k iterations for the first two stages is ambiguous between per-stage and combined duration. e07e08
Action fine-tuning updates both Video DiT and Planning DiT for 44k steps, with batch 192, learning rate 3e-5 and weight decay 1e-5. It uses four past camera frames and 4-second trajectory supervision at 2 Hz. Freezing the video backbone would therefore depart from the reported recipe; VAE and text-encoder freezing are not specified. e08
During inference
The paper specifies 30 sampling steps for video generation and five for trajectory planning. Cached first-step video features condition all action steps, so this does not establish that planning waits for all 30 video steps. No sampling solver, measured latency or physical actuator-control procedure is provided. e06e08
5.5 Implementation flow
- Encode and model driving context
A causal spatiotemporal VAE compresses video into 128-channel latents; the paper reports 1:192 compression. Video DiT blocks process these tokens with self-attention and text conditioning. Recent ego kinematics become structured language prompts. Query/key RMSNorm and fractional-coordinate rotary embeddings support attention stability and spatiotemporal consistency. e03e04
- Regenerate selected visual detail
Predict a clean latent, temporarily decode it, and find large grayscale Laplacian responses. Threshold at beta times the response standard deviation, then resize the binary mask with nearest-neighbor interpolation. Add Gaussian noise only within the latent mask before the next denoising update. The noise strength is manually tuned. e05
- Pass video features to the planner
Cache features from every Video DiT block during its first denoising step. The Action DiT repeatedly uses these features alongside encoded action noise, ego status and command. This is an inference-time conditioning pathway, beyond auxiliary future-image supervision. Separate output branches produce video and trajectory. e03e06
- Refine the generated video
A time-conditioned VAE decoder maps a late noisy latent into pixels for final detail refinement. This video-output operation is distinct from extracting intermediate planning features; the paper does not require a completed decoded video as the planner input. e04e06
6. Experiments & results
DriveLaW uses a video diffusion model as the perception backbone for a separate trajectory diffusion model. Features cached during the first video denoising step condition every action denoising step, connecting video pretraining to planning. The reported gains cover nuScenes video generation and NAVSIM planning, but do not prove physical driving safety or guaranteed agreement between generated scenes and trajectories.
6.1 Read the original evidence
Table 1. The strongest prior comparator differs between the two generation metrics. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across each metric row separately; both downward arrows mean lower is better. The final DriveLaW column is 4.6 for FID and 81.3 for FVD. To understand the abstract’s relative improvements, find the best prior value in each row rather than using the same comparator twice. Vista has the lowest prior FID, 6.9, whereas Epona has the lowest prior FVD, 82.8. Epona’s FID is 7.5 and Vista’s FVD is 89.4. Section 4.1 places this comparison on nuScenes validation, and the table caption scopes the claim to single-view video generation. e09e10e18
What it supports. DriveLaW improves on the strongest listed prior FID by about 33.3% and the strongest prior FVD by about 1.8%, using different baseline methods. The table supports better reported generation scores across both measures. It supplies no numerical bridge from either score to improved trajectory execution or collision avoidance.
Where the evidence stops. These are reported point estimates without uncertainty. The supplied experiment description does not fully specify the evaluation clip count, horizon and feature-extraction implementation, so a reproduction must recover those choices before treating small FVD differences as directly comparable.
Table 2. DriveLaW leads the listed aggregate planning scores, with a different ordering on progress. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the Image and Lidar columns to see which sensor inputs each method uses. DriveLaW has an image checkmark only. Move right to NC (no-at-fault collisions), DAC (drivable-area compliance), TTC (time-to-collision), comfort and EP (ego progress). Section 4.1 describes PDMS as combining collision/compliance penalties with weighted progress, TTC and comfort measures; it is not simply the mean of the displayed columns. Compare the final shaded column within and across the two method groups. The retained caption supplies the dagger key: DriveVLA-W0 uses the same flow-matching objective, which does not imply all other training conditions are identical. e09e11e12
What it supports. DriveLaW’s 89.1 PDMS exceeds Epona’s 86.2, DriveVLA-W0’s 87.2 and PWM’s 88.1. Its NC/DAC/TTC scores are 99.0/97.1/96.7, with comfort 100. The component view also reveals a tradeoff: its EP is 81.3, below DiffusionDrive’s 82.2, despite the higher aggregate score.
Where the evidence stops. These are NAVSIM benchmark scores described as closed-loop metrics, not evidence of physical deployment. Sensor inputs differ and training budgets are not matched here. The separate nuScenes open-loop table even shows worse one-second collision percentage despite a better average.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIM Navtest trajectory planning Image-based DriveLaW; 12k Navtest samples; benchmark closed-loop metrics, without reported RL post-training or learned scorers. | 89.1 PDMS; components 99.0/97.1/96.7/100/81.3. PDMS ↑; NC/DAC/TTC/comfort/EP ↑ | Epona 86.2; DriveVLA-W0 87.2; PWM 88.1; WoTE 88.3. Best PDMS among listed methods, but not best progress: DiffusionDrive has EP 82.2. Sensors differ and matched training budgets are not established. e09e11 |
| nuScenes validation single-view video generation Reported single-view comparison on validation data. | 4.6 FID; 81.3 FVD. FID ↓; FVD ↓ | Vista has best prior FID 6.9; Epona best prior FVD 82.8. The abstract’s 33.3% FID and 1.8% FVD reductions use different baselines. These are generation metrics, not driving success. e09e10e18 |
| nuScenes validation open-loop trajectory planning Displacement/collision evaluated at 1, 2 and 3 seconds. | 1.15 m; 0.24%. Average L2 (m) ↓; average collision (%) ↓ | Epona: 1.25 m; 0.36%. Average improvement hides one-second collision deterioration: 0.15% versus 0.01%. The table does not support improvement at every horizon. e12 |
| Video-pretraining scale ablation NAVSIM Navtest after planning fine-tuning; 0/76k/3.8M/7.6M video-pretraining samples. | 85.9/87.0/87.8/89.1. PDMS ↑ | Largest versus zero driving-domain pretraining: +3.2 points. Monotonic aggregate gains support transfer from video data; four configurations without uncertainty or matched compute do not establish a general scaling law. e13 |
| Planning representation and feature-timing ablations NAVSIM Navtest; representation conditions and denoising-step conditions evaluated separately. | BEV/VLM/video: 84.1/86.5/89.1; steps 1/5/10: 89.1/86.9/23.2. PDMS ↑ | Video exceeds BEV by 5.0 and VLM by 2.6 points. Early-step features work best in this setup. The late-step collapse does not isolate redundant pixels as its cause. e14e15 |
| Video-training-stage ablation nuScenes video evaluation with one pretraining stage removed. | Complete: 4.6/81.3. FID ↓; FVD ↓ | Without stage one: 5.0/109.3; without stage two: 5.0/93.2. Both stages help reported generation quality; removing stages also changes the training recipe, with no matched-budget control reported. e09e16 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 2. Selected examples show the visual effect of perturbing high-frequency regions during denoising. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read vertically within each labeled column. The top row omits reinjection and the bottom row includes it; the left overview locates the enlarged regions. Column (a) highlights road-letter sharpness, (b) vehicle structure, and (c) a spurious-looking dark shape in the road scene. The method behind these examples is more specific than adding global noise. It temporarily decodes a clean-latent prediction, computes grayscale Laplacian magnitudes, and keeps pixels strictly above beta times their response standard deviation. A nearest-neighbor resize brings that mask into latent space, where it gates a Gaussian perturbation before the next denoising operation. e04e05
What it supports. The selected lower-row patches have sharper markings, more coherent vehicle appearance and fewer visible artifacts than their upper-row counterparts. They illustrate the intended repair mechanism. They do not supply an average improvement across scenes, a quantitative isolated reinjection effect, or evidence that the action planner becomes safer.
Where the evidence stops. Section 3.3 first says the decoder is used only for the final step, then describes temporary decoding to compute this mask. That implementation tension remains unresolved. Beta and the manually tuned reinjection-strength schedule are not numerically specified in the supplied paper.
Table 4. More driving-video pretraining improves the aggregate planning score in the reported sweep. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The first column changes the number of video-pretraining samples used before NAVSIM planning fine-tuning. Read the PDMS column from top to bottom: 85.9, 87.0, 87.8 and 89.1. Then check the components rather than assuming they all rise together. EP reaches 82.2 at 3.8M samples and falls to 81.3 at 7.6M. The zero-size row is labeled scratch, but the accompanying prose describes it as lacking driving-domain pretraining. Because the implementation otherwise starts from LTX-Video weights, the table alone does not establish that this row abandons every form of foundation-model initialization. e08e13
What it supports. The largest reported pretraining corpus improves PDMS by 3.2 points over zero driving-domain pretraining, with intermediate sample counts between those endpoints. This supports the practical value of adapting video representations before training the planner. The aggregate gain does not imply uniform improvement in every driving component.
Where the evidence stops. The four configurations do not establish a universal scaling law. No uncertainty, matched pretraining compute, or full sample-selection protocol is reported here. The scratch label also needs clarification before a reproduction chooses its initialization baseline.
Table 6. Later video-denoising features are substantially worse planning conditions in this reported experiment. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The left column identifies which video denoising step supplies the Action DiT’s conditioning features. Treat these labels as the table’s step indices, not as the continuous time variable in the flow-matching equations. Section 3.4 caches per-block features at the first video step; this is the top-row configuration. The PDMS column falls from 89.1 at step 1 to 86.9 at step 5 and 23.2 at step 10. Inspect the component columns to see that the last result is a broad deterioration, including comfort 0 and EP 15.4, rather than a small isolated change in the aggregate. e06e15
What it supports. The experiment makes feature timing a central design choice: the most denoised video representation is not automatically the most useful planning representation. The first-step condition performs best among the three reported configurations. The severe step-10 collapse warrants investigation of feature-distribution changes and planner adaptation before assigning a single cause.
Where the evidence stops. The authors attribute the decline to redundant video information, but this table does not isolate that mechanism. It does not say whether each planner was retrained for its conditioning distribution. A mismatched planner and a genuinely unsuitable representation remain distinct possible explanations.
7. Analysis & limitations
7.1 What the evidence leaves open
The experiments establish benchmark prediction/planning scores and selected visual examples. They do not directly measure rare-event robustness, physical execution safety, or trajectory–video agreement. No confidence intervals are given in the result tables. e09e10e11e12e17
Decoder usage is internally unclear: Section 3.3 says it is used only for the final step, then describes temporary decoding during reinjection. Likewise, the claimed avoidance of gradient interference is not reconciled with updating both DiTs during planning fine-tuning. Neither a stop-gradient rule nor a detailed gradient-routing algorithm is supplied. e02e04e05e08
7.2 Questions for discussion
- Would later-step features recover planning quality if the action model were retrained on their distribution?
- Does noise reinjection improve planning, or only generated appearance?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the LTX-Video initialization, nuScenes/nuPlan video preprocessing, NAVSIM splits, the staged training settings and per-block feature interface. The main paper omits hardware, software versions, optimizer identity, exact video-evaluation sampling protocol, reinjection beta/noise schedules and detailed latent-to-action fusion implementation. e03e05e06e08e09
Proposed first checks are to retrain planners separately for each denoising-step feature distribution and to compare no, global and masked reinjection under equal sampling budgets. These distinguish representation suitability from distribution shift and test whether localized perturbation supplies measurable benefit. e05e15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Distinguish feature timing from planner mismatch
Reader-proposed check: hold the pretrained video checkpoint, NAVSIM split, planner architecture, training steps and action-sampling budget fixed. Train separate planners on features from video steps 1, 5 and 10, using identical seed sets, then evaluate a train-step by test-step matrix. Report PDMS and all components, with resampled uncertainty. If the step-10 planner recovers when tested on its own feature distribution but fails with the step-1 planner, distribution mismatch explains at least part of Table 6. If independently adapted late-step planners remain worse across seeds, the early-representation advantage is more persuasive. This comparison was not performed in this reading. e06e08e09e15
Check 2: Isolate the value and cost of selective reinjection
Reader-proposed check: use identical held-out nuScenes clips, prompts, checkpoints and random seeds to compare no reinjection, global reinjection and the paper’s Laplacian-mask reinjection. Give the two noisy conditions the same total perturbation energy and keep denoiser-evaluation counts equal, with a compute-matched no-noise refinement control if reinjection requires an extra prediction. Sweep beta and noise strength only on a separate tuning subset; document temporary decoder calls. Measure FID/FVD, blinded ratings of the three Figure 2 failure types, and latency. Consistent gains over both global noise and extra refinement would support the localized mechanism; gains disappearing under these controls would weaken that interpretation. e04e05e08e09
8.3 Reading coverage
Visual audit: The title/author page, all method and training pages, Figures 1–3 and Tables 1–7 were rendered and visually inspected. All six final original crops were viewed at their helper-returned dimensions. Figure 1 arrows were checked against Sections 3.2–3.4; the strict mask threshold and flow target were checked against Eqs. (5)–(9). The Table 2 crop retains its short caption because it contains the dagger key. Pages 6–8 support evaluation details and proposed controls even where their tables are not cropped. References on pages 9–12 were read in the complete text but were not part of the visual pass. No appendix or separate supplement was supplied.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (p. 1)
- 1. Introduction (pp. 1–2)
- 2. Related Work, including 2.1–2.3 (pp. 2–3)
- 3. Method, including 3.1 representations, 3.2 architecture, 3.3 video model, 3.4 planner and 3.5 training (pp. 3–6)
- 4. Experiment, including 4.1 setup, 4.2 main results and 4.3 ablations (pp. 6–8)
- 5. Conclusion (p. 8)
- References (pp. 9–12)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction limitation was addressed by inspecting PDF pages 1–8, all three figures and all seven tables; all six final crops were also visually inspected.
- No appendix is present in the supplied 12-page PDF. Separate supplements were not supplied or inspected.
- The title and complete author list match. This report reads the CVF Open Access accepted-paper edition, identified by the supplied PDF hash. Its watermark states that only the watermark differs from the accepted version. The final proceedings edition was not compared; no separate revision number or revision date was established.
- Code was not inspected, linked resources were not opened, and experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1 (39701), title block and CVF watermark
The title and all twelve authors match the catalog. Affiliations are Huazhong University of Science and Technology and Xiaomi EV. The CVF Open Access notice says the PDF is identical to the accepted version except for its watermark; the final proceedings version is elsewhere.
Go to primary source ↓e02PDF pp. 1–2 (39701–39702), Abstract and Section 1
The authors motivate directly conditioning planning on video-generator representations, contrasting this with simulation, auxiliary future supervision, and parallel generation/planning. They claim a chained design avoids gradient interference and improves consistency.
Go to primary source ↓e03PDF pp. 3–4 (39703–39704), Sections 3.1–3.2, Eqs. (1)–(2), Figure 1 and caption
Video frames and motion-related text condition the Video DiT. Downward arrows pass internal video features to a distinct Action DiT; action noise and encoded command/ego status also enter the action branch. Video and trajectory have separate output decoders.
Go to primary source ↓e04PDF p. 4 (39704), Section 3.3, Spatiotemporal VAE and Video Transformer Architecture, Eqs. (3)–(4)
The causal VAE has 128 latent channels, reported compression 1:192 and pixel-to-token ratio 1:8192. A time-conditioned decoder performs late pixel refinement. Video attention uses query/key RMSNorm and fractional-coordinate rotary embeddings. Recent kinematics become structured text prompts. This passage says the decoder is used only for the final step.
Go to primary source ↓e05PDF p. 5 (39705), Figure 2 and Section 3.3 Noise Reinjection, Eqs. (5)–(6)
A predicted clean latent is temporarily decoded; grayscale Laplacian magnitude is thresholded at beta times its standard deviation, using a strict greater-than test. The binary mask is resized with nearest-neighbor interpolation and gates Gaussian latent perturbations with manually tuned strength. Figure 2 compares blur, structural inconsistency and artifacts, without a quantitative reinjection ablation.
Go to primary source ↓e06PDF p. 5 (39705), Section 3.4, Eqs. (7)–(9)
The Action DiT uses encoded noised actions, ego status and high-level commands, conditioned on per-block features cached during the first video denoising step. Those features are reused at every action flow-matching step. The printed interpolation is a_t=(1-t)a_0+t epsilon and the target is a_0-epsilon.
Go to primary source ↓e07PDF pp. 5–6 (39705–39706), Section 3.5
Video training first uses 740×352×121 clips, then 1280×704×25 clips, prioritizing long motion before fine spatial detail. The third stage trains trajectory planning conditioned on the learned video features.
Go to primary source ↓e08PDF p. 6 (39706), Section 4.1 Implementation Details
The video DiT has 2B parameters initialized from LTX-Video, and the planner has 133M. Video training uses 8 Hz nuScenes/nuPlan frames, batch 4, learning rate 1e-5 and weight decay 5e-2; the first two stages are described as trained for 30k iterations without saying whether that is per stage or combined. Action fine-tuning updates both DiTs using four past frames and 4-second trajectories at 2 Hz, batch 192, 44k steps, learning rate 3e-5 and weight decay 1e-5. Inference uses 30 video and 5 trajectory sampling steps.
Go to primary source ↓e09PDF p. 6 (39706), Section 4.1 Dataset and Metrics; Section 4.2 motion-planning results
Video generation is evaluated on nuScenes validation with FID/FVD. NAVSIM has 103k Navtrain and 12k Navtest samples; planning uses 2 Hz camera data. PDMS combines NC and DAC penalties with weighted EP, TTC and comfort. The authors report no RL post-training or learned-scorer post-processing for DriveLaW.
Go to primary source ↓e10PDF p. 7 (39707), Table 1, FID and FVD rows, all methods
On nuScenes validation, DriveLaW reports FID 4.6 and FVD 81.3. Vista reports 6.9/89.4 and Epona 7.5/82.8. The lowest prior FID and FVD belong to different baselines.
Go to primary source ↓e11PDF p. 7 (39707), Table 2 and caption, DriveLaW and comparison rows
DriveLaW uses images without LiDAR and reports NC 99.0, DAC 97.1, TTC 96.7, comfort 100, EP 81.3 and PDMS 89.1 on Navtest. PDMS is 86.2 for Epona, 87.2 for DriveVLA-W0, 88.1 for PWM and DiffusionDrive, and 88.3 for WoTE. DiffusionDrive has EP 82.2 and uses LiDAR. The dagger denotes the same flow-matching objective.
Go to primary source ↓e12PDF p. 7 (39707), Table 3, all time horizons and average columns; PDF p. 6, Section 4.2 Additional performance
Open-loop nuScenes validation L2 errors at 1/2/3 seconds and average are 0.44/1.10/1.91/1.15 m for DriveLaW versus 0.61/1.17/1.98/1.25 m for Epona. Collision percentages are 0.15/0.10/0.48/0.24 versus 0.01/0.22/0.85/0.36, so the one-second collision result is worse despite the better average.
Go to primary source ↓e13PDF p. 7 (39707), Table 4 and Section 4.3 scaling paragraph
For video pretraining sizes 0 (scratch), 76k, 3.8M and 7.6M, Navtest PDMS is 85.9, 87.0, 87.8 and 89.1. EP is 80.8, 80.8, 82.2 and 81.3. The prose describes the zero-size comparison as without driving-domain pretraining and claims a scaling law.
Go to primary source ↓e14PDF p. 8 (39708), Table 5 and representation-ablation paragraph
Under the same diffusion-based planner, BEV features, VLM hidden states and video latents produce Navtest PDMS 84.1, 86.5 and 89.1 respectively. The paper does not give matched representation-backbone compute or detailed training controls in this comparison.
Go to primary source ↓e15PDF p. 8 (39708), Table 6 and denoising-step ablation paragraph
Video denoising steps labeled t=1, t=5 and t=10 produce Navtest PDMS 89.1, 86.9 and 23.2. At t=10 comfort is 0 and EP is 15.4. The authors attribute worse late-step planning to redundant video information; the table does not isolate that explanation or state whether planners were retrained for each feature distribution.
Go to primary source ↓e16PDF p. 8 (39708), Table 7 and training-strategy discussion
Removing the first video-training stage gives FID/FVD 5.0/109.3; removing the second gives 5.0/93.2; the complete scheme gives 4.6/81.3. No uncertainty or matched total optimization budget is reported in the table.
Go to primary source ↓e17PDF p. 8 (39708), Figure 3 and caption; PDF p. 7, qualitative-results continuation
Selected nuScenes frames compare Epona and DriveLaW with highlighted vehicles and pedestrians. The authors interpret better-looking examples as evidence of visual quality, subject preservation and semantic understanding; this is not a measured detection or planning-consistency score.
Go to primary source ↓e18PDF p. 1 (39701), Abstract; PDF p. 7 (39707), Table 1
The abstract reports 33.3% FID and 1.8% FVD improvements. Table 1 supports these rounded relative reductions using Vista for FID (6.9 to 4.6) and Epona for FVD (82.8 to 81.3).
Go to primary source ↓8.5 Primary sources
DriveLaW: Unifying Planning and Video Generation in a Latent Driving World ↗
PDF · 8,571 extracted words
Source fingerprint
74c3b1f2a64283620be6b01769fb036d2cb8e135f83d9bd5b2b8d2167c351f09