DriveVA: Video Action Models are Zero-Shot Drivers
1. Paper overview
In one sentence: DriveVA couples future video and trajectory generation in one transformer, improving reported planning and transfer while leaving the model vulnerable to mutually consistent but incorrect futures. e-identitye-architecturee-jointe-navsime-transfere-dpvoe-failuree-closed-loop
| At a glance | What to know |
|---|---|
| Research problem | Source description A driving model should transfer across visual domains while keeping planned motion compatible with its imagined scene evolution. The authors target both weaknesses by adapting video priors and supervising future observations together with actions. e-problem |
| Core mechanism | Source description One diffusion transformer jointly generates future video latents and trajectory tokens, with mutual target attention during denoising. e-architecturee-joint |
| A key reported result | Zero-shot transfer to nuScenes: 0.84 m; 0.06% Average L2 (m) ↓; collision rate (%) ↓. NAVSIM-trained models, no target fine-tuning; 150-scene validation split; Table 2 reports 1-, 2- and 3-second values and author-labelled Avg summaries. PWM: 3.99 m, 0.36%; DriveVLA-W0: 1.43 m, 0.77%. The reported PWM reductions are 78.9% and 83.3% relative, not percentage-point reductions, calculated from the printed Avg entries. PWM's displayed collision values 0.12%, 0.15% and 0.86% have an arithmetic mean of about 0.38%, whereas the table prints 0.36%. The source does not explain this aggregation difference, so the printed collision average and derived reduction remain qualified author-reported values. This diagnostic recomputation does not replace an experimental result. These trajectory metrics do not establish sustained interactive driving. e-transfere-protocol |
| Reading caution | Source description Figure 8 shows consistently incorrect stopping under cyclist interaction and at an intersection. Video/action agreement is therefore insufficient for choosing the correct future mode. e-failure |
Core contributions
- Source description
One diffusion transformer jointly generates future video latents and trajectory tokens, with mutual target attention during denoising. e-architecturee-joint
- Source description
History-conditioned video continuation supports repeated short predictions. The reported experiments examine transfer, supervision, rollout length, tuning strategy and sampling budget. e-interfacee-main-ablatione-further-ablation
Figure 2. One decoder generates both descriptions of the future, while observed history anchors each continuation. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the left: the video VAE encodes history observations, and the text encoder produces instruction tokens. On the right, distinguish the solid condition tokens from striped noisy future-frame and action tokens. The upward arrows lead through one unified DiT to both outputs; they do not show a video generator followed by an action model. Appendix B specifies mutual attention between the two target modalities. The bottom strip uses solid yellow outlines for existing frames and dashed blue outlines for the next chunk. For actual rolling-horizon operation, Section 4.1 specifies acquiring new observations after executing the action chunk and updating the history buffer. e-architecturee-interfacee-encodinge-flowe-joint
What it supports. The central architectural fact is joint target denoising within one DiT. This supports the catalog's One Model × Joint prediction classification. The future video is part of the generative target during both learning and inference, while the action output consists of ego positions and yaw rather than direct actuator commands.
Where the evidence stops. Figure 2 visually groups language with the other conditions, but Section 4.2 explicitly routes text through cross-attention, not stream concatenation. The crop is schematic: use Eqs. (6)–(8) for the condition/target interface, and do not infer a detailed attention mask from these arrows.
2. Motivation
2.1 The problem and the proposed response
A driving model should transfer across visual domains while keeping planned motion compatible with its imagined scene evolution. The authors target both weaknesses by adapting video priors and supervising future observations together with actions. e-problem
2.2 What this reading follows
A video of a plausible turn is useful for driving only if the planned motion follows it—and if turning is the right decision. DriveVA addresses the first issue by adapting a pretrained video generator to denoise video latents and ego-trajectory tokens together. This reading follows the observed July 2026 v2 through its conditioning interface, training interventions, NAVSIM scores and direct transfer tests. An odometry diagnostic asks whether generated frames imply the predicted motion. The appendix then exposes the remaining gap: the model can imagine stopping and produce a matching trajectory when the demonstrated vehicle proceeds. Its small Dev10 experiment provides preliminary closed-loop evidence, not a comprehensive deployment assessment. e-identitye-architecturee-jointe-navsime-transfere-dpvoe-failuree-closed-loop
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | One Model |
| Prediction paradigm | Joint prediction |
| Quadrant | Q1 · One Model × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded One Model × Joint prediction classification is supported by architecture: a single DiT jointly denoises future video and trajectory tokens, with bidirectional target interaction. Separate encoders do not constitute separate world/action predictors. The IDM-style factorization is explanatory, not a cascaded inverse-dynamics model. e-architecturee-joint
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate the observed past from the generated future
DriveVA begins with a driving context: a history buffer, current ego velocity and a command. The causal video VAE converts images into temporally compressed latents; MLPs embed ego state and the position/yaw actions. History and state remain noise-free, whereas the future video and action tokens are corrupted together during flow-matching training. The model learns the velocity that moves this target block from noise toward data. At inference, no future observations are supplied: the same type of clean context conditions generation from Gaussian noise. Text has its own cross-attention route, despite appearing beside other conditions in the schematic. This separation explains what must be available online and why a generated future cannot be treated as an observation of what actually happened. e-architecturee-encodinge-flowe-jointe-interface
5.2 Understand action grounding without inventing a second model
Appendix B writes the joint distribution as video continuation multiplied by action grounding conditional on that video. The chain rule makes this interpretation valid, but it does not impose a two-stage implementation. DriveVA instead denoises both target modalities in one DiT, allowing their tokens to exchange information. A useful reader interpretation is that each predicted trajectory must share a representational account of the future with the generated scene. The ablations support the importance of the video branch: removing its loss lowers PDMS substantially, and restricting mutual attention gives a smaller loss. They do not fully isolate whether gains arise from pretrained video knowledge, dense additional supervision, or the precise coupling mechanism. Distinguishing these explanations requires matched interventions, especially because removing video loss and removing video targets produce different reported scores. e-jointe-main-ablatione-mask-ablatione-further-ablation
5.3 Keep transfer, consistency and execution as separate tests
The evidence answers three different questions. Table 2 asks whether a NAVSIM-trained planner retains low trajectory error when tested directly on another domain. Table 4 asks whether a generated video implies motion resembling the paired predicted trajectory after similarity alignment. Table 10 asks whether the system completes routes in a small closed-loop diagnostic. Strong performance in one does not logically settle the others. Figure 8 supplies the decisive counterexample: an imagined stop and its matching action sequence can both disagree with the demonstrated future. A reader should therefore treat consistency as a useful diagnostic property, then separately examine interaction correctness, progress and closed-loop behavior. The source's own Dev10 qualification is essential here; it prevents extending a preliminary simulation result to a complete deployment claim. e-transfere-dpvoe-closed-loope-failure
5.4 Training and inference
During training
Training uses 832×480 clips, NVIDIA H20 GPUs and distributed bf16 AdamW, with learning rate 10^-4 and weight decay 0.01. It runs 20k steps at batch 80, then 10k at effective batch 640 through accumulation. A 1k-step linear warm-up starts at 10^-3 of the base rate, followed by a constant rate. e-training
Full backbone fine-tuning performs best. Equation (9) describes flow matching over joint targets, while Section 5.2 describes video flow loss plus trajectory loss; exact modality weighting and the additional trajectory-loss formulation remain unspecified. e-flowe-traininge-further-ablatione-implementation-gaps
During inference
The reported configuration uses two sampling steps. One step scores 13.2 PDMS; two and three both score 90.9. This establishes a sampling-quality tradeoff, but the source supplies no measured end-to-end latency establishing real-time deployment. e-traininge-further-ablation
5.5 Implementation flow
- Encode context
Wan2.2-TI2V-5B supplies a frozen text encoder and a causal video VAE. Text enters through cross-attention. Flattened/projected history latents and MLP-embedded ego state form the clean condition stream. e-architecturee-encodinge-training
- Generate a common future
Future video latents and MLP-embedded action tokens form one noisy target block. A single DiT predicts their conditional flow velocity; default target attention permits information exchange in both directions. e-flowe-joint
- Continue with observations
Inference encodes only observed history, generates both future modalities from noise, and decodes video latents. The formulation then executes the action chunk, acquires observations and slides the history window. Waypoints require a controller; the paper does not fully specify that integration. e-interfacee-encodinge-flowe-implementation-gaps
6. Experiments & results
DriveVA adapts a pretrained video generator to denoise future video and ego-trajectory tokens together. Its strongest evidence combines NAVSIM planning scores, direct transfer to nuScenes and CARLA, and ablations of video supervision. Jointly consistent outputs can still choose the wrong behavior; closed-loop evidence remains preliminary.
6.1 Read the original evidence
Table 1. DriveVA leads the listed NAVSIM comparison, with training-data and sensor qualifications. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Locate the bold Ours row and read the shaded PDMS column first. Then move left to check whether the overall score conceals a weak component: the row reports NC, DAC, TTC, comfort and ego progress separately. The Image and Lidar columns matter because this comparison contains different sensor configurations; Section 5.3 identifies DriveVA's visual input as the front camera. Finally, connect this table to Table 5 rather than treating every DriveVA result as one training setting. The entire Ours metric vector matches Table 5's configuration with CARLA mixing enabled, while removing that mixing yields a separate NAVSIM-only score. e-navsime-protocole-sim-mixe-main-ablatione-closed-loop
What it supports. DriveVA reports 90.9 PDMS against WoTE's 88.3 and PWM's 88.1. Its NC, DAC, TTC, comfort and ego-progress entries are 99.2, 97.5, 98.7, 100 and 83.5. These are the paper's benchmark scores; they do not directly measure success in a sustained interactive deployment.
Where the evidence stops. Table 1 does not label the mixing regime in its row. Table 5 and the simulation-enhanced discussion identify the matching 90.9 configuration as mixed training. Sensor and data differences, plus absent uncertainty, limit causal attribution of the margin.
Table 2. Direct transfer improves the reported short-horizon trajectory metrics on both target datasets. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read this as two dataset blocks, each with L2 and collision sub-blocks. Within each metric, the three horizon columns precede the shaded author-labelled Avg column; compare like horizons and units, and check the aggregation qualification below. The Finetune crosses mean no target-domain fine-tuning, and the caption fixes NAVSIM as the training dataset for all rows. The left block concerns nuScenes validation, whose 150-scene scope is specified in Section 5.1. The right concerns Bench2Drive validation in CARLA. These results must remain separate from simulation-mixed training evaluated back on NAVSIM and from the appendix's explicit closed-loop Dev10 study. The table itself reports neither Driving Score nor route Success Rate. e-transfere-protocole-sim-mixe-closed-loop
What it supports. On nuScenes, the printed Avg comparison is PWM's 3.99 m and 0.36% collision versus DriveVA's 0.84 m and 0.06%. On Bench2Drive, it is 2.80 m and 3.76% versus 1.33 m and 1.79%. Lower L2 errors and lower listed per-horizon collision rates support transfer under this protocol. The exact collision-summary magnitudes retain the unresolved aggregation qualification below.
Where the evidence stops. Zero-shot here means no target-domain fine-tuning after NAVSIM training, not learning driving without driving data. Table 2 has two unresolved collision-aggregation differences: DriveVA's Bench2Drive values 1.38%, 1.97% and 2.65% have an arithmetic mean of 2.00%, versus printed Avg 1.79%; PWM's nuScenes values 0.12%, 0.15% and 0.86% have a mean of about 0.38%, versus printed 0.36%. The source does not explain the aggregation protocol sufficiently to reconcile these entries. These means are diagnostic recomputations, not corrected experimental results. Retain the original table and printed summaries with this uncertainty. Neither the collision percentages nor these summaries are full-route closed-loop success rates.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| NAVSIMv1 planning Navtest; front-camera DriveVA; Table 1's metric vector matches the simulation-mixed configuration in Table 5. | 90.9 PDMS ↑ | WoTE 88.3; PWM and DiffusionDrive 88.1; NAVSIM-only DriveVA 90.5. Best reported score among these rows, but differing sensor inputs and simulation mixing prevent attributing the whole margin solely to architecture. e-navsime-sim-mixe-protocol |
| Zero-shot transfer to nuScenes NAVSIM-trained models, no target fine-tuning; 150-scene validation split; Table 2 reports 1-, 2- and 3-second values and author-labelled Avg summaries. | 0.84 m; 0.06% Average L2 (m) ↓; collision rate (%) ↓ | PWM: 3.99 m, 0.36%; DriveVLA-W0: 1.43 m, 0.77%. The reported PWM reductions are 78.9% and 83.3% relative, not percentage-point reductions, calculated from the printed Avg entries. PWM's displayed collision values 0.12%, 0.15% and 0.86% have an arithmetic mean of about 0.38%, whereas the table prints 0.36%. The source does not explain this aggregation difference, so the printed collision average and derived reduction remain qualified author-reported values. This diagnostic recomputation does not replace an experimental result. These trajectory metrics do not establish sustained interactive driving. e-transfere-protocol |
| Zero-shot real-to-simulation transfer NAVSIM training; direct Bench2Drive validation evaluation without target fine-tuning; Table 2 reports 1–3-second values and author-labelled Avg summaries. | 1.33 m; 1.79% Average L2 (m) ↓; collision rate (%) ↓ | PWM: 2.80 m, 3.76%; DriveVLA-W0: 3.00 m, 2.52%. This transfer table reports trajectory metrics; the separate Dev10 study supplies the explicitly closed-loop evidence. Its DriveVA collision values of 1.38%, 1.97% and 2.65% have an arithmetic mean of 2.00%, while the printed Avg is 1.79%. The source does not explain the aggregation difference. Retain 1.79% as the author-reported summary, with this unresolved qualification; the diagnostic mean is not a replacement experimental result. e-transfere-protocole-closed-loop |
| Video–trajectory consistency DPVO reconstruction, per-trajectory 2D similarity alignment, future 4 seconds. | Predicted-video reconstruction versus predicted trajectory: NAVSIM 0.16; nuScenes 0.14. Average L2 ↓; units not explicitly stated in Table 4 | Ground-truth video reconstruction versus ground-truth trajectory: 0.09 and 0.07. Supports aligned geometric consistency after fitting a similarity transform; it does not establish absolute-scale accuracy or behavioral correctness. e-dpvoe-failure |
| Video supervision and continuation ablation NAVSIM PDM evaluation; Table 5 changes one listed switch relative to its full configuration. | Full 90.9; no video loss 71.4; no continuation 84.6; no CARLA mix 90.5. PDMS ↑ | Table 11 separately reports causal/bidirectional masks at 90.1/90.9 and action-only/joint targets at 47.0/90.9. Video supervision matters strongly here. Removing its loss and removing video prediction targets are distinct interventions; their action-only scores must not be merged. e-main-ablatione-mask-ablation |
| Preliminary closed-loop driving Bench2Drive Dev10, a small diagnostic route split. | 67.47 DS; 70.00 SR (reported scale). Driving Score ↑; Success Rate ↑ | DriveMamba-12L: 66.50/40.00; DriveTransformer: 60.45/30.00. Auxiliary simulated closed-loop evidence only; the authors explicitly caution against treating Dev10 as full Bench2Drive evaluation. e-closed-loop |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. Removing video supervision produces the largest loss among the three listed interventions. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use row 4 as the reference: all three switches are enabled. Compare it separately with row 1 to remove video loss, row 2 to remove CARLA mixing, and row 3 to remove video continuation. The caption defines continuation as video-to-video generation, making this a test of temporal conditioning rather than merely a different display output. Read the individual planning columns alongside PDMS: the no-video-loss setting especially reduces ego progress. Keep this table distinct from Appendix E.2, where an Action Only prediction target scores 47.0. Disabling a supervision term and deleting a generative target need not create the same trained system. e-main-ablatione-sim-mixe-mask-ablatione-implementation-gaps
What it supports. Relative to 90.9, disabling video loss loses 19.5 PDMS points, disabling continuation loses 6.3, and disabling simulation mixing loses 0.4. This supports a strong role for future-video supervision within this implementation. It also establishes that the reported simulation-mixing increment is much smaller than the video-loss ablation gap.
Where the evidence stops. The table does not supply seed variation or complete implementation details for each intervention. Appendix Table 11's 47.0 action-only result is a separate target ablation, not a replacement for the 71.4 no-video-loss row. Their relationship requires a controlled reproduction.
Table 4. Odometry provides an external consistency check after trajectory alignment. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The two numerical columns represent different reference pairs. On the left, run DPVO on ground-truth video and compare its reconstructed motion with the ground-truth trajectory. On the right, run DPVO on generated video and compare its reconstructed motion with DriveVA's predicted trajectory. Appendix D first fits a separate 2D similarity transform to each reconstruction and then averages positional L2 discrepancy over four seconds. Consequently, the right column does not measure predicted trajectory error against the real future. The ground-truth column is a useful reconstruction reference, while the bottom Average row summarizes the two displayed dataset entries rather than establishing a sample-weighted population estimate. e-dpvoe-qualitativee-failure
What it supports. The predicted branch reports 0.16 on NAVSIM and 0.14 on zero-shot nuScenes, compared with ground-truth-branch values of 0.09 and 0.07. These results support correspondence between the two generated modalities after alignment. Together with the qualitative DPVO overlays, they provide a more specific test of motion consistency than visual plausibility alone.
Where the evidence stops. A similarity transform can absorb scale, rotation and translation discrepancies. Table 4 does not state units, sample counts or uncertainty. Low aligned error neither validates absolute geometry nor rules out the consistently wrong stopping behavior shown in Figure 8.
Figure 8. Agreement between imagination and trajectory can preserve a mistaken stop. Original paper, p. 27 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. For each case, scan the generated frames from one to three seconds, then compare the ground-truth row immediately below. In the cyclist case, the generated sequence preserves a stopping interpretation while the ground-truth vehicle bypasses the cyclist. At the intersection, the model again selects a near-stationary future although the demonstrated vehicle proceeds. The right-hand plots use forward and left coordinates, with blue history, green ground-truth future, red predicted future and a black current marker. Their red predictions stay close to the present position while the green trajectories advance. Read this alongside the consistency diagnostic: agreement within a prediction pair can coexist with disagreement against reality. e-failuree-dpvo
What it supports. These cases establish a concrete limitation of the proposed coupling: both modalities can agree on the wrong future mode. The failure is therefore not exhausted by mismatched arrows or visibly degraded frames. As the authors discuss, selecting appropriate interaction behavior remains a separate problem from keeping video and trajectory mutually aligned.
Where the evidence stops. These are representative failures, not a measured frequency or a complete safety analysis. A stopped forecast differs from the demonstrated behavior, but these images alone do not determine the safest counterfactual action in every comparable traffic situation.
7. Analysis & limitations
7.1 What the evidence leaves open
Figure 8 shows consistently incorrect stopping under cyclist interaction and at an intersection. Video/action agreement is therefore insufficient for choosing the correct future mode. e-failure
DPVO alignment absorbs scale, rotation and translation differences. Table 4 supplies neither sample counts nor uncertainty; the main comparison and ablation tables also omit uncertainty. Selected qualitative examples cannot measure failure frequency. e-dpvoe-navsime-transfere-main-ablatione-qualitative
7.2 Questions for discussion
- How much transfer benefit survives equalizing data exposure and training budget?
- Does video–trajectory agreement predict correct interaction decisions once stopping failures and similarity alignment are accounted for?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the Wan backbone, driving clips/commands/ego states, the specified frame timing and optimizer schedule, and separate NAVSIM-only and simulation-mixed checkpoints. GPU count, software versions, action normalization, mixing ratio and controller details remain unresolved. e-encodinge-traininge-protocole-sim-mixe-implementation-gaps
Reader-proposed checks: distinguish zeroing video loss from deleting video targets under identical training conditions; then test whether aligned DPVO agreement predicts correct progress on held-out stopping/interaction cases. Neither check was performed in this reading. e-main-ablatione-mask-ablatione-dpvoe-failure
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Disentangle loss removal from target removal
Reader-proposed, not run: start from one Wan initialization and identical NAVSIM splits, clip sampling, optimizer schedule and sampling budget. Compare the full joint target, the same target stream with video loss set to zero, and removal of future-video targets. Hold simulation mixing fixed and report the resulting compute difference. Add a causal-mask versus bidirectional comparison with all other settings unchanged. Evaluate PDMS and DPVO consistency across repeated seeds. If loss removal and target removal reproduce distinct degradations, the 71.4 and 47.0 reports become interpretable as different interventions; if not, undocumented training or architectural changes must be investigated. e-traininge-main-ablatione-mask-ablatione-dpvoe-implementation-gaps
Check 2: Test whether consistency predicts correct interaction
Reader-proposed, not run: select held-out cyclist and intersection cases before generating predictions, and include matched ordinary-progress cases. Compare the full model and the controlled reduced-coupling variant using identical observations and a fixed sampling budget. Record the paper's per-clip similarity-aligned DPVO error, reference-trajectory error, progress and collision outcomes; separately report odometry failures. Where feasible, add an independently calibrated metric-motion check to avoid fitting scale to each predicted reference. The falsifiable observation is whether low aligned consistency error still occurs with incorrect stops or poor route progress. Repeat on Dev10 only as a diagnostic, reporting route-level outcomes without claiming a full Bench2Drive reproduction. e-failuree-dpvoe-mask-ablatione-closed-loope-training
8.3 Reading coverage
Visual audit: Visually inspected the title/author/revision block; Figure 2 and the equations/conditioning description; all Tables 1–11; Figures 3–12; Appendix B's target attention interface; Appendix D's alignment protocol; and the Dev10 and failure analyses. All six final original crops were inspected after extraction. Claim-supporting method, training, numerical and proposed-check pages are included even when uncropped. Figure 1 and reference-page layouts were not visually inspected; their supplied text was read. All seven text chunks, including embedded appendices, were read completely. Separate supplemental material availability remains unverified.
PDF pages inspected for this edition: 1, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 22, 23, 24, 25, 26, 27, 28, 29, 30. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2: Introduction and Related Work
- Section 3: Preliminary flow matching
- Sections 4.1–4.4: formulation, preprocessing, joint generation and objective
- Sections 5.1–5.7: datasets, training, comparisons, transfer, consistency, qualitative results and ablations
- Section 6; Acknowledgements; References
- Appendices A–G, including joint formulation, DPVO verification, Dev10, design ablations, failures and additional visualizations
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied artifact is arXiv:2604.04198v2, dated 1 July 2026. Its title and all ten authors match the catalog; the catalog submission date is 5 April 2026. The earlier version was not supplied, so changes between versions were not compared.
- Text extraction does not reconstruct figure images; this was addressed by inspecting the retained PDF, with visual scope recorded separately.
- Separate supplemental material availability has not been fully verified. The appendices embedded in the supplied PDF were read completely.
- Code was not inspected and experiments were not reproduced. No external sources were accessed.
- Figure 1 was read through its extracted labels and caption but not visually inspected; Figures 2–12 and Tables 1–11 were visually inspected.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block and arXiv margin
The title and ten authors match the supplied catalog. The artifact is arXiv:2604.04198v2 [cs.CV], dated 1 July 2026; the affiliations are University of Twente, Xiaomi EV, University of Cambridge and University of Bath.
Go to primary source ↓e-problemPDF p. 1, Abstract; p. 8, Section 4.1 continuation
The authors motivate transferable driving with aligned visual and trajectory forecasts and propose adapting pretrained video dynamics through a shared generative process.
Go to primary source ↓e-architecturePDF p. 6, Figure 2 and caption; p. 8, Section 4.2
Figure 2 shows encoded history, language and ego state conditioning a unified DiT with noisy future video and action tokens. Section 4.2 specifies that text enters by cross-attention, rather than concatenation with the visual/action stream.
Go to primary source ↓e-interfacePDF p. 7, Section 4.1, numbered outputs and rolling-horizon paragraph; p. 15, Section 5.7, Prediction time horizon
Each action represents ego x, y and yaw. The model predicts an action chunk and future video; the stated rolling-horizon procedure executes actions, obtains observations and updates the history buffer. The trajectory horizon is K=8, spanning 4 seconds.
Go to primary source ↓e-encodingPDF p. 8, Section 4.2, Eq. (4); p. 9, Section 4.3, Eqs. (5)–(7)
A frozen Wan2.2-TI2V-5B text encoder supplies text tokens. A 3D causal video VAE produces temporally downsampled latents; visual latents are flattened/projected, and MLPs embed ego state and actions. History/state form a fixed clean condition block.
Go to primary source ↓e-flowPDF p. 9, Sections 4.3–4.4, Eqs. (6)–(9); p. 7, Section 3, Eq. (3)
Future video and action tokens comprise the generative target. Linear interpolation runs from Gaussian noise at s=0 to clean targets at s=1; the DiT regresses the clean-target-minus-noise velocity, conditioned on history/state and text.
Go to primary source ↓e-jointPDF pp. 22–23, Appendix B, Eqs. (10)–(11) and Conditioning interface
The video/action joint distribution admits a chain-rule interpretation as video continuation and action grounding, but implementation uses one DiT. Target modalities attend bidirectionally during denoising; history/state conditions remain noise-free.
Go to primary source ↓e-trainingPDF p. 10, Section 5.2, Training Details
Wan2.2-TI2V-5B training uses four history and eight future frames at 2 FPS, 832×480 images, NVIDIA H20 GPUs, AdamW, learning rate 10^-4, weight decay 0.01 and distributed bf16. Training runs 20k steps at batch 80, then 10k at effective batch 640; warm-up lasts 1k steps. Inference uses two sampling steps.
Go to primary source ↓e-protocolPDF pp. 9–10, Section 5.1; p. 11, Table 2 caption and Section 5.3
NAVSIMv1 Navtest uses PDM-based metrics. Zero-shot evaluation trains on NAVSIM and tests directly on the 150-scene nuScenes validation split and Bench2Drive validation split without target fine-tuning. DriveVA uses front-view camera input; simulation-enhanced training is a separate setting.
Go to primary source ↓e-navsimPDF p. 10, Table 1, Ours, WoTE, PWM and DiffusionDrive rows
Ours reports NC 99.2, DAC 97.5, TTC 98.7, comfort 100, ego progress 83.5 and PDMS 90.9. WoTE reports 88.3 PDMS; PWM and DiffusionDrive each report 88.1. Sensor modalities differ across rows.
Go to primary source ↓e-transferPDF p. 11, Table 2, all rows and dataset/metric headers; p. 12, Section 5.4 continuation
For nuScenes, DriveVA/PWM/DriveVLA-W0 report Avg L2 0.84/3.99/1.43 m and collision 0.06/0.36/0.77%. On Bench2Drive they report 1.33/2.80/3.00 m and 1.79/3.76/2.52%. Table 2 places author-labelled Avg columns beside 1-, 2- and 3-second values. Two collision summaries do not reproduce the arithmetic mean of their displayed horizons: PWM nuScenes 0.12/0.15/0.86 gives about 0.38, versus printed 0.36; DriveVA Bench2Drive 1.38/1.97/2.65 gives 2.00, versus printed 1.79. No aggregation explanation resolving these differences was located in the supplied text. These recomputations diagnose an unresolved source inconsistency and do not replace the printed experimental values.
Go to primary source ↓e-dpvoPDF p. 13, Table 4; pp. 23–24, Appendix D, Eq. (12)
DPVO reconstructs motion from ground-truth and generated videos; each reconstruction is aligned to its own reference by a 2D similarity transform before 4-second average L2 evaluation. NAVSIM GT/predicted errors are 0.09/0.16; nuScenes errors 0.07/0.14; cross-dataset averages 0.08/0.15. Table 4 does not explicitly label units or sample counts.
Go to primary source ↓e-main-ablationPDF p. 13, Table 5, rows 1–4 and caption; p. 14, Section 5.7, Effect of key designs
Full configuration has PDMS 90.9. Removing video loss gives 71.4; removing CARLA mix training gives 90.5; removing video continuation gives 84.6. Other switches remain enabled in each corresponding row.
Go to primary source ↓e-sim-mixPDF p. 13, Table 5, rows 2 and 4; Simulation-Enhanced Real-World Transferring paragraph
Mixing NAVSIM and simulation data raises PDMS from 90.5 to 90.9. The mixed configuration reproduces the full metric vector of Table 1; it must be distinguished from NAVSIM-only zero-shot training.
Go to primary source ↓e-further-ablationPDF p. 13, Tables 6–9; p. 15, Section 5.7
Four/eight/twelve future frames yield PDMS 82.1/90.9/86.7. Scratch/LoRA/full fine-tuning yields 62.9/74.9/90.9. One/two/three sampling steps yield 13.2/90.9/90.9. The 14B LoRA model scores 80.6, below the 5B fully fine-tuned model's 90.9.
Go to primary source ↓e-mask-ablationPDF p. 26, Table 11 and Appendix E.2
Causal-mask and bidirectional variants score 90.1 and 90.9 PDMS. The prediction-target ablation reports Action Only 47.0 and Video + Action 90.9. The text says the causal mask restricts interaction between future video and action tokens.
Go to primary source ↓e-closed-loopPDF p. 26, Table 10 and Appendix E.1
On the small Bench2Drive Dev10 diagnostic split, DriveVA reports DS 67.47 and SR 70.00; DriveMamba-12L reports 66.50/40.00 and DriveTransformer 60.45/30.00. The authors explicitly treat this as preliminary auxiliary evidence, not full Bench2Drive evaluation.
Go to primary source ↓e-failurePDF p. 27, Figure 8, caption and Appendix F
The two failures predict stopping when ground truth bypasses a cyclist or proceeds through an intersection. Generated video and predicted trajectory remain mutually consistent despite selecting the wrong future mode.
Go to primary source ↓e-qualitativePDF p. 12, Figure 3; p. 14, Figure 4; p. 15, Figure 5; pp. 24–25, Figures 6–7; pp. 28–30, Figures 9–12
Qualitative examples compare forecasts and trajectories, show DPVO reconstructions, and illustrate transfer to nuScenes and CARLA. They are selected visual examples rather than a quantified failure-frequency study.
Go to primary source ↓e-implementation-gapsPDF p. 9, Eq. (9); p. 10, Sections 5.1–5.2; pp. 22–23, Appendix B; p. 26, Appendices E.1–E.2
The source gives token noising, optimizer and training schedule, but does not fully specify loss weighting, action normalization, CARLA mixing ratio, exact mask implementation, GPU count, software versions or trajectory-to-control integration. Section 5.2 mentions a trajectory loss alongside video flow matching, without a separate complete formula.
Go to primary source ↓8.5 Primary sources
DriveVA: Video Action Models are Zero-Shot Drivers ↗
PDF · 10,706 extracted words
Source fingerprint
1e991299738a5dfc300567d09f90f8ce3188bc47d39094ee8b81bdbcec49ab74