DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
1. Paper overview
In one sentence: DrivingGen reveals tradeoffs between convincing driving videos and faithful ego motion, while making trajectory evaluation depend on imperfect motion reconstruction. e-flowe-datae-evaluatione-slame-limitations
| At a glance | What to know |
|---|---|
| Research problem | Source description Generic video realism can conceal implausible ego motion, unstable agent identity or failure to follow an intended route. DrivingGen asks whether these properties can be measured together across diverse driving conditions, while keeping evaluation affordable enough for repeated comparison. e-probleme-data |
| Core mechanism | Source description A 400-sample evaluation dataset pairs 200 internet-sourced open-domain cases with 200 ego-conditioned cases from five driving datasets, using front-view images, scene descriptions and optional trajectories. e-data |
| A key reported result | Ego-conditioned trajectory alignment: Vista: ADE 19.70; DTW 1216. ADE and DTW, lower is better; table does not state units.. Ego-conditioned track: 200 cases; 100-frame prediction horizon; Table 3. UniFuture: 20.21 / 1352; Kling 2.1: 29.97 / 2310. Vista subjective quality is 0.4146 versus Kling's 0.5468. Vista leads reported alignment while lagging this visual-quality comparator. Errors include reconstruction effects; uncertainty intervals and per-model conditioning adapters are not reported. e-datae-evaluatione-ego-resultse-alignment |
| Reading caution | Source description The authors limit conclusions to open-loop single-front-camera generation. Long-tail coverage is constrained; downstream driving performance, scene-content control and counterfactual reasoning are unevaluated. e-limitations |
Core contributions
- Source description
A 400-sample evaluation dataset pairs 200 internet-sourced open-domain cases with 200 ego-conditioned cases from five driving datasets, using front-view images, scene descriptions and optional trajectories. e-data
- Source description
A modular metric suite couples visual distribution, quality and consistency with trajectory distribution, kinematic quality, consistency and alignment; fourteen existing generators are compared. e-metricse-evaluation
Figure 1. Generated pixels supply both the visual evidence and the recovered-motion evidence. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at World Condition on the left. The solid Vision box supplies a front-view image; the dashed Language and Action boxes mark optional conditions. Action is an ego-trajectory input, rather than an executed command. Follow the large arrows down into a generator, right into Generated Videos and up into the evaluator. Inside DrivingGen Metrics, the gray SLAM arrow runs from Videos to Trajectories. Both representations connect to distribution, quality and temporal consistency, whereas the dashed trajectory-alignment connection comes from trajectories. Sections 3.1–3.2 clarify that alignment compares recovered motion with the conditioning trajectory; the diagram abbreviates that reference input. e-flowe-datae-metricse-alignmente-evaluatione-slame-limitations
What it supports. The figure exposes a shared dependency: a generated image artifact can influence a visual metric directly and a trajectory metric through SLAM. The evaluation has complementary outputs, but these outputs are not independent measurements of a known physical state. Their agreement is evidence to examine, rather than proof of correctness.
Where the evidence stops. This is an evaluation architecture. It specifies no action decoder, training loss or feedback controller. Optional conditioning in the overview also leaves model-specific trajectory-input adapters unresolved in the reported experiments.
2. Motivation
2.1 The problem and the proposed response
Generic video realism can conceal implausible ego motion, unstable agent identity or failure to follow an intended route. DrivingGen asks whether these properties can be measured together across diverse driving conditions, while keeping evaluation affordable enough for repeated comparison. e-probleme-data
2.2 What this reading follows
A driving video can look coherent while its implied vehicle motion is implausible or ignores the requested route. DrivingGen turns that mismatch into an evaluation problem. It supplies separate open-domain and ego-conditioned tracks, then scores generated pixels, tracked agents and trajectories reconstructed from those pixels. The useful result is a profile of strengths and failures across fourteen generators. Reading the benchmark also requires reading its evaluator: a reconstruction fallback keeps difficult videos in the analysis, and learned visual judges contribute their own uncertainty. The figures below connect dataset design, metric flow, model comparisons and validation without treating these open-loop scores as evidence of safe autonomous driving. e-flowe-datae-evaluatione-slame-limitations
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded benchmark classification is supported by a dataset-plus-metrics pipeline comparing independent generators. DrivingGen supplies neither a unified action/world-prediction architecture nor an inverse-dynamics controller. Architecture, prediction paradigm and quadrant are appropriately not applicable. e-flowe-metricse-evaluation
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Separate scenario coverage from the ability to follow a route
DrivingGen's two tracks answer different questions. The open-domain track begins with internet driving footage to broaden scene conditions; the ego-conditioned track uses existing driving datasets that provide the trajectories needed for alignment evaluation. Each sample supplies a front-view image and a description produced using Qwen, with an ego trajectory available as an optional action condition in the second track. These descriptions concern future dynamics and camera movement, so the evaluation is explicitly conditioned generation rather than an unconditional forecast from pixels alone. After generation, source videos and trajectories supply distributional references. The track distinction is essential when reading Table 3: an FVD change across tracks also changes the reference data. The dataset's intended generalization role does not itself establish model-specific exclusion from pretraining. e-datae-flowe-evaluation
Figure 2(a). The open-domain distribution broadens weather, lighting and regional coverage. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel independently: its colors encode weather, time of day or region, and the vertical axis represents percentages. Compare the Ours bar with the two validation-set bars, rather than comparing colors across panels. DrivingGen's weather bar allocates 51.8% to normal conditions, 22.5% to rain, 13.1% to snow and 12.6% to fog; its daytime share is 50.5%. Section 3.1 associates these distributions with the open-domain track. The ego-conditioned track has separate statistics in Figure 3 on page 19, which was also inspected. Thus this crop should not be read as one distribution describing every benchmark case. e-datae-data-visual
What it supports. The plotted sample mix gives adverse weather and non-daytime driving more representation than the comparison validation sets. This supports the authors' motivation for broader evaluation coverage. It does not establish that any particular model has never encountered these scenes or similar internet material during training.
Where the evidence stops. The regional graphic prints Middle East & North Africa as 12.2%, while page 6 says 12.1%; both remain unresolved. Figure 3's ego-track weather/daytime percentages also differ from the text's 60% description. Sampling diversity alone does not verify training-set separation.
5.2 Reconstruct motion before interpreting a motion score
A trajectory score in this benchmark is downstream of a vision system. SIFT, RANSAC, PnP and metric depth first translate a generated video into an estimated camera path. FTD then encodes normalized ten-step windows through MTR, averages the window embeddings and compares their distribution with reference embeddings. Quality and consistency metrics instead operate on path derivatives. As a reader's deduction, a single geometry artifact can therefore affect several reported motion measurements through the same upstream failure. The fallback prevents failed reconstructions from disappearing from the dataset, but inserts constant-velocity motion and random orientation noise. Meanwhile, the quality recipe skips NaNs for certain static or short trajectories. These two rules make it important to distinguish completed reconstruction, valid metric inputs and trustworthy physical motion. e-metricse-ftde-slame-trajectory-qualitye-consistency
5.3 Interpret agreement and tradeoffs without turning scores into safety claims
The ego-track table gives a concrete example of why the full score profile matters. Vista leads ADE and DTW, but Kling and Gen-3 score better on subjective image quality. The open-domain table provides another warning: VaViM's strong video consistency coexists with weak trajectory consistency. These are observed comparisons, while explanations involving generator collapse or perception error require further testing. Human alignment provides supporting validation of six metric categories, with weaker correlations for trajectory measurements, but does not separate generation error from reconstruction error. My interpretation is that DrivingGen is most useful for locating disagreements worth investigating. Its current experiments cannot establish how a planning stack behaves in interactive traffic, because closed-loop interaction, downstream driving tasks and scene-content controllability are explicitly outside the benchmark's evaluated scope. e-ego-resultse-open-resultse-humane-limitations
5.4 Training and inference
During training
DrivingGen evaluates existing generators and uses existing feature, depth, detection, tracking and reasoning models. It specifies no new generator-training objective or training schedule; this is an evaluation pipeline. Exact checkpoint choices and per-generator configuration are not comprehensively specified in the paper. e-metricse-temporale-disappearancee-evaluation
During inference
Generate a 100-frame prediction, then evaluate video and reconstructed motion. No action is executed and no feedback loop is evaluated. Although Table 3 lists alignment scores for all models, the text does not explain model-by-model adaptation of numeric ego trajectories to every generator's input interface. e-evaluatione-alignmente-ego-resultse-limitations
5.5 Implementation flow
- Construct separate evaluation tracks
Open-domain cases broaden weather, lighting and geographic coverage. Ego-conditioned cases combine Zod, DrivingDojo, COVLA, nuPlan and WOMD, whose source videos and trajectories define the reference distributions. Qwen supplies descriptions of future scene dynamics and camera movement. e-data
- Recover motion from generated pixels
SIFT/RANSAC/PnP with UniDepthV2 recovers camera trajectories. Failed estimates trigger constant-velocity extrapolation with small random orientation perturbations. This retains failed videos, but makes trajectory scores depend partly on evaluator behavior. e-metricse-slam
- Compare distributions and quality
FVD evaluates video distributions. FTD applies a Gaussian Fréchet comparison to pooled MTR trajectory embeddings. CLIP-IQA+ estimates perceptual quality; MMP assesses luminance modulation. Trajectory quality geometrically combines comfort, mobility and curvature. e-metricse-ftde-image-qualitye-trajectory-quality
- Track consistency and conditioning
Optical-flow-adaptive sampling precedes DINOv3 scene comparisons. YOLOv10 first-frame detections and SAM2 tracks support agent appearance comparisons. Cosmos-Reason1 classifies disappearances from three frames. Separate kinematic consistency, ADE and DTW scores measure motion stability and conditioning alignment. e-temporale-disappearancee-consistencye-alignment
6. Experiments & results
DrivingGen evaluates driving video generators through complementary visual and recovered-motion measurements. Its two tracks separate broad scenario coverage from ego-trajectory alignment. The central finding is a metric tradeoff: attractive videos, stable agents and accurate commanded motion do not share one universal winner. Trajectory conclusions remain mediated by the reconstruction pipeline.
6.1 Read the original evidence
Table 3, Open-Domain Track. Open-domain rankings vary with the property being measured; asterisks mark commercial models. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read a row horizontally before consulting Avg. Rank. FVD and FTD are distances, so lower values indicate closer distributions; quality and consistency scores reward larger values. The source highlights the best three entries in red, orange and blue, and uses an asterisk for commercial closed-source models; that shared key is retained with the second table crop. Agent Missing is particularly easy to misread: Section 3.2.3 defines it as the share of videos with no abnormal disappearance, so higher is better. Compare Cosmos-Predict2's FVD with Kling's FTD and subjective quality, then inspect VaViM's visual and trajectory consistency columns. e-datae-metricse-evaluatione-open-resultse-disappearance
What it supports. Cosmos-Predict2 leads FVD at 524.1, whereas Kling's FVD is 693.4 and its FTD is 26.73. VaViM reports video consistency 0.9159 alongside trajectory consistency 0.0914. These examples show why one favorable image or video statistic cannot summarize the benchmark's motion-related findings.
Where the evidence stops. The 200 open-domain cases share a 100-frame evaluation horizon, but the paper does not establish equal training data or generation compute. Average rank is explicitly a summary; no uncertainty intervals accompany this table.
Table 3, Ego-Conditioned Track and shared key. Vista leads reported trajectory alignment, while visual-quality leaders differ. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Move to the two Trajectory Alignment columns near the right edge. ADE compares positions at corresponding times; DTW permits nonlinear time alignment and measures path discrepancy. Both are better when smaller. Vista's 19.70 and 1216 are the lowest entries, followed by UniFuture's 20.21 and 1352. Now move left to Subjective Quality: Vista's 0.4146 trails Kling's 0.5468 and Gen-3's 0.5740. Wan2.2-I2V instead leads FVD at 194.4. The retained source key explains ranking colors and commercial-model asterisks. All these comparisons belong to the ego-conditioned track and should be kept separate from the open-domain numbers. e-datae-evaluatione-ego-resultse-alignmente-slame-limitations
What it supports. The table demonstrates a practical selection tradeoff within this protocol: the model with the smallest reported alignment errors does not have the highest subjective image-quality score. Vista is displayed at rank 5 despite leading ADE and DTW. A research use case that prioritizes controllability should therefore inspect the alignment columns directly.
Where the evidence stops. ADE/DTW units and full per-model conditioning interfaces are not specified. The accompanying text attributes errors to both generated motion and SLAM artifacts. Low alignment error is not a measurement of safe executed driving, and cross-track distance values use different reference distributions.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Ego-conditioned trajectory alignment Ego-conditioned track: 200 cases; 100-frame prediction horizon; Table 3. | Vista: ADE 19.70; DTW 1216. ADE and DTW, lower is better; table does not state units. | UniFuture: 20.21 / 1352; Kling 2.1: 29.97 / 2310. Vista subjective quality is 0.4146 versus Kling's 0.5468. Vista leads reported alignment while lagging this visual-quality comparator. Errors include reconstruction effects; uncertainty intervals and per-model conditioning adapters are not reported. e-datae-evaluatione-ego-resultse-alignment |
| Open-domain visual realism versus recovered dynamics Open-domain track: 200 cases; 100-frame prediction horizon. | Cosmos-Predict2 FVD: 524.1. Kling FTD: 26.73; subjective quality: 0.5538; displayed rank: 1. FVD and FTD lower; subjective quality higher. | Kling FVD is 693.4. VaViM combines video consistency 0.9159 with trajectory consistency 0.0914. Different metrics select different strengths. A high visual consistency score alone does not establish plausible motion; aggregate rank is only a summary. e-datae-evaluatione-open-results |
| Trajectory-reconstruction failure handling Separate diagnostic on 20 nuPlan videos generated with Vista. | With failure handling: 20/20; ADE 16.84. Valid reconstruction count, higher; ADE over successful runs, lower. | Without: 17/20 and 15.18. GEM pipeline: 17/20 and 14.61; DrivingDojo pipeline: 16/20 and 14.99. Completeness improves, but the ADE averages cover different subsets. This does not demonstrate improved pose accuracy on a common sample set. e-slam |
| Metric agreement with human preferences Model-level pairwise win ratios; six selected metric categories in Figure 5. | Video realism/quality/consistency: 0.8899 / 0.8344 / 0.8757. Spearman correlation, higher. | Trajectory realism/quality/consistency: 0.7682 / 0.6261 / 0.7467. Trajectory metrics show weaker agreement. Participant counts, comparison counts and confidence intervals are absent; this is preference alignment rather than driving-safety validation. e-humane-limitations |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Fallback increases reconstruction coverage, while changing the population included in ADE. Original paper, p. 20 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the last two rows first because they compare the authors' evaluator with and without failure handling. The success count rises from 17/20 to 20/20, while ADE changes from 15.18 to 16.84. Appendix B.2 explains the intervention: if pose estimation fails, propagate the last pose using constant velocity and add small random orientation perturbations. The other rows compare GEM's DROID-SLAM plus depth pipeline and DrivingDojo's COLMAP pipeline with ground-truth scale alignment. Crucially, the retained caption defines ADE over successful reconstructions. Each row can therefore average a different set of videos. e-slam
What it supports. The diagnostic supports improved coverage: all twenty videos receive a valid trajectory under the fallback strategy. It does not show improved geometric accuracy. Comparing ADE only on the common successful subset would answer a different question from the reported comparison, and would help separate estimator accuracy from the cost of retaining difficult cases.
Where the evidence stops. This is an evaluator ablation on twenty Vista/nuPlan videos, not a generator-training ablation or a proof of robustness across the full dataset. Extrapolated trajectories contain assumed motion; the random perturbation magnitude and seed are unspecified.
Figure 5. Human agreement is stronger for the three visual measurements than for their trajectory counterparts. Original paper, p. 23 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the panel headings and printed correlation coefficients to navigate the figure. The top row concerns video realism, quality and consistency; the bottom row concerns trajectories. Each point represents a model, with metric win rate on the horizontal axis and human win rate on the vertical axis. Appendix B.9 explains that a human pairwise win contributes one point, a loss zero and a tie one half, divided by the model's comparison count. The six tested metrics are FVD, FTD, subjective image quality, trajectory quality, video consistency and trajectory consistency. The plotted lines help show association rather than defining a calibrated safety threshold. e-humane-limitations
What it supports. The three video correlations are 0.8899, 0.8344 and 0.8757. The trajectory correlations are 0.7682, 0.6261 and 0.7467, with trajectory quality weakest. The authors suggest noisy monocular reconstruction as an explanation for the gap; the figure establishes the association but does not isolate that causal explanation.
Where the evidence stops. The source omits annotator counts, comparison counts and uncertainty intervals. Several point labels overlap, and one is clipped within the original graphic; no missing label is reconstructed here. This validation covers six selected metrics, not the entire suite or downstream driving safety.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors limit conclusions to open-loop single-front-camera generation. Long-tail coverage is constrained; downstream driving performance, scene-content control and counterfactual reasoning are unevaluated. e-limitations
SLAM fallback includes every video in trajectory reconstruction, yet trajectory-quality averages skip NaNs, including some static/short cases. Reconstruction completeness therefore does not guarantee identical denominators across metrics. e-slame-trajectory-quality
MMP is described as a time-window fraction in the main text but as a single-clip binary FFT test in Appendix B.4. The latter is the explicit recipe used here; their equivalence remains unresolved. e-image-quality
Internet sourcing is intended to test unseen scenarios, but the paper provides no model-specific training-overlap audit. Reported dataset percentages also differ internally: Figure 2 gives 12.2% for Middle East & North Africa versus 12.1% in text; ego-track weather/daytime values differ from the text's 60% description. e-datae-data-visual
7.2 Questions for discussion
- How much of the trajectory ranking survives when reconstruction failures are evaluated separately on matched videos?
- Can a stationary video still score highly after adaptive sampling, and how should valid stops be distinguished from generator collapse?
- What conditioning interface and training-overlap checks are needed before comparing controllability across general and driving-specific models?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the curated cases, reference trajectories, captions, generator checkpoints/interfaces and evaluator components. FTD specifies ten-step windows, agent-centric normalization and fixed vehicle dimensions. Further clarification is needed for SLAM perturbation magnitude, sampling thresholds, zero denominators and NaN aggregation. e-datae-ftde-slame-temporale-consistencye-trajectory-quality
The source estimates 1–2 days for all metrics on 400 100-frame videos on one modern GPU. Wan2.2-14B generation is approximately 20–30 minutes per video with at least 40 GB GPU memory. These are coarse estimates; exact GPU model and software versions are unstated. e-runtime
The proposed checks in the illustrated edition isolate fallback sensitivity and resistance to near-static videos. Both would test evaluator behavior while holding generated content fixed; neither has been run for this report. e-slame-temporale-trajectory-quality
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate fallback coverage from reconstruction accuracy
Reader-proposed check, not performed: reuse the same twenty Vista-generated nuPlan videos and reference paths from the Table 4 diagnostic. Hold frames, calibration and depth predictions fixed within the authors' pipeline; compare failure handling disabled, constant-velocity continuation alone, and continuation with the stated orientation perturbations. Since the perturbation magnitude is unspecified, document a prespecified sensitivity sweep and repeated random seeds. Report reconstruction counts, imputed-frame fractions and ADE both on the common successful subset and on all retained videos. Also compare trajectory quality and consistency. If scores or rankings change mainly with perturbation settings while input videos remain fixed, the evaluator is materially determining the apparent motion fidelity; stable scores would weaken that concern. e-slame-trajectory-qualitye-consistency
Check 2: Test whether near-static video can still exploit consistency scoring
Reader-proposed check, not performed: select benchmark clips with clear forward motion and retain genuine stationary driving scenes as a separate control. For each moving clip, construct a frozen-first-frame version and versions with increasing frame repetition, keeping the evaluated length at 100 frames. Compare fixed-stride DINOv3 similarity with the paper's optical-flow-adaptive sampling; hold the backbone and all other settings constant. Report motion score, trajectory-quality NaN counts and the number of valid samples alongside video consistency. Frozen footage should not retain an apparent advantage that is hidden by excluding invalid trajectory scores. If it does, the stated defense against near-static score inflation is incomplete; the stationary-scene control tests whether legitimate stops are penalized indiscriminately. e-temporale-evaluatione-trajectory-qualitye-consistency
8.3 Reading coverage
Visual audit: The title, authors, affiliations and v2/date were inspected on page 1. Visual inspection covered Figure 1's optional-condition boxes, SLAM arrow and trajectory-only alignment connection; all panels of Figure 2; Figures 3–4; Tables 1–5; and all six panels of Figure 5. Pages 5–8 and 19–23 were visually read for dataset, metric, equation, fallback, human-validation and resource details, including every page supporting the proposed checks. All six final 300-DPI crops were individually viewed; bounds were adjusted for readability. The Table 3 shared color/asterisk key and Table 4 successful-run averaging note are retained because they determine interpretation. Source-overlapping/clipped labels in Figure 5 remain unchanged. Reference pages 11–17 and related-work page 18 were read as text, not rendered; no report claim requires an unseen page image. No separate supplement, code or experiment execution was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 19, 20, 21, 22, 23. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and Sections 1–2: motivation and related-work framing (pp. 1–3)
- Section 3.1: dataset and both evaluation tracks (pp. 4–6)
- Sections 3.2.1–3.2.4: distribution, quality, temporal consistency and trajectory alignment (pp. 6–8)
- Sections 4–6: experiments, observations, conclusion and limitations (pp. 8–10)
- References (pp. 11–17)
- Appendix A.1–A.2: related works (p. 18)
- Appendix B.1–B.9: gallery, SLAM, FTD, image/trajectory quality, disappearance, consistency, resources and human alignment (pp. 19–23)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The complete supplied extraction was read in all eight chunks. Text extraction does not reconstruct figure images; this acquisition limitation was addressed by inspecting the retained PDF's relevant pages and every final crop.
- Separate supplemental material availability has not been fully verified.
- No external source, linked code, dataset files or project site was inspected; no experiments were reproduced.
- Identity: the observed title and all six authors match the catalog. The inspected artifact is arXiv:2601.01528v2, dated 7 March 2026, and labels itself an ICLR 2026 conference paper. The catalog specifies no revision; v1 and a separate venue edition were not supplied, so revision differences cannot be determined.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, author affiliations, equal-contribution footnote and arXiv margin
The title matches the supplied observed title. Authors are Yang Zhou, Hao Shao, Letian Wang, Zhuofan Zong, Hongsheng Li and Steven L. Waslander. Affiliations are University of Toronto and CUHK MMLab. Zhou and Shao contributed equally. The artifact states arXiv:2601.01528v2, 7 March 2026, and publication at ICLR 2026.
Go to primary source ↓e-problemPDF p. 2, Introduction and Table 1, DrivingGen row
The proposed evaluation covers visual and trajectory distribution, quality, temporal consistency and trajectory alignment; Table 1 leaves downstream task evaluation absent.
Go to primary source ↓e-flowPDF p. 3, Figure 1 and caption
A front-view image with optional language/action conditions feeds a video generator. Generated videos feed evaluation directly and via SLAM-reconstructed trajectories. Language and action boxes are dashed; trajectory alignment is connected to trajectories by a dashed line.
Go to primary source ↓e-dataPDF pp. 5–6, Section 3.1, Dataset Construction and diversity bullets; p. 19, Appendix B.1
There are 400 samples, 200 per track. Open-domain samples come from internet video; the ego-conditioned track aggregates Zod, DrivingDojo, COVLA, nuPlan and WOMD. Inputs include front-view RGB, Qwen scene descriptions and optional ego trajectories. The ego-track source videos and trajectories define reference distributions.
Go to primary source ↓e-data-visualPDF p. 4, Figure 2(a); p. 6, geographic-coverage bullet; p. 19, Figure 3(a)
Figure 2 shows open-domain normal weather 51.8%, rain 22.5%, snow 13.1%, fog 12.6%, and daytime 50.5%. Its Middle East & North Africa label is 12.2%, whereas p. 6 says 12.1%. Figure 3 shows ego-track normal weather 55.4% and daytime 54.5%, whereas p. 6 describes normal weather/daytime as 60%.
Go to primary source ↓e-metricsPDF p. 6, Table 2 and Sections 3.2–3.2.1
Four metric groups cover distribution, quality, temporal consistency and trajectory alignment. Trajectory recovery uses SIFT, RANSAC, PnP and UniDepthV2. FVD follows the StyleGAN-V computation protocol; FTD uses MTR trajectory embeddings.
Go to primary source ↓e-ftdPDF pp. 20–21, Appendix B.3, FTD equation and Practical recipe
MTR's agent polyline encoder encodes non-overlapping H=10-step windows after agent-centric translation/rotation. Window embeddings are averaged per trajectory. Generated/reference empirical means and covariances enter a Gaussian Fréchet distance. The recipe adds epsilon I with epsilon=10^-6, permits covariance shrinkage, and fixes vehicle dimensions at (4.5, 2.0, 1.8) m.
Go to primary source ↓e-image-qualityPDF p. 7, Section 3.2.2, Visual Quality; p. 21, Appendix B.4, Definition, Computation and Defaults
CLIP-IQA+ estimates subjective image quality. Appendix B.4 defines MMP from a single FFT of frame-mean grayscale luminance, a dominant non-DC peak and its band-power ratio A. If the peak is below 0.2 Hz, MMP=1; otherwise MMP is the indicator A<0.05, using a 0.5-Hz band half-width and 10 fps. Section 3.2.2 instead describes a fraction of time windows.
Go to primary source ↓e-trajectory-qualityPDF p. 21, Appendix B.5, all submetrics
Trajectory quality combines comfort, motion and curvature by a weighted geometric mean, equally weighted by default. Finite differences estimate kinematics. Never-moving trajectories have motion score zero; comfort is NaN for non-moving or at-most-1-m paths, curvature is NaN for non-moving paths, and dataset means skip NaNs. The moving threshold is 0.1 m/s.
Go to primary source ↓e-temporalPDF p. 7, Section 3.2.3, Video Consistency and Agent Appearance Consistency
Median optical flow controls adaptive temporal downsampling before consecutive-frame DINOv3 comparisons. Agents detected in the first frame by YOLOv10 are tracked using SAM2; crop features are compared across consecutive frames and against the first frame.
Go to primary source ↓e-disappearancePDF pp. 7–8, Section 3.2.3, Agent Abnormal Disappearance; p. 22, Appendix B.6
Cosmos-Reason1 judges natural versus unnatural disappearance from the first visible, last visible and first absent frames. Visible-frame boxes are green. A video is clean only when all evaluated tracklets are not abnormal; the score is the clean-video percentage.
Go to primary source ↓e-consistencyPDF p. 22, Appendix B.7, equations
Speed and acceleration dispersion ratios are std(v)/mean(v) and std(a)/mean(|a|); their negative exponentials are averaged. The printed equations do not specify zero-denominator handling.
Go to primary source ↓e-alignmentPDF p. 8, Section 3.2.4
ADE compares generated and conditioning trajectories pointwise over the prediction horizon. DTW uses nonlinear temporal alignment and Euclidean pointwise cost to assess path-shape discrepancy.
Go to primary source ↓e-evaluationPDF p. 8, Section 4, Evaluation Setup and Section 4.1 introduction
Fourteen models comprise seven general video, two physical-world and five driving-specific models. Every model is evaluated over 100 frames. Average rank is a quick summary, explicitly not a definitive score.
Go to primary source ↓e-open-resultsPDF p. 9, Table 3, Open-Domain Track, Kling 2.1, Cosmos-Predict2, VaViM and UniFuture rows
Kling has displayed rank 1, FTD 26.73 and subjective quality 0.5538. Cosmos-Predict2 has FVD 524.1 versus Kling's 693.4. VaViM has video consistency 0.9159 and trajectory consistency 0.0914. UniFuture's objective quality is 0.9054. Red/orange/blue denote first/second/third best; stars denote commercial closed-source models.
Go to primary source ↓e-ego-resultsPDF p. 9, Table 3, Ego-Conditioned Track, Vista, UniFuture, Kling 2.1, Wan2.2-I2V and Gen-3 Alpha Turbo rows; paragraph on trajectory alignment
Vista has ADE 19.70, DTW 1216, subjective quality 0.4146 and displayed rank 5; UniFuture has ADE 20.21 and DTW 1352. Kling has ADE 29.97, DTW 2310 and subjective quality 0.5468. Wan has FVD 194.4; Gen-3 has subjective quality 0.5740. The text attributes alignment error to both reconstruction artifacts and imperfect generated motion.
Go to primary source ↓e-slamPDF pp. 19–20, Appendix B.2; p. 20, Table 4 and caption
Failed pose estimates are replaced with constant-velocity extrapolation and small random orientation perturbations. On 20 nuPlan videos generated by Vista, the basic pipeline reconstructs 17/20 with ADE 15.18; failure handling yields 20/20 with ADE 16.84. GEM's pipeline yields 17/20 and 14.61; DrivingDojo's yields 16/20 and 14.99. ADE averages only successful runs.
Go to primary source ↓e-humanPDF p. 23, Figure 5, all six panels and caption; Appendix B.9
Spearman correlations for video realism, quality and consistency are 0.8899, 0.8344 and 0.8757; trajectory counterparts are 0.7682, 0.6261 and 0.7467. Human pairwise wins score 1, losses 0 and ties 0.5, normalized by comparisons per model. Six primary metrics are tested. Annotator count, comparison count and confidence intervals are not provided here.
Go to primary source ↓e-runtimePDF p. 22, Table 5 and caption; p. 23, Appendix B.8
All metrics for 400 100-frame videos are estimated at 1–2 days on one modern GPU. Wan2.2-14B generation is described as about 20–30 minutes per video on a single GPU with at least 40 GB memory. These are approximate hardware-dependent estimates; an exact GPU model and software environment are not stated.
Go to primary source ↓e-limitationsPDF p. 10, Section 6, all limitation paragraphs
The authors identify limited long-tail coverage, open-loop-only evaluation, single-front-camera data, absence of downstream driving tasks, scene-content controllability and counterfactual evaluation, and the non-definitive nature of average rank.
Go to primary source ↓e-releasePDF p. 3, Extensive Benchmarking and Insights
The authors state that the dataset and evaluation code are publicly released; the paper's assertion does not verify the linked artifacts or implementation.
Go to primary source ↓8.5 Primary sources
DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving ↗
PDF · 11,876 extracted words
Source fingerprint
63b9a93db20653423afac7cf3f68ab94dcfb573f605745fd6eb596add603a63e