ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
1. Paper overview
In one sentence: ProphetDWM links an MLP action forecaster to a diffusion video model through learned action features, improving reported nuScenes prediction metrics while leaving closed-loop driving and precise counterfactual dynamics untested. E02E03E04E05E08E09E10E14E15
| At a glance | What to know |
|---|---|
| Research problem | Source description The authors seek prediction beyond the horizon of supplied low-level controls. In their finite-horizon POMDP formulation, the known future action sequence has length η, shorter than the desired observation horizon ΔT. Rewards are excluded. The model must therefore extend speed and steering signals while predicting their associated visual future. This is supervised modeling of recorded driving, without a reported reward optimizer or executed closed-loop controller. E02E03 |
| Core mechanism | |
| A key reported result | Single-view driving video generation: Reported FID 6.9; FVD 190.5. FID ↓; FVD ↓. nuScenes; setting 1 uses 192×384, sequence length 8, one reference frame and 2-fps keyframes. Evaluation split and sample count are not stated. Table 2: GenAD-nus 15.4/244.0; Drive-WM 23.3/228.5 (FID/FVD). Conditioning inputs differ. Best listed scores. The condition column says first image, although the method also uses actions. ADriver-I and extra-data GenAD/Vista settings are excluded by the authors; this is not a universal matched-protocol ranking. E07E08E17 |
| Reading caution | Reader analysis Action-control examples and long rollouts demonstrate selected generated futures. They do not quantify counterfactual physical accuracy, collision avoidance or closed-loop success. The qualitative comparison also uses different model resolutions. Claims of accurate long-term dynamics or absence of error accumulation exceed what selected images alone establish. E14E15E19 |
Core contributions
Figure 2. Two prediction modules share latent actions and refresh their context after each rollout. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the upper route first: the reference observation and supplied controls enter q_θ, which predicts the control suffix beyond the known action horizon. The green downward branch carries latent action features to q_φ, the video transition model. A separate dashed observation route supplies state information. Equations (2)–(4) make this dependency explicit: video prediction uses the current latent state and the action representation. The optional reference images correspond to additional observed context, not unrestricted future knowledge. Read the gray return arrows as iterative reuse of predicted imagery and controls. The training-only label refers to the noisy target-data construction; Section 3.3 still uses observation context during generation. E03E04E05E14
What it supports. The diagram supports two distinct learned components with a useful internal dependency: action features are produced before they condition video. Joint optimization couples their objectives, but the architecture does not show one backbone denoising both modalities. This is why the catalog's dual-system, joint-prediction classification is appropriate.
Where the evidence stops. The noisy target branch is a training construction, not a source of future observations at inference. Likewise, the outer feedback arrows depict model rollouts; neither the figure nor the reported experiment supplies a demonstrated physical control loop.
2. Motivation
2.1 The problem and the proposed response
The authors seek prediction beyond the horizon of supplied low-level controls. In their finite-horizon POMDP formulation, the known future action sequence has length η, shorter than the desired observation horizon ΔT. Rewards are excluded. The model must therefore extend speed and steering signals while predicting their associated visual future. This is supervised modeling of recorded driving, without a reported reward optimizer or executed closed-loop controller. E02E03
2.2 What this reading follows
A driving forecast needs both a picture of what comes next and controls consistent with that picture. ProphetDWM starts with reference imagery and a short action sequence, predicts the missing controls, and lets the same latent action features guide video generation. Its two modules learn together, while a separate visual-context pathway anchors the generated scene. This reading follows those two information routes, then separates the video scores, action errors and mechanism ablations. The results support useful predictive coupling on nuScenes, but the illustrated turns and long rollouts remain generated examples. They do not establish that the predicted controls can safely drive a vehicle. E02E03E04E05E08E09E10E14E15
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded Dual-system × Joint prediction classification is supported by the separate MLP action model q_θ and diffusion transition q_φ, coupled through latent actions and a joint loss. Joint prediction here includes action-to-video conditioning within one system; it does not imply one shared backbone or simultaneous action/video denoising. Actions are forecast from supplied context, not recovered through inverse dynamics from completed future video. E03E04E05
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Extend the controls before asking for the complete visual future
The central constraint is η<ΔT: supplied actions cover less time than the requested video. ProphetDWM addresses that missing suffix in its action module. An embedding and MLP extract features from known controls; transformed observation features help another MLP forecast future action features. Those features have two uses: a projection yields speed and steering predictions, while concatenation with the known-action features creates the sequence that conditions the video model. This explains how the video horizon can exceed the supplied-control horizon without demanding a complete external action plan. As a reader interpretation, the connection is best understood as a shared predictive representation, not a guarantee that every generated frame physically matches its predicted control. Action L1 and video metrics test different aspects of that claim. E02E03E04E08E09
5.2 Keep the action pathway separate from the scene anchor
The action features communicate anticipated dynamics, but the video model also needs a direct account of the observed scene. Section 3.3 creates this second route from VAE state features, repeating the last available observation state to construct context. At multiple U-Net blocks, zero-initialized convolutions transform that context into scale and shift terms used in the residual update of Equation (10). Latent actions enter through temporal cross-attention instead. During training, the diffusion model sees noisy known and future target states, and action L1 is added to its noise-prediction objective. At inference the target future is unavailable; generated frames and controls refresh the next rollout's references. This separation explains why the fusion comparison and the joint-training comparison probe different parts of the method. E04E05E10E11E14
5.3 Separate distributional quality, action error and rollout length
The strongest quantitative claim comes from the low-resolution nuScenes setting: one reference, eight-frame sequences and 2-fps keyframes yield FID 6.9 and FVD 190.5. Action prediction is a separate measurement, with average L1 0.0915 and a steering score slightly worse than multimodal ADriver-I. Longer illustrated predictions use other configurations. Appendix B reports that 16 rollouts yield 100 total frames with four references or 145 with one, so equal rollout counts do not mean equal generated lengths. The reference-frame ablation improves FVD as context grows, whereas the resolution ablation improves FID but worsens FVD. My reading is that these measurements describe different tradeoffs; none independently establishes reliable long-horizon control, and the missing evaluation details limit exact reproduction. E07E08E09E12E13E14E17
5.4 Training and inference
During training
LLaVA-NeXT-Video supplies nuScenes scene annotations; Stable Diffusion v1.4 is adapted on images for 50 epochs. Video training uses batch size 16, 30,000 steps, learning rate 10⁻⁴ and cosine scheduling, taking a reported 40 hours on four A100 GPUs. E06
Training noises known and ground-truth future states and combines action L1 loss with diffusion noise-prediction loss. Context is randomly dropped for classifier-free guidance. The text states optimization over all parameters but does not separately enumerate VAE/text-encoder freezing or the context-drop probability. E05E17
During inference
Start with reference imagery and controls, predict future actions and latent actions, then generate and decode the conditioned video. No ground-truth future video is available at inference. Appendix B reports 16 rollouts yielding 100 total frames with four references or 145 with one; these are illustrated generation lengths, not measured driving-success horizons. E03E05E14
5.5 Implementation flow
- Encode observations
A VAE maps observation o_t to state s_t and decodes predicted states into frames. Section 3.1 formulates the single-observation case; the context module and experiments also use multiple reference frames. E02E03E05E07
- Forecast controls in feature space
An embedding and MLP extract known-action features h. A transform projects visual features into action space, and another MLP predicts future features. Concatenating current and predicted features produces the latent-action sequence supplied to the transition model; predicted features also decode into low-level controls. E04
- Condition the video transition
The spatiotemporal U-Net receives latent actions through temporal cross-attention. Separately, VAE observation features form state context by repeating the last available state; zero-initialized convolutions produce scale and shift terms for multiple U-Net blocks. These are distinct action and visual conditioning routes. E05
- Decode and refresh the rollout
The transition model predicts future latent states, which the VAE decodes. Generated reference frames and actions then replace the previous references for another rollout. Feedback here comes from predictions, without a demonstrated vehicle/environment execution loop. E03E14
6. Experiments & results
ProphetDWM predicts driving actions and videos from observations and a short supplied action sequence. An MLP forecasts actions and exposes latent action features to a diffusion video model; a separate visual-context pathway anchors generation. Joint training improves reported nuScenes video metrics, while action prediction improves average L1 error but does not lead on steering alone. The evidence concerns prediction and generated rollouts, with deployment and evaluation-protocol gaps remaining [E03–E10, E17].
6.1 Read the original evidence
Table 2. ProphetDWM has the lowest listed video-distribution distances under the paper's comparison. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the downward arrows: smaller FID and FVD are better in the reported evaluation. The last row contains ProphetDWM, and the condition column distinguishes first-image inputs from HD-map and 3D-box priors used by several baselines. Appendix A identifies the quantitative configuration as 192×384, eight-frame sequences, one reference and 2-fps keyframes. Compare this table with its stated baseline selection: the authors exclude ADriver-I's different frame protocol and the extra-data GenAD/Vista settings, while retaining a row labeled GenAD-nus. These scores belong to the quantitative configuration, not automatically to the higher-resolution videos elsewhere in the paper. E03E07E08E17
What it supports. The reported 6.9 FID and 190.5 FVD are lower than every listed competitor. For reference, GenAD-nus reports 15.4/244.0 and Drive-WM 23.3/228.5. This establishes a favorable reported distributional comparison for this table, without measuring the physical correctness of each predicted future or the success of executed controls.
Where the evidence stops. The first-image label is an incomplete input inventory: the method also consumes actions. Baseline conditions differ, and exact evaluation splits, sample counts and uncertainty are missing. Treat the ranking as the paper's reported comparison, not a fully controlled independent replication.
Table 3. Lower average error comes from improved speed prediction, with a small steering tradeoff. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read speed and steering separately before looking at the average. The dagger note marks action-only baselines; the unmarked ADriver-I row uses multimodal inputs and is the stronger direct comparison in this table. ProphetDWM is the final row. Its bold speed and average entries identify the best listed values, while the bold steering entry belongs to ADriver-I. Keep the DriveDreamer dashes as missing values rather than zeros or estimates; the source table spells this baseline DriverDreamer. Section 4.3 motivates averaging the two signals by their common projection from features, but does not provide the normalization needed to interpret that average physically. E09E17
What it supports. ProphetDWM reports speed L1 0.090 versus 0.103 for multimodal ADriver-I, and average L1 0.0915 versus 0.0975. Steering moves slightly in the opposite direction: 0.093 versus 0.092. The supported claim is lower speed and average error, with comparable rather than best steering accuracy.
Where the evidence stops. These are prediction errors, not driving-success rates. Units, signal normalization, precise forecast aggregation and the evaluation split are unspecified in the supplied paper. Without that information, the numerical average cannot be assigned a clear physical scale.
Figure A10. Different low-level controls lead to visibly different generated paths from the same scene. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the common reference stack at the left into the upper Go row and the lower Left row. Time labels run across both branches, making the changing road perspective the main comparison. Appendix C states that speed and steering values implement these control choices; the words in the figure name the resulting scenarios rather than establishing a natural-language control interface. The lower branch shows the generated viewpoint entering a different street direction, while the upper branch continues forward. Read this together with the architecture: provided controls inform the action representation that conditions future video, even though controls need not span the entire generated sequence. E03E05E15
What it supports. The example shows the intended counterfactual behavior qualitatively: changing the supplied low-level action sequence can change the generated future while starting from common reference imagery. It is useful evidence that control information is expressed visually. It does not provide a population-level measure of responsiveness or calibration.
Where the evidence stops. The figure omits the actual control values, sampling seeds and a quantitative dynamics comparison. Scene divergence could also depend on generation randomness; a controlled intervention test would hold that randomness fixed before attributing all differences to the actions.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Single-view driving video generation nuScenes; setting 1 uses 192×384, sequence length 8, one reference frame and 2-fps keyframes. Evaluation split and sample count are not stated. | Reported FID 6.9; FVD 190.5. FID ↓; FVD ↓ | Table 2: GenAD-nus 15.4/244.0; Drive-WM 23.3/228.5 (FID/FVD). Conditioning inputs differ. Best listed scores. The condition column says first image, although the method also uses actions. ADriver-I and extra-data GenAD/Vista settings are excluded by the authors; this is not a universal matched-protocol ranking. E07E08E17 |
| Driving speed and steering prediction nuScenes, following ADriver-I's L1 reporting; exact split, forecast aggregation and physical units/normalization are not given. | Reported 0.090; 0.093; 0.0915, respectively. Speed L1 ↓; steering-angle L1 ↓; average L1 ↓ | Multimodal ADriver-I: 0.103/0.092/0.0975. Action-only ADriver-I: 0.122/0.101/0.1115. Improves speed and average error; steering is slightly worse than multimodal ADriver-I. Missing DriveDreamer steering/average cells remain unreported. The cross-signal average's physical meaning is unclear without normalization. E09E17 |
| Latent-action joint-training ablation nuScenes video prediction; Table 4 compares no latent action, frozen action module and joint training. | Joint training: 6.9/191. FID ↓; FVD ↓ | No latent action: 7.2/265; frozen action module: 7.1/227. Supports useful latent conditioning and further benefit from joint optimization. Preserve Table 4's FVD 191 separately from Table 2's 190.5; no rounding rule or uncertainty is supplied. E10E08 |
| Reference-frame-count ablation nuScenes; Table 5 varies training reference count. | Four references: 7.1/182.5. FID ↓; FVD ↓ | One: 6.9/190.5; two: 6.9/188.3. More references improve reported FVD, while FID is not monotonically improved. This training comparison should not be equated with holding a rollout model fixed. E12 |
| Training-resolution ablation nuScenes; Table 6 compares 192×384 and 256×448 training. | 256×448: 6.2/238.0. FID ↓; FVD ↓ | 192×384: 6.9/190.5. Higher resolution improves FID but worsens FVD. The authors' explanation about harder motion learning is a hypothesis, not an isolated causal test. E13 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Both latent conditioning and joint optimization improve the reported video scores. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two checkmark columns as experimental switches. The first row has no latent-action conditioning. In the second row, latent actions are present but the action module is frozen, according to the accompanying ablation paragraph. The third row adds joint training. This order separates adding an action representation from letting it adapt with the video objective. Compare each metric vertically rather than interpreting a blank checkmark cell as missing data: here blanks indicate an absent component or training choice. All three rows report video quality; none reports the corresponding speed or steering error for the altered action module. E05E08E10
What it supports. FVD decreases from 265 without latent actions to 227 with a frozen action module and 191 with joint training. FID changes more modestly, from 7.2 to 7.1 to 6.9. These results support the conditioning pathway and its optimization, particularly through the reported video metric.
Where the evidence stops. Table 4 prints FVD 191; Table 2 prints 190.5, without explaining the difference. No uncertainty or matched action-error ablation is supplied. Better generated video alone cannot show that the jointly optimized controls also improved.
Figure 7. Observation features can enter throughout the U-Net instead of only at its input. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the shared reference stack on the left and follow the two labeled arrows. The upper branch uses the proposed multiscale fusion, whereas the lower branch concatenates context with noisy latent features at the input. Section 4.4 says the latter changes conv_in to eight channels and compares models at the same training step count. Section 3.3 explains the proposed route: context produces scale and shift terms through zero-initialized convolutions at U-Net blocks. Inspect the cones, barriers and road boundaries in the two output rows as the local qualitative evidence. The blue check and red cross are the authors' annotations. E05E10E11
What it supports. The upper samples preserve the roadwork layout more clearly than the lower samples. This is consistent with the authors' argument that a short observation pathway helps scene preservation during prediction. The result complements Table 4: one comparison concerns visual context, while the other concerns latent-action conditioning and joint training.
Where the evidence stops. The caption claims faster convergence, but this visual supplies no convergence curve or numerical rate. A same-step image comparison also does not establish equal compute or quantify performance across the full evaluation set.
7. Analysis & limitations
7.1 What the evidence leaves open
Action-control examples and long rollouts demonstrate selected generated futures. They do not quantify counterfactual physical accuracy, collision avoidance or closed-loop success. The qualitative comparison also uses different model resolutions. Claims of accurate long-term dynamics or absence of error accumulation exceed what selected images alone establish. E14E15E19
The fusion ablation shows a qualitative same-step comparison, with no learning curve or numerical convergence-rate measurement. Table 4 provides stronger quantitative evidence for joint training, but reports neither repeated seeds nor action accuracy under the alternative training strategies. E10E11
7.2 Questions for discussion
- Does joint optimization improve action prediction as well as FVD?
- How strongly do generated turns respond to low-level controls when diffusion randomness is held fixed?
- What action scaling makes average speed/steering L1 interpretable?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs aligned nuScenes images/actions, generated scene annotations, the specified image-model initialization and separate quantitative/qualitative configurations. The source omits exact splits, evaluation sample counts, action scaling, MLP dimensions, loss weights, optimizer details, sampling steps/guidance and seed variability; code availability is not established here. E04E05E06E07E17
Resolve two implementation ambiguities before claiming fidelity: Equation (8) prints a noise multiplier of (1−bar α_τ) without a square root; Appendix A states both 4 fps and two non-keyframes between 2-fps keyframes, with the latter pattern also drawn in Figure A8. Do not silently substitute a schedule or infer exact timestamps. E16E18
Proposed checks: repeat frozen-versus-joint training with matched data, compute and seeds while measuring both modalities; then vary supplied steering/speed with fixed references and diffusion noise, measuring whether video motion follows the intervention. These test the claimed coupling rather than visual plausibility alone. E05E10E15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does joint training help both modalities?
Reader-proposed experiment: start from a common action checkpoint and compare a frozen action module, full joint training, and joint training with video gradients detached at the latent-action interface. Keep the video initialization, declared scene-disjoint split, action scaling, batch size and 30,000-step budget fixed; repeat across at least three seeds. Use the paper's 192×384, eight-frame quantitative configuration and report FID/FVD plus separate speed/steering L1. An FVD gain without an action-error gain would narrow the joint-prediction claim. If full joint training does not beat the detached condition beyond run variability, the benefit cannot confidently be assigned to video gradients improving the action representation. E04E05E06E07E10E17
Check 2: Does the generated turn follow the control intervention?
Reader-proposed experiment: for each held-out reference scene, generate paired straight/left futures using identical diffusion noise and sampling settings. Hold supplied speeds fixed and vary only steering within a documented data-supported range. Add a control that reuses the same latent-action conditioning despite changing the supplied steering. Measure predicted steering and a declared visual yaw/trajectory proxy at matched horizons, then repeat over scenes and seeds. Consistent signed visual changes in the responsive condition, but not the fixed-conditioning control, would support the claimed pathway. No directional response, or changes no larger than seed variation, would weaken the interpretation of Figure A10; a visual proxy still would not prove physical driving safety. E03E04E05E15E17
8.3 Reading coverage
Visual audit: All 14 supplied PDF pages were rendered and visually inspected, alongside complete reading of all four text chunks. The pass covered the title/version/affiliations on page 1; Figure 1 and Table 1 on page 2; method equations and Figures 2–3 on pages 3–5; Figures 4–6 and Table 2 on pages 6–7; Figure 7 and Tables 3–6 on page 8; the conclusion on page 9; Appendix A–D and Table A7 on page 10; Figures A8–A9 on page 11; Figure A10 on page 12; Figure A11 on page 13; and references through page 14. Supplement pages restart printed numbering at 1 on PDF page 10; all asset links and inspectedPages use PDF page indices. All six final original crops were separately viewed, with headers, branch labels and the Table 3 dagger note retained. Architecture arrows and training labels were checked against Equations (2)–(4) and Section 3.3; interpretation of the noisy branch follows the text. The printed Equation (8) and Appendix A/Figure A8 timing ambiguities remain explicit in the base report. No separate supplement, executable implementation or additional edition was inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1. Introduction
- 2. Related Work
- 2.1. World Model
- 2.2. Video Generation
- 3. Method
- 3.1. World Model Learning
- 3.2. Light-weight Action Model
- 3.3. Video Transition Model
- 4. Experiment
- 4.1. Setting
- 4.2. Qualitative Results
- 4.3. Quantitative Evaluation
- 4.4. Ablation Study
- 5. Conclusion
- A. Experiment details
- B. Quality comparison
- C. Action Control
- D. More results
- References
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction limitation was addressed by visually inspecting all 14 supplied PDF pages and every final crop. Embedded supplementary material on PDF pages 10–14 was read; no separate supplement was supplied.
- Identity note: PDF page 1 establishes arXiv:2505.18650v1 [cs.CV], 24 May 2025, with the exact catalog title and both catalog authors. This bundle reviews that version; no other edition or revision history was supplied or compared.
- Code, model weights and external links were not inspected; no experiment was reproduced. All four supplied text chunks were read completely.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
E01PDF p. 1, title/author block and arXiv margin
The title is ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos; authors are Xiaodong Wang and Peixi Peng, both affiliated with Peking University and Peng Cheng Laboratory. The artifact is arXiv:2505.18650v1 [cs.CV], 24 May 2025.
Go to primary source ↓E02PDF pp. 1–3, Introduction and §3.1, including Eq. (1) and footnote 1
The formulation uses observations and continuous driving actions, excludes reward, assumes supplied-action horizon η<ΔT, and encodes/decodes images with a VAE. Section 3.1 presents the current-observation case.
Go to primary source ↓E03PDF p. 4, Figure 2, §3.1, Eqs. (2)–(5)
Separate q_θ action and q_φ video modules share latent-action conditioning. The action suffix is forecast beyond known controls, state predictions are decoded into video, and outputs can be rolled out iteratively. The noisy-data branch is marked training only.
Go to primary source ↓E04PDF pp. 4–5, §3.2, Figure 3 and Eqs. (6)–(7)
Action embedding/MLPs, transformed visual features and temporal concatenation construct known/future latent actions. Future features project to controls supervised with L1 loss. Layer widths and loss-weight values are not listed.
Go to primary source ↓E05PDF p. 5, §3.3, Eqs. (8)–(11)
The video U-Net predicts noise using latent actions via temporal cross-attention and visual state context via zero-initialized multiscale scale/shift residual fusion. The last observation state is repeated, context is randomly dropped, and the overall objective is action loss plus video loss.
Go to primary source ↓E06PDF p. 5, §3.3 opening; PDF p. 10 (supplement p. 1), Appendix A, training paragraph
LLaVA-NeXT-Video annotates nuScenes scenes. Stable Diffusion v1.4 is image-finetuned for 50 epochs; video training uses batch 16, 30k steps, learning rate 1e-4, cosine scheduling and 40 hours on four A100 GPUs.
Go to primary source ↓E07PDF pp. 5–6, §4.1; PDF p. 10 (supplement p. 1), Table A7 and Appendix A
Quantitative setting 1 uses 192×384, one reference, eight-frame sequences and 2 fps. Qualitative settings use 256×448, one/four references, sequence length 10 and stated 4 fps. FID/FVD and speed/steering L1 are the reported metrics.
Go to primary source ↓E08PDF p. 7, Table 2, all rows/Condition/FID/FVD columns, and §4.3 video-generation paragraph
Ours reports 6.9/190.5, GenAD-nus 15.4/244.0 and Drive-WM 23.3/228.5. Table conditions vary; Ours is labeled first image. The text excludes ADriver-I's different frame protocol and extra-data GenAD/Vista results.
Go to primary source ↓E09PDF p. 8, Table 3, dagger note and all metric columns; PDF pp. 6–7, §§4.1 and 4.3 action-prediction text
Ours speed/steering/average L1 is 0.090/0.093/0.0915; multimodal ADriver-I is 0.103/0.092/0.0975 and action-only ADriver-I 0.122/0.101/0.1115. DriveDreamer has only speed values, 0.150 action-only and 0.140 multimodal; other cells are dashes. The table spells this baseline DriverDreamer.
Go to primary source ↓E10PDF p. 8, §4.4 Benefits of joint training and Table 4, all rows
Without latent actions FID/FVD is 7.2/265; with a frozen action module it is 7.1/227; with joint training it is 6.9/191. No action-error ablation or uncertainty accompanies this table.
Go to primary source ↓E11PDF p. 8, Figure 7/caption and §4.4 Multi-scale fusion vs. Concatenation
The comparison uses equal training steps and modifies conv_in to eight channels for input concatenation. The displayed fusion result retains clearer scene structure. The caption claims faster convergence, but supplies example images rather than a convergence curve.
Go to primary source ↓E12PDF p. 8, Table 5 and §4.4 Different reference frame
One/two/four training reference frames yield FID 6.9/6.9/7.1 and FVD 190.5/188.3/182.5.
Go to primary source ↓E13PDF p. 8, Table 6 and §4.4 Different resolution
192×384 training gives FID/FVD 6.9/190.5; 256×448 gives 6.2/238.0. Harder motion learning is offered as a possible explanation for worse FVD.
Go to primary source ↓E14PDF p. 6, Figure 4 and §4.2; p. 7, Figure 6; p. 10 (supplement p. 1), Appendices A/B/D; p. 11, Figure A9; p. 13, Figure A11
Figure 4 uses four references and two actions. References and actions are refreshed after rollouts. Sixteen rollouts produce illustrated totals of 100 frames with four references or 145 with one. Long sequences and associated action plots are examples, not aggregate closed-loop evaluations.
Go to primary source ↓E15PDF p. 10 (supplement p. 1), Appendix C; PDF p. 12 (supplement p. 3), Figure A10/caption
Low-level speed and steering controls are used for straight/left examples. Figure A10 branches from a shared four-frame reference stack into Go and Left rows through T=50; it supplies no numerical control values or aggregate intervention metric.
Go to primary source ↓E16PDF p. 10 (supplement p. 1), Appendix A and Table A7; PDF p. 11 (supplement p. 2), Figure A8/caption
Appendix A states 2-fps keyframes for setting 1 and 4-fps qualitative data with two non-keyframes between keyframes. Figure A8 also depicts two intermediate images per keyframe interval and identifies inputs in orange, with nine subsequent frames and two actions predicted.
Go to primary source ↓E17PDF pp. 4–5, §§3.2–3.3; pp. 5–8, §§4.1–4.4 and Tables 2–6; p. 10 (supplement p. 1), Appendix A
These method/experiment descriptions provide selected architecture and training settings but do not specify exact train/evaluation splits, evaluation counts, action units/normalization and aggregation, loss weights, complete module freezing, sampling configuration or repeated-run uncertainty.
Go to primary source ↓E18PDF p. 5, §3.3, Eq. (8) and following symbol description
The printed noisy-state equation uses sqrt(bar α_τ) on the clean state and (1−bar α_τ) on ε, with no square root on the noise coefficient. The source does not resolve whether this is an intended schedule or a typographical error.
Go to primary source ↓E19PDF p. 6, Figure 5/caption and §4.2 comparison; PDF p. 7, continuation of qualitative comparison
The qualitative comparison uses CogVideoX-2b-nus at 480×720, Vista at 576×1024 and ProphetDWM at 256×448. Misalignment/corruption are annotated in selected images; the prose argues against error accumulation without a corresponding long-horizon aggregate error curve.
Go to primary source ↓8.5 Primary sources
ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos ↗
PDF · 6,826 extracted words
Source fingerprint
fd06f99b122c6a42fc96c38a41658d4acd81c53f309603cbfd23b86824f045a2